Hi,
the battery should be at >4.1V when fully charged.
I guess there is something wrong with your charger module.
I am not familiar with the model you are using, mine have only one LED which changes color (orange/red when charging and blue or green when fully charged).
Looks to me like your...