Most rechargeable batteries are lithium ion. The faster you charge them, the more you take off their lifespan. It is 'safe' to charge at up to 2C (where C is the capacity of the battery) but generally preferred to charge at 0.5C or no more than 1C to avoid degrading the battery faster.

What that means is that any lithium ion battery can be charged to almost full (at 1C) in an hour (or an hour and a half allowing for the final slow trickle charge), and there's no need for it to ever take more than three hours (at 0.5C inc trickle charge).

Bigger batteries often take longer because the charger is not as fast as it could be given the capacity of the battery. That won't apply to sex toys which generally have very small batteries. So the electronics may be faulty or the contacts poor.

FWIW lithium ion batteries are generally best charged to around 80% and not allowed to fall below 20%. So if you're having trouble charging it to full, that might be a good thing in terms of how long the toy will last before its battery reaches the end of its lifespan. But it depends how often you use it and how much charge it can take in between times.

Very long charge times are unlikely to be an intentional design feature so if it bothers you, contact the manufacturer.