It's not that there's a limit, but know that when you handle them, it adds to their stress level. Keeping them stress as low as possible is key. Also know that they're cold blooded animals. So if you leave them out laying on your bed or desk, know that they won't be able to keep warm for long.
Leopard geckos generally tolerate handling, but they don't necessarily like it. So just remember that, you have to just judge things on your own. If your gecko starts to move around fast like it wants to get somewhere, that's likely a good time to put the gecko back in the enclosure.