Here is what I understand based on what I've read and what I've been told by a very reputable and knowledgeable herp vet in my area:
Most reptiles, even captive bred ones, carry a very minimal parasite load that's naturally occurring and usually poses no risk or harm to the reptile, even when the reptile is stressed. Bearded dragons, for example, have a natural coccidia load that usually has no impact on their health, provided they are kept in proper conditions. In many cases, the minimal parasite load is beneficial and helps with digestion and other bodily processes.
Other reptiles, particularly wild caught ones, almost always carry higher parasite loads which, if the animal lives in a stress-free environment and its enclosure is regularly cleaned and they are provided with a constant source of clean food and water, will not negatively impact them either. In some instances, they can actually overcome the higher parasite load on their own through the passing of eggs via fecal matter that's promptly cleaned up so they can't re-infect themselves.
However, the danger with wild caught reptiles is that adjusting to captivity is often harder than it is for captive bred reptiles, so that in itself leads to stress. Stress exacerbates underlying parasites, causing them to multiply and potentially overload the system. As parasites are usually transferred via fecal matter, even miniscule amounts of fecal matter in the enclosure can re-infect the reptile, which also contributes to system overload. The system overload is where the danger lies, even with pinworms which can kill your reptile in large numbers.
I have several wild caught reptiles, and out of necessity for some and curiosity for others, I've had them all tested.
My water dragon (and previous dragon) were tested for internal parasites because parasites are the number one killer of new dragon acquisitions. They stress very easily, can re-infect themselves quite well because they defecate in their water, and are generally a more fragile species, especially when young. It's imperative to catch and treat parasites early with this species. In both cases, both of my dragons had heavy parasite loads (one unknown, one pinworms), with one showing symptoms, and the other seeming to be unaffected. Unfortunately, my first one (the symptomatic one, with unknown parasites) succumbed to them because she had numerous health issues at the time (being a rescue) and she couldn't be treated for everything at once due to the potency of the medicine being administered. My current dragon was successfully treated, and she hasn't had issues with them since.
My Cave gecko was also tested. He tested negative twice, which means his parasite load was likely too small/miniscule to detect.
Finally, my Red-Eyed croc skink was tested and found to have a very minimal parasite load (pinworms). My vet said that administering a simple dewormer would be fine if I chose in order to reduce the numbers, but that it wasn't necessary because the parasites likely wouldn't pose any problems, provided I am keeping him in proper conditions and he's not constantly exposed to stress. I've had him for over a year now (was purchased from a previous owner who'd had him for over a year as well) and have never had any issues with him. In this case, the cons of taking him to the vet outweighed the pros - it's a long drive to the vet, which would stress him out, and as mentioned, medicines all carry inherent risks.
I think the choice of treating parasites is ultimately up to you, and is largely dependent on the species. Some species are more sensitive to parasites than others, and it's important to distinguish the ones that are. As well, when you have a wild caught animal that hasn't been tested/treated for parasites, it is important you have a system in place to monitor their eating/behavior/weight to ensure you catch any early symptoms of a parasite overload so they can be treated. You also have to ensure strict hygiene practices when interacting with your different reptiles to ensure you don't pass on parasites that don't affect your one, but may seriously affect another. It's often advised to handle/clean/feed all wild caught reptiles in your collection last to reduce the possibility of parasite transfer.
While I'm neither for nor against preventative deworming, it does need to be noted that there are risks with whatever you decide. I am strongly in favor of treating any reptiles with higher parasite loads though, as the consequences of not doing so far outweigh the potential risks of the medication. I would lean towards trusting the guidance/instinct of a reputable vet before making the decision though.