They really haven't had a downside for me. They hugely enable my functionality. I've no desire to take more than I need. If I stop cold turkey, I return to my default unfocused state. That's everything.
As far as I have issues with stims, they're typically due to harmful decisions by the DEA (because LEO are a poor choice for making medical policy).
To folks outside the IT industry, I'm a specialist. To those inside, I'm a generalist. My job is solving whatever tech issue the customer has - including working with systems I have no experience with.