If you want to wear speedos then go for it. Hopefully, you have a body that goes well with them - ie no paunch, no forest on your shoulders, and something between your legs that's worth highlighting in wet elastane or nylon.
If you want some top quality all-nylon speedos, check out Caussies online. They're an Australian brand and come in black, navy and maroon. I purchased them direct from the manufacturer. Speedy delivery, reasonable prices and excellent pre and post-sales service.
As for comments, well, I mix it up a bit and have yet to provoke any adverse reaction. I generally wear board shorts to the beach but I wear speedos underneath and strip down to them when sunbathing. At the pool, where my chief focus is laps, I wear speedos for practical reasons - they dry quickly and they streamline the body. Speedos are always worn when playing water polo - it's a brutal sport - lots of nut-crunching from one's opponents and they try to drag you underwater. It's harder to grab a pair of speedos than it is any other form of swimwear.