What makes Mc Gregor sexy isn't so much his body, but also his rather easy going personality and he is not ashamed of his body and very comfortable with his masculinity. He is the type that you could have a beer with in a bar.
This is different from some of the high end hollywood hunks (Pitt, Gere and others) who are in such a different class that you know you wouldn't even be good enough to talk to them. (in real life I don't know if they are like that, but they have that image).
Put it another way: one can easily relate to McGregor. But you can't relate to people like Brad Pitt and Richard Gere because you have nothing in common with them.