I think it's less about them being afraid a guy is going to find them attractive and more about what being attractive to a gay man means to them. Straight men who have that concern think that if a gay man hits on them, they aren't masculine enough or they're giving of some sort of homo-"signal." Which can be true and it can be just imagined.

On the other hand, it could be their degree of masculinity that makes the gay guy wanna make a move.

Gay guys like me are interested in a completely masculine, brooding hunk of Brawny-monster truck and chopper bike-full-sleeve tattooed-Guinness drinking-wall-punching-ax-weilding-pissing-contest-winning-"I don't give a fuck if you suck my cock, just don't look me in the eye" Straight guy.

In my defense, I don't come onto guys at all. I just don't. However, I have had a guy find out that I was gay and say "Hey man, I don't swing that way...don't get any ideas." A simple "Please, you're not even in my league" put him straight in check.