I also think I might have few nerve endings, because I don't have much "feeling" on my penis, except the head.

I think most of the nerve endings are at the head actually. I've also heard that guys can lose some sensation by masturbating too often. I'd assume too much sex could cause the same thing, although I personally wouldn't know with either.