Well, there's also natural selection in other forms, and genetic drift. I mean, we all know body hair varies across human populations. (I've always used blood types as an example of genetic drift: There's absolutely no natural or sexual selection, but you still see different frequencies in different populations.)