I say go for it, of course. Others will have their own reactions for sure. I someone is put off by your choice of heart, then s/he'll just have to deal with it, it's not your responsibility to tiptoe around people who don't recognize a good thing (a happy couple) when they see it.

I (white boy) was with a black woman for almost two years. She is ten years my senior, and transsexual. It was my first time in an interracial relationship, and I had some time to get used to stares in public. But at the time I was proud, because I was so thrilled to spend time with her. We got ugly looks and muttered comments from people on the street, but such people can all go fuck themselves, am I right?

Don't let people who limit themselves to the 'right' kind of lover choose your lovers for you.