You know, I thought about this a bit when that "guy" got pregnant a couple weeks ago, and was all over the news...
I don't think that just because you get a boob job, take some hormones for a few months, and get your sex legally changed, your sex is actually changed. You gotta figure out what to do with your sexual organs before you figure that out.
Biologically, if you're able to produce the smaller gamete (sperm), you're a male... if you're able to produce the larger gamete (ovum), you're female... there's no middle-ground.
If you're able to produce an ovum, you're a female - despite whatever your driver's license might say.
If you're _permanently_ able to do neither (by choice or circumstance), and then want to give yourself some cosmetic internal and/or external surgery to suit your definition of happiness... then sure... pick the sex that goes with your current or surgically altered private parts... or mix things up, and call yourself what you wanna call yourself, i guess.
Generally, though, lacking gamete production capability - I'd think if you have a penis, you're a male... a vagina, a female... and if you have the combo platter (down under), hermaphrodite.
Or am I wrong?