My take is that it boils down to increasing the probability of fertilization, primarily in our early eukaryotic ancestors, in which sexual reproduction first evolved.
On the level of gametes (the reproductive cells), it's about the difference between isogamy (gametes being the same) and anisogamy (involving a larger and a smaller gamete, which are then by definition are the female egg cell and male sperm cell). The egg cells provide almost everything the eventual zygote needs to develop, while the main role of the sperm cell is to transport a set of genes to an egg cell. As it turns out, the sperm cell can do this with much less volume, and therefore, it is more cost-effective to produce them as small as possible, but in much higher numbers compared to egg cells. Making them smaller also makes them more mobile. Both high numbers and mobility help increase the chance of fertilization, especially in species with external fertilization, as the first animals would have been. When the fertilization is done externally, the animals don't exactly "bear" offspring, so there is no tradeoff in that regard.
Once there was a distinction between the gametes, other adaptations would have made sense, such as specialised organs for the deployment of eggs/sperm and traits that allow potential mates to recognize each other. The same goes for increasingly complex behaviour to ensure that those eggs and sperm are deposited in the right place at the right time and that the offspring develops well (including courtship displays, competition, nesting, brooding, etc.), much of which is sex-specific. Of course, in some animals this ultimately led to internal fertilization, where the benefits of anisogamy are not so relevant anymore, but the animals in question would have relied on a variety of sex-specific adaptations to even get to that point, and evolution only marches forward. The existence of some species that are hermaphroditic now certainly proves that it is possible to evolve in that direction, but as you say, there are surely various hurdles with fixating such a state on a large scale.
Thanks for the encouragement.
I've come across your work, but have to admit that I've only read parts of it so far. I'm still in the process of catching up to the state of the art.
Probably too late, but I wanted to submit this:
https://www.lesserwrong.com/posts/8JQQLkqjTPka9mJ4K/belief-alignment
Yes, but the distinction between gametes had to evolve first from asexual, i.e., undifferentiated reproduction, while hermaphroditism requires some added complexities, as it has to combine both sexes in a way that works. And as long as divided sexes work well enough, there isn't much selection pressure to go that route.
As for why this is more common in plants, I'm intuitively guessing that there are a bunch of issues involved that I'm not familiar with either. But a major driver is probably that (land) plants are sessile and tend to require some space for themselves. The problem for plants with differentiated sexes is then that if their nearest neighbours happen to be of the same sex, the chance of their spores reaching those of the opposite sex is drastically lower than for hermaphodites, as the latter have potential partners in every neighbour.
This wouldn't generally be an issue in mobile animals, who have to find and approach their mates anyway.