Implementation details: https://www.gwern.net/TWDNE
And now a full guide to using StyleGAN: https://www.gwern.net/Faces
I now have poetry samples up using a retrained/finetuned version of GPT-2-small: https://www.gwern.net/RNN-metadata#finetuning-the-gpt-2-small-transformer-for-english-poetry-generation
It would be cool to try some style-matching between the text and images. Ultimately, having some "personality vector" which would be used both in image and text generation. (A very crude version could be to create a NN translator from the style space to word2vec space and include the words in the GPT prompts)
Amusingly, one of the sample texts contained the Japanese "一生える山の図の彽をふるほゥていしまうもようざないかった" which google translate renders as "I had no choice but to wear a grueling of a mountain picture that would last me" (no, it doesn't make sense in context).
Nothing makes sense when Google Translates tries Japanese. Although the fact that that is still better than pre-RNN shows you how much of an upgrade that was.