This would require you to sample from GPT during training. If you want a sentence with 500 words you need to evaluate GPT 500 times. As a result, it would slow down training 500 times. The clever thing with GPT (and other autoregressive models) is that they circumvent sampling during training!
That might work. Maybe have the adversarial network try to distinguish GPT-3 text from human text? That said, GPT-3 is already trying to predict humanlike text continuations, so there's a decent chance that having a separate GAN layer wouldn't help. It's probably worth doing the experiment though; traditional GANs work by improving the discriminator as well as the desired categorizer, so there's a chance it could work here too.
You might find this interesting:
https://www.gwern.net/GPT-2-preference-learning#bradley-terry-preference-learning
I am wondering if anyone tried to combine GPT-3 with GAN, basically trying to train network that would feed GPT-3 questions and then judge responses as correct/incorrect, thus providing GPT-3 opportunity to improve.
Does my question even make sense or I am far off base?