Getting a validation accuracy of 50% in a binary classification task isn't "surprisingly well". It means your model is as good as random guessing: if you flipped a coin, you would get the right answer half the time, too. Getting 0% validation accuracy would mean that you are always guessing wrong, and would get 100% accurate results by reversing your model's prediction. So, yes, just like the article says, the discriminator does not generalize.
No worries, was worth clarifying. I edited the post to link this comment thread.
Disclaimer: I just started reading about GANs, so am almost certainly missing some context here.
Something that surprised me from the BigGAN paper:
I'm not sure how to interpret this. The validation accuracy being close to 50% seems strange -- if the discriminator has 'memorized' the training set and has only seen training set vs generated images, why would it not guess close to 0% on things in the test set? Presumably they are both 1. not-memorized and 2. not optimized to fool the discriminator like generated images are. Maybe the post title is misleading, and we should think of this as "discriminators generalize surprisingly well despite also 'memorizing' the training data." (EDIT: See comment thread here for clarification)
Note that the discriminator has far fewer parameters than there are bytes to memorize, so it necessarily is performing some sort of (lossy) compression to do well on the training set. Could we think of the generator as succeeding by exploiting patterns in the discriminator's compression, which the discriminator then works to obfuscate? I would expect more obfuscation to put additional demands on the discriminator's capacity. Maybe good generator task performance then comes from defeating simpler compression schemes, and it so happens that simple compression schemes are exactly what our visual system and GAN metrics are measuring.
Does this indicate that datasets are still just too small? Later in the same paper, they train on the much larger JFT-300M dataset (as opposed to ImageNet above) and mention:
They don't mention whether this also increases discriminator generalization or decreases training set accuracy, which I'd be interested to know. I'd also be interested in connecting this story to mode collapse somehow.