Nice work! This is important to do, but I want to argue this was squarely my prior, and argue that this is extremely unsurprising, and very natural behavior. Not just in the "wanting-good-number-go-up" sense like a great comment above, but in an even simpler sense: I find it to be parallel to "humans care about attaining sugar/sex/dopamine (score) regardless of whether it will be used in training (natural selection)".
If today was going to be reset at the end of the day and no resource changes carry over tomorrow, would humans still engage in score-seeking ... (read more)
If you are intelligent enough, you can deduce the laws of the universe from a surprisingly small amount of data. In the vein of your example, there is the story of Darwin deducing the existence of a moth with a long proboscis after seeing an orchid with a particular shape, and proving to be right. Perhaps papers from pre-2010 don't have the right models, but maybe they have enough information and data for a sufficiently intelligent being to piece together from them whatever is missing?
Nice work! This is important to do, but I want to argue this was squarely my prior, and argue that this is extremely unsurprising, and very natural behavior. Not just in the "wanting-good-number-go-up" sense like a great comment above, but in an even simpler sense: I find it to be parallel to "humans care about attaining sugar/sex/dopamine (score) regardless of whether it will be used in training (natural selection)".
If today was going to be reset at the end of the day and no resource changes carry over tomorrow, would humans still engage in score-seeking ... (read more)