This is a point that has puzzled me for a long time: if human-level reasoning ability, at its essence, is also a form of "pattern matching," then there is still room for improvement in the Transformer architecture. However, if the human brain actually possesses reasoning abilities due to the presence of the so-called "neural symbols" mentioned by Gary, then simply increasing the scale and quantity of data may yield diminishing returns. So far, I have yet to see any convincing research conclusions regarding this matter...
This is a point that has puzzled me for a long time: if human-level reasoning ability, at its essence, is also a form of "pattern matching," then there is still room for improvement in the Transformer architecture. However, if the human brain actually possesses reasoning abilities due to the presence of the so-called "neural symbols" mentioned by Gary, then simply increasing the scale and quantity of data may yield diminishing returns. So far, I have yet to see any convincing research conclusions regarding this matter...