This is a special post for quick takes by anaguma. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
Google/Deepmind has publicly advocated preserving CoT Faithfullness/Moniterability as long as possible. However, they are also leading the development of new architectures like Hope and Titans which would bypass this with continuous memory. I notice I am confused. Is the plan to develop these architectures and not deploy them? If so, why did they publish them?
Edit: Many people have pointed out correctly that Hope and Titans don't break CoT and it's a separate architectural improvement. Therefore I no longer endorse the above take. Thanks for correcting my confusion!
Yep, I think you're right, thanks for pointing this out.
2lilkim2025
I don't see how this breaks CoT. The memory module in Titans stores surprising information as it's encountered and then allows the transformer to look at it later on, but it doesn't synthesize new information. Strikes me as two entirely compatible augmentations of the transformer architecture.