I think many people experiment with creating different digital personas but with low effort, like "You are Elon Musk".
I personally often ask LLM to comment on my drafts as Yudkowsky and other well known LWers. What such answers lack is extreme unique insight which is often for real EY.
The essence of human genius is missed and this is exactly why we still don't have AGI.
Also, for really good EY model we may need more data about his internal thought stream and biographical details which only he can collect. It seems that he is not interested and even if he would, it would be time consuming (but he write quickly). One thousand pages of unedited thought stream may significantly improve the model.
IMO that's because it's not relatively easy to create a good replica of a person; LLMs fine-tuned to speak like a particular target retain LLM-standard confabulation, distractibility, inability to learn from experience, etc which will make them similarly ineffective at alignment research. I'd suggest looking into the AI Village for a better sense of how LLMs do at long-horizon tasks. (Also, I want to point out that inference is costly. The AI village, which only has four agents and only runs them for two hours a day, costs $3700 per month; hundreds of always-on agents would likely cost hundreds of times that much. This could be a good tradeoff if they were superhumanly effective alignment researchers, but I think current frontier LLMs are capable only of subhuman performance).
I agree with you on most points.
BTW, I'm running a digital replica of myself. The setup is as follows:
The answers are surprisingly good at times, reflecting non-trivial aspects of my mind.
From many experiments with the digital-me, I conclude that a similar setup for Yudkowksy could be useful even with today's models (assuming large-enough budgets).
There will be no genius-level insights in 2025, but he could automate a lot of routine alignment work, like evaluating models.
Given that models may become dramatically smarter in 2026-2027, the digital Yudkowksy may become dramatically more useful too.
I open-sourced the code:
https://github.com/Sideloading-Research/telegram_sideload
Yes, if MIRI spends a year on building as good model of Yudkowsky as possible, it can help in alignment and its measurable and doable thing. They can later ask that model about failure modes of other AIs and it will cry "Misaligned!"
These days, it's relatively easy to create a digital replica of a person.
You give the person's writings to a top LLM, and (with a clever prompt) the LLM starts thinking like the person. E.g. see our experiments on the topic.
Of course, it's far away from a proper mind uploading. But even in this limited form, it could be highly useful for AI alignment research:
Why no one is doing this?
Given the short timelines and the low likelihood of AI slowdown, this may be the only way to get alignment before AGI, by massively (OOMs) accelerating the alignment research.