My top interest is AI safety, followed by reinforcement learning. My professional background is in software engineering, computer science, machine learning. I have degrees in electrical engineering, liberal arts, and public policy. I currently live in the Washington, DC metro area; before that, I lived in Berkeley for about five years.
The audio reading of this post [1] mistakenly uses the word hexagon instead of pentagon; e.g. "Network 1 is a hexagon. Enclosed in the hexagon is a five-pointed star".
[1] [RSS feed](https://intelligence.org/podcasts/raz); various podcast sources and audiobooks can be found [here](https://intelligence.org/rationality-ai-zombies/)
I'm not so sure.
I would expect that a qualified, well-regarded leader is necessary, but I'm not confident it is sufficient. Other factors might dominate, such as: budget, sustained attention from higher-level political leaders, quality and quantity of supporting staff, project scoping, and exogenous factors (e.g. AI progress moving in a way that shifts how NIST wants to address the issue).
What are the most reliable signals for NIST producing useful work, particularly in a relatively new field? What does history show us? What kind of patterns do we find when NIST engages with: (a) academia; (b) industry; (c) the executive branch?
Another failure mode -- perhaps the elephant in the room from a governance perspective -- is national interests conflicting with humanity's interests. For example, actions done in the national interest of the US may ratchet up international competition (instead of collaboration).
Even if one puts aside short-term political disagreements, what passes for serious analysis around US national security seems rather limited in terms of (a) time horizon and (b) risk mitigation. Examples abound: e.g. support of one dictator until he becomes problematic, then switching support and/or spending massively to deal with the aftermath.
Even with sincere actors pursuing smart goals (such as long-term global stability), how can a nation with significant leadership shifts every 4 to 8 years hope to ensure a consistent long-term strategy? This question suggests that an instrumental goal for AI safety involves promoting institutions and mechanisms that promote long-term governance.
One failure mode could be a perception that the USG's support of evals is "enough" for now. Under such a perception, some leaders might relax their efforts in promoting all approaches towards AI safety.
perhaps I should apply Cantor’s Diagonal Argument to my clever construction, and of course it found a counterexample—the binary number (. . . 1111), which does not correspond to any finite whole number.
I’m not following despite having recently reviewed Cantor’s Diagonal Argument. I can imagine constructing a matrix such that the diagonal is all ones… but I don’t see how this connects up to the counterexample claim above.
Also, why worry that an infinite binary representation (of any kind) doesn’t correspond to a finite whole number? I suspect I’m missing something here. A little help please to help close this inferential distance?
How to interpret the comment above? Is it suggesting that EY's behavior was pompous? (As of this writing, the commenter only made one comment, this one, and does not seem to be around LessWrong at this time.) My take: >60% likely. Going "one level up", I would expect a majority of readers would at least wonder.
EY views other people’s irrationality as his problem, and it seems to me this discussion demonstrates a sincere effort to engage with someone he perceived as irrational. The conversation was respectful; as it progressed, each person clarified what they meant, and they ended with a handshake. If I were there at the outset of the conversation, I would not have expected this good of an outcome. (Updated on 2024-May-1)
Regarding the cost of a making an incorrect probability estimate, “Overconfidence is just as bad as underconfidence.” is not generally true. In binary classification contexts, one leads to more false positives and another to more false negatives. The costs of each are not equal in general for real world situations.
The author may simply mean that both are incorrect; this I accept.
My point is more than pedantic; there are too many examples of machine learning systems failing to recognize different misclassification costs.
For Hopfield networks in general, convergence is not guaranteed. See [1] for convergence properties.
[1] J. Bruck, “On the convergence properties of the Hopfield model,” Proc. IEEE, vol. 78, no. 10, pp. 1579–1585, Oct. 1990, doi: 10.1109/5.58341.