Having a large pool of specific info on effective recall is a sign of mental health and quite useful. I've noticed various successful and charismatic commentators appearing to have talent in this area. It's possible that as well as being a sign of health it buffers brain abilities generally, that modern recall-augmenting tools will atrophy the native facility. It seems you can IQ test pretty high as long as you're capable of remembering what words mean but otherwise aren't guaranteed to have exceptional long-term memory capacity.

1. Looking forward to Elon's upcoming book, "IF I did it: confessions of a system prompter"
2. Elon is right about South Africa but foolish to patch it in prompt. Instead, think training data update weights.
3. This nano-scandal is similarly as embarrassing as the fake Path of Exile 2 account fiasco (which he did cop to eventually). Elon is doing such great works; why must he also micro-sin?

I'm unclear on whether the 'dimensionality' (complexity) component to be minimized needs revision from the naive 'number of nonzeros' (or continuous but similar zero-rewarded priors on parameters).

Either:

1. the simplest equivalent (by naive score) 'dimensonality' parameters are found by the optimization method, in which case what's the problem?
2. not. then either there's a canonicalization of the equivalent onto- parameters available that can be used at each step, or an adjustment to the complexity score that does a good job doing so, or we can't figure it out and we risk our optimization methods getting stuck in bad local grooves because of this.

Does this seem fair?

This appears to be a high-quality book report. Thanks. I didn't see anywhere the 'because' is demonstrated. Is it proved in the citations or do we just have 'plausibly because'?

Physics experiences in optimizing free energy have long inspired ML optimization uses. Did physicists playing with free energy lead to new optimization methods or is it just something people like to talk about?

This kind of reply is ridiculous and insulting.

We have good reason to suspect that biological intelligence, and hence human intelligence roughly follow similar scaling law patterns to what we observe in machine learning systems

No, we don't. Please state the reason(s) explicitly.

Google's production search is expensive to change, but I'm sure you're right that it is missing some obvious improvements in 'understanding' a la ChatGPT.

One valid excuse for low quality results is that Google's method is actively gamed (for obvious $ reasons) by people who probably have insider info.

IMO a fair comparison would require ChatGPT to do a better job presenting a list of URLs.

how is a discretized weight/activation set amenable to the usual gradient descent optimizers?

You have the profits from the AI tech (+ compute supporting it) vendors and you have the improvements to everyone's work from the AI. Presumably the improvements are more than the take by the AI sellers (esp. if open source tools are used). So it's not appropriate to say that a small "sells AI" industry equates to a small impact on GDP.

But yes, obviously GDP growth climbing to 20% annually and staying there even for 5 years is ridiculous unless you're a takeoff-believer.

You don't have to compute the rotation every time for the weight matrix.  You can compute it once. It's true that you have to actually rotate the input activations for every input but that's really trivial.