Cruise is also operating with a publicly (though, public waitlist) in a few cities: SF, Austin, Phoenix. Recently announced Miami and Nashville, too. I have access.
Edit: also Houston and Dallas. Also probably Atlanta and other locations on their jobs page

Thanks for taking the time to write out these reflections. 

I'm curious about your estimates for self driving cars in the next 5 years, would you take the same bet at 50:50 odds for a 2028 July date?

There is a general phenomenon in tech that has been expressed many times of people over-estimating the short-term consequences and under-estimating the longer term ones (e.g., "Amara's law").

I think that often it is possible to see that current technology is on track to achieve X, where X is widely perceived as the main obstacle for the real-world application Y. But once you solve X, you discover that there is a myriad of other "smaller" problems Z_1 , Z_2 , Z_3 that you need to resolve before you can actually deploy it for Y.

And of course, there is always a huge gap between demonstrating you solved X on some clean academic benchmark, vs. needing to do so "in the wild". This is particularly an issue in self-driving where errors can be literally deadly but arises in many other applications.

I do think that one lesson we can draw from self-driving is that there is a huge gap between full autonomy and "assistance" with human supervision. So, I would expect we would see AI be deployed as (increasingly sophisticated) "assistants' way before AI systems actually are able to function as "drop-in" replacements for current human jobs. This is part of the point I was making here. 

One IMO important thing that isn't mentioned here is scaling parameter count. Neural nets can be fairly straightforwardly improved simply by making them bigger. For LLMs and AGI, there's plenty of room to scale up, but for the neural nets that run on cars, there isn't. Tesla's self-driving hardware, for example, has to fit on a single chip and has to consume a small amount of energy (otherwise it'll impact the range of the car.) They cannot just add an OOM of parameters, much less three. 
 

related but tangential: Coning self driving vehicles as a form of urban protest

I think public concerns and protests may have an impact on the self-driving outcomes you're predicting. And since I could not find any indication in your article that you are considering such resistance, I felt it should be at least mentioned in passing.

Why no mention of the level 4 autonomous robobuggies from Starship. These buggies have been exponentially ramping up now for over 10 years and they can make various grocery deliveries without human oversight. Autonomous vehicles have arrived and they are navigating our urban landscapes! There have been many millions of uneventful trips to date. What I find surprising is that some sort of an oversized robobuggy has not been brought out that would allow a person to be transported by them. One could imagine for example, that patrons of bars who had too many drinks to drive home could be wheeled about in these buggies. This could already be done quite safely on sidewalks at fairly low speed. for those who have had a few too many speed might not be an overly important feature. Considering how many fatalities are involved with impaired drivers it surprises me that MADD has not been more vocal in advocating for such a solution. So, in a sense there is already widespread autonomous vehicles currently operating on American streets. Importantly these vehicles are helping us to reimagine what transport could be. Instead of thinking in terms of rushing from one place to another, people might embrace more of a slow travel mentality in which instead of steering their vehicles they could do more of what they want to like surf the internet, email, chat with GPT etc.. The end of the commute as we know it?   

Hi Paul.  I've reflected carefully on your post.  I have worked for several years on a SDC software infrastructure stack and have also spent a lot of time comparing the two situations.

Update: since commentators and downvoters demand numbers: I would say the odds of criticality are 90% by July 2033.   The remaining 10% is that there is a possibility of a future AI winter (investors get too impatient) and there is the possibility that revenue from AI services will not continue to scale.

I think you're badly wrong, again, and the consensus of experts are right, again.

First, let's examine your definition for transformative.  This may be the first major error:

(By which I mean: systems as economically impactful as a low-cost simulations of arbitrary human experts, which I think is enough to end life as we know it one way or the other.)

This is incorrect, and you're a world class expert in this domain.  

Transformative is a subclass of the problem of criticality.  Criticality, as you must know, means a system produces self gain larger than it's self losses.  For AGI, there are varying stages of criticality, which each do settle on an equlibria:

Investment criticality : This means that each AI system improvement or new product announcement or report of revenue causes more financial investment into AI than the industry as a whole burned in runway over that timestep.  

Equilibrium condition: either investors run out of money, globally, to invest or they perceive that each timestep the revenue gain is not worth the amount invested and choose to invest in other fields.  The former equilibrium case settles on trillions of dollars into AI and a steady ramp of revenue over time, the later is an AI crash, similar to the dotcom crash of 2000.

Economic Criticality: This means each timestep, AI systems are bringing in more revenue than the sum of costs  [amortized R&D, inference hardware costs, liability, regulatory compliance, ...]

Equilibrium condition: growth until there is no more marginal tasks an AI system can perform cheaper than a human being.  Assuming a large variety of powerful models and techniques, it means growth continues until all models and all techniques cannot enter any new niches.  The reason why this criticality is not exponential, while the next ones are, is because the marginal value gain for AI services drops with scale.  Notice how Microsoft charges just $30 a month for Copilot, which is obviously able to save far more than $30 worth of labor each month for the average office worker.  

Physical Criticality: This means AI systems, controlling robotics, have generalized manufacturing, mining, logistics, and complex system maintenance and assembly.  The majority, but not all, of labor to produce more of all of the inputs into an AI system can be produced by AI systems.  

Equilibrium condition: Exponential growth until the number of human workers on earth is again rate limiting.  If humans must still perform 5% of the tasks involved in the subdomain of "build things that are inputs into inference hardware, robotics", then the equilibria is when all humans willing, able to work on earth are doing those 5% of tasks.  

AGI criticality: True AGI can learn automatically to do any task that has clear and objective feedback.  All tasks involved in building computer chips, robotic parts (and all lower level feeder tasks and power generation and mining and logistics) have objective and measurable feedback.  Bolded because I think this is a key point and a key crux, you may not have realized this.  Many of your "expert" domain tasks do not get such feedback, or the feedback is unreliable.  For example an attorney who can argue 1 case in front of a jury every 6 months cannot reliably refine their policy based on win/loss because the feedback is so rare and depends on so many uncontrolled variables.

  AGI may still be unable to perform as well as the best experts in many domains.  This is not relevant.  It only has to perform well enough for machines controlled by the AI to collect more resources/build more of themselves than their cost.  

A worker pool of AI systems like this can be considerably subhuman across many domains, or rely heavily on using robotic manipulators that are each specialized for a task, being unable to control general purpose hands, relying heavily on superior precision and vision to complete tasks in a way different than how humans perform it.  They can make considerable mistakes, so long as the gain is positive - miswire chip fab equipment, dropped parts in the work area cause them to flush clean entire work areas, wasting all the raw materials - etc.  I am not saying the general robotic agents will be this inefficient, just that they could be.

Equilibrium Condition: exponential growth until exhaustion of usable elements in Sol.  Current consensus is earth's moon has a solid core, so all of it could potentially be mined for useful elements  A large part of Mars, it's moons, the asteroid belt, and Mercury are likely mineable.  Large areas of the earth via underground tunnel and ocean floor mining.  The Jovian moons.  Other parts of the solar system become more speculative but this is a natural consequence of machinery able to construct more of itself.

Crux : AGI criticality seems to fall short of your  requirement for "human experts" to be matched by artificial systems.  Conversely, if you invert the problem: AGI cannot control robots well, creating a need for billions of technician jobs, you do not achieve criticality, you are rate limited on several dimensions.  AI companies collect revenue more like consulting companies in such a world, and saturate when they cannot cheaply replace any more experts, or the remaining experts enjoy legal protection.  

Requirement to achieve full AGI criticality before 2033: You would need a foundation model trained on all the human manipulation you have the licenses for the video.  You would need a flexible, real time software stack, that generalizes to many kinds of robotic hardware and sensor stack.  You would need an "app store" license model where thousands of companies could contribute, instead of just 3, to the general pool of AI software, made intercompatible by using a base stack.  You would need there to not be hard legal roadblocks stopping progress.  You would need to automatically extend a large simulation of possible robotic tasks whenever surprising inputs are seen in the real world.

Amdahl's law applies to the above, so actually, probably this won't happen before 2033, but one of the lesser criticalities might.  We are already in the Investment criticality phase of this.  

Autonomous cars:  I had a lot of points here, but it's simple:
(1) an autonomous robo taxi must collect more revenue than the total costs, or it's subcritical, which is the situation now.  If it were critical, Waymo would raise as many billions as required and would be expanding into all cities in the USA and Europe at the same time.  (look at a ridesharing company's growth trajectory for a historical example of this)

(2) It's not very efficient to develop a realtime stack just for 1 form factor of autonomous car for 1 company.  Stacks need to be general.

(3) There are 2 companies allowed to contribute.  Anyone not an employee of Cruise or Waymo is not contributing anything towards autonomous car progress.  There's no cross licensing, and it's all closed source except for comma.ai.  This means only a small number of people are pushing the ball forward at all, and I'm pretty sure they each work serially on an improved version of their stack.  Waymo is not exploring 10 different versions of n+1 "Driver" agent using different strategies, but is putting everyone onto a single effort, which may be the wrong approach, where each mistake costs linear time.  Anyone from Waymo please correct me.  Cruise must be doing this as they have less money.