Subtitle: you're too poor to use AGI. WTF is WTP? In Econ 101, you learn about something called willingness to pay (WTP). WTP is the highest price at which you're willing to buy some good or service. As long as the equilibrium price is less than your WTP, you'll get...
I mean this is a summary of a talk I didn't see so I want to reserve some judgement. But at the same time I just can't imagine being in a situation where I don't want to use the word signal because the other person might think I'm talking about cell signal, so instead I bust out "kodo"
In general, I think we should be pretty wary of taking basic ideas and dressing them up in fancy words. It serves no purpose other than in-group signaling.
That makes sense. However, I do see this as being meaningfully different from evals or mech interp in that to make progress you really kind of want access to frontier models and lots of compute/tooling, so for individuals who want to prioritize this approach it might make sense to try to join a safety team at a lab first.
How are these ideas different from "signal" and "noise"?
Do you think building a new organization around this work would be more effective than implementing these ideas at a major lab?
I think this might be a result of o-series being trained in a non-chat setup for most of the CoT RL phase and then being hamfistedly finetuned right at the end so it can go into ChatGPT, which just makes them kind of bad at chat and so o3 gets confused when the conversation has a lot of turns. Retraining it to be good at multi-turn chat with separate reasoning traces would probably just be super expensive and not worth the squeeze. (this is just a guess)
No, that is not the definition of a prior. There are priors which imply an expected number of buses, and priors that don't. If you select a prior that doesn't, you can still get a meaningful posterior distribution even if that posterior distribution doesn't have a real-valued mean.
No, that's not the posterior distribution--clearly, the number of buses cannot be lower than 1546, but that distribution has material probability mass on low integers. I'm not quite sure how you got that equation.
But regardless, I think this shows where we disagree. That prior has mean 2... that's a pretty strong assumption about the distribution of n. If you want to avoid that kind of assumption, you can get posterior distributions but not a posterior expectation.
I'm not disagreeing with that categorically--for many priors the posterior distribution is well defined. But all of those priors carry information (in the information theoretical sense) about the number of buses. If you have an uninformative reference prior, your posterior distribution does not have a mean.
You can see the sketch of this proof if you consider the likelihoods of seeing the bus for any given n. If there are 1546 buses, there was a 1/1546 chance you saw this one. If there were 1547, there was a 1/1547 chance you saw this one. This is the harmonic series, which diverges. That divergence is the fundamental issue that's going to cause the mean... (read more)
The intuition is that if we both saw bus 1546, and you guessed that there were 1546 buses and I guessed that there were 1547, you would be a little more likely to be correct but I would almost certainly be closer to the real number.
The Bayesian update isn't generally well-defined because you get a divergent mean. Your implicit prior is 1/n which is an improper prior. This is fine for deriving a posterior median, which in this case happens to be about 3,100 buses, and a posterior distribution, which in this case is a truncated zeta distribution with s=2 and k=1546. But the posterior mean does not exist.
Subtitle: you're too poor to use AGI.
In Econ 101, you learn about something called willingness to pay (WTP). WTP is the highest price at which you're willing to buy some good or service. As long as the equilibrium price is less than your WTP, you'll get the good, otherwise you'll prefer to keep your money.
But there's a wrinkle: what if my demand isn't independent of yours? This happens all the time. My WTP for a social media platform will be higher if lots of other people are also using the platform. My WTP for a rare collectible will be higher if fewer other people have it.
That second situation creates an... (read 1233 more words →)
People should take the time to understand the ecosystem and economics of enterprise software. I expect that to be a major determinant of the incentive landscape around AGI and have found in conversations that people often don't really get it.
I think even if the model wasn't specifically trained for geolocation, it's a reasonable assumption that metadata from photos (which often includes geo data) somehow gets passed to the models and this created a huge annotated dataset of (geo, photo) pairs during training for stuff like searching Google images.