Shortform Content

Falling birthrates is the climate change of the right:

  • Vaguely tribally valenced for no really good reason
  • Predicted outcomes range from "total social collapse, failed states" to "slightly lower economic growth"
  • People use it as an excuse to push drastic social and political changes when the real solutions are probably a lot simpler if you're even slightly creative

I did an ADHD test in Germany. They asked me questions at 1:30 and then said I have ADHD, and no further testing was required. If the interview had not been conclusive they would have done some other tests. They ask about symptoms like "Can you not sit still", "Do you forget appointments" and things like that.

The most interesting part was the preinterview part.

Scott writes here on how psychiatrists are the gatekeepers to Adderall:

Aren’t psychiatrists creepy wizards who can see through your deceptions? There are people like that. They’re call

... (read more)

How many real numbers can be defined?

On one hand, there are countably many definitions. Each definition can be written on computer in a text file; now take its binary form as a base-256 integer.

On the other hand, Cantor's diagonal argument applies here, too. I mean, for any countable list of definable real numbers, it provides a definition of a real number that is not included in the list.

Funny, isn't it?


Fractal Fuzz: making up for size

GPT-3 recognizes 50k possible tokens. For a 1000 token context window that means there are  possible prompts. Astronomically large. If we assume the output of a single run of gpt is 200 tokens then for each possible prompt there are  possible continuations. 

GPT-3 is probabilistic, defining for each possible prompt  () a distribution  on a set of size , in other words a  dimensional space. [1]

Mind-boggingly large... (read more)

Showing 3 of 7 replies (Click to show all)
3Alexander Gietelink Oldenziel1d
Obviously this is all speculation but maybe I'm saying that the universal approximation theorem implies that neural architectures are fractal in space of all distributtions (or some restricted subset thereof)? Curious what's your beef with universal approximation? Stone-weierstrass isn't quantitative - is that the reason? If true it suggest the fractal dimension (probably related to the information dimension I linked to above) may be important.
1Zach Furman21h
Oh I actually don't think this is speculation, if (big if) you satisfy the conditions for universal approximation then this is just true (specifically that the image of W is dense in function space). Like, for example, you can state Stone-Weierstrass as: for a Hausdorff space X, and the continuous functions under the sup norm C(X,R), the Banach subalgebra of polynomials is dense in C(X,R). In practice you'd only have a finite-dimensional subset of the polynomials, so this obviously can't hold exactly, but as you increase the size of the polynomials, they'll be more space-filling and the error bound will decrease. The problem is that the dimension of W required to achieve a given ϵ error bound grows exponentially with the dimension d of your underlying space X. For instance, if you assume that weights depend continuously on the target function, ϵ-approximating all Cn functions on [0,1]d with Sobolev norm ≤1 provably takes at least O(ϵ−d/n) parameters (DeVore et al.). This is a lower bound. So for any realistic d universal approximation is basically useless - the number of parameters required is enormous. Which makes sense because approximation by basis functions is basically the continuous version of a lookup table. Because neural networks actually work in practice, without requiring exponentially many parameters, this also tells you that the space of realistic target functions can't just be some generic function space (even with smoothness conditions), it has to have some non-generic properties to escape the lower bound.

Ooooo okay so this seems like it's directly pointing to the fractal story! Exciting!

Imagine if a magic spell was cast long ago, that made it so that rockets would never explode. Instead, whenever they would explode, a demon would intervene to hold the craft together, patch the problem, and keep it on course. But the demon would exact a price: Whichever humans were in the vicinity of the rocket lose their souls, and become possessed. The demons possessing them work towards the master plan of enslaving all humanity; therefore, they typically pretend that nothing has gone wrong and act normal, just like the human whose skin they wear would h... (read more)

I've had success with something: meal prepping a bunch of food and freezing it.

I want to write a blog post about it -- describing what I've done, discussing it, and recommending it as something that will quite likely be worthwhile for others as well -- but I don't think I'm ready. I did one round of prep that lasted three weeks or so and was a huge success for me, but I don't think that's quite enough "contact with reality". I think there's a risk that, after more "contact with reality", it proves to be not nearly as useful as it currently seems. So yeah, ... (read more)

Since there are basically no alignment plans/directions that I think are very likely to succeed, and adding "of course, this will most likely not solve alignment and then we all die, but it's still worth trying" to every sentence is low information and also actively bad for motivation, I've basically recalibrated my enthusiasm to be centered around "does this at least try to solve a substantial part of the real problem as I see it". For me at least this is the most productive mindset for me to be in, but I'm slightly worried people might confuse this for ... (read more)

SLT and phase transitions

The morphogenetic SLT story says that during training the Bayesian posterior concentrates around a series of subspaces  with rlcts   and losses . As the size of the data sample  is scaled the Bayesian posterior makes transitions  trading off higher complexity (higher ) for better accuracy (lower loss ).

This is the radical new framework of SLT: phase transitions happen i... (read more)

Alignment by Simulation?

I've heard this alignment plan that is a variation of 'simulate top alignment researchers' with an LLM. Usually the poor alignment researcher in question is Paul. 

This strikes me as deeply unserious and I am confused why it is having so much traction. 

That AI-assisted alignment is coming (indeed, is already here!) is undeniable. But even somewhat accurately simulating a human from textdata is a crazy sci-fi ability, probably not even physically possible. It seems to ascribe nearly magical abilities to LLMs. 

Predicting... (read more)

There should be a PDF version of Ajeya Cotra's BioAnchors report on Arxiv. Having it only as a Google Drive folder ( makes it very hard to find and cite.

I created one for Cotra 2020 and Davidson 2023. Hope they're correct.

Nice, thanks! Arxiv would still be good for searchability but maybe the authors have to do that

Most justice systems seem to punish theft on a log scale. I'm not big on capital punishment, but it is actually bizarre that you can misplace a billion dollars of client funds and escape the reaper in a state where that's done fairly regularly. The law seems to be saying: "don't steal, but if you do, think bigger."

Showing 3 of 6 replies (Click to show all)

Hm, I hadn't thought about it that way. I was just thinking that the goal of the fine is some combination of 1) punitive and 2) deterrent, and neither of those goals are accomplished if you fine Bill Gates $200. But yeah, I guess if you make the fine large enough such that the state is ambivalent, maybe it all works out.

Theft of any amount over a hundred or so dollars is evil and needs to be punished. Let's say you punish theft of $100 by a weekend in jail. Extrapolate that on a linear scale and you'll have criminals who non-violently stole $20,000 doing more than double the jail time that a criminal who cold-cocked a stranger and broke his jaw would get. Doesn't really make sense.
This isn't the source of the trend; the sentencing guidelines for fraud are actually literally, explicitly logarithmic. The government recommends directly that sentences follow a curve of 2x price --> 2 more years. There seems to be a MAX_PUNISHMENT in the justice system (we don't devolve into torture, etc.), which is reasonable. But with things like armed robbery you would get convicted for each individual count, not on a log scale. This is (I suspect) a very common strategy among even regular criminals. You can think of it like a trade between law enforcement and gangs; the government gets their clearances and avoids the potential embarassment of a partially-solved case, and the serial killers send only the John Wayne Gacy to jail.

Since Raemon's Thinking Physics exercise I've been toying with writing physics puzzles along those lines. (For fun, not because I'm aiming to write better exercise candidates.) If you assume an undergrad-level background and expand to modern physics and engineering there are interesting places you can go. I think a lot about noise and measurement, so that's where my mind has been. Maybe some baseline questions could look like the below? Curious to hear anyone's thoughts.

Pushing a thermal oscillator

You're standing at one end of a grocery aisle. In your cart... (read more)

Molecular electromechanical switch

You've attached one end of a conductive molecule to an electrode. If the molecule bends by a certain distance  at the other end, it touches another electrode, closing an electrical circuit. (You also have a third electrode where you can apply a voltage to actuate the switch.)

You're worried about the thermal bending motion of the molecule accidentally closing the circuit, causing an error. You calculate, using the Boltzmann distribution over the elastic potential energy in the molecule, that the probability of a ... (read more)

I'm looking for AI safety projects with people with some amount of experience. I have 3/4 of a CS degree from Caltech, one year at MIRI, and have finished the WMLB and ARENA bootcamps. I'm most excited about making activation engineering more rigorous, but willing to do anything that builds research and engineering skill.

If you've published 2 papers in top ML conferences or have a PhD in something CS related, and are interested in working with me, send me a DM.

I’m not the person you are looking for, but I think it’s a great idea to put this out there and try to find collaborators, especially in the case of independent researchers. I’ll be actively trying to do the same.

I’m often reminded of a productivity tip by Spencer Greenberg:

4. Involve other people in projects in such a way that they rely on you to get your parts done. 

For me personally, this is probably the most powerful technique I've ever discovered for getting work done efficiently. When I know someone needs something by 2pm, and will be waiting if

... (read more)

On applying generalization bounds to AI alignment. In January, Buck gave a talk for the Winter MLAB. He argued that we know how to train AIs which answer on-distribution questions at least as well as the labeller does. I was skeptical. IIRC, his argument had the following structure:


1. We are labelling according to some function f and loss function L.

2. We train the network on datapoints (x, f(x)) ~ D_train.

3. Learning theory results give (f, L)-bounds on D_train. 


4. The network should match f's labels on the rest of D_train, on av

... (read more)

Huh, well that's something.

I'm curious, who else got this? And if yes, anyone click the link? Why/why not?

Showing 3 of 6 replies (Click to show all)

I got both mails (with a different virtue).  I clicked on it.

I think this is a meta-petrov, where everyone has the choice to make their preference (likely all in the minority, or stated as such even if not) the winner, or to defer to others.  I predict that it will eventually be revealed that the outcome would be better if nobody clicked the second link.  I defected, because pressing buttons is fun.

A small extra detail not mentioned: the end of the linked URL is "unilateralism=true".
I got it both messages. Only clicked on the first. I guess other admins besides you were working on this and didn't say anything to you?

For the record, to mods: I waited till after petrov day to answer the poll because my first guess upon receiving a message on petrov day asking me to click something is that I'm being socially engineered. Clicking the next day felt pretty safe.

Does anybody know if consensus algorithms have been proposed that try to reduce centralization by requiring quick coordination across large parts of the network, i.e., it doesn't work well to have machines only in one place?  

Showing 3 of 9 replies (Click to show all)

you can only prove upper bounds on latency

Have them prove an upper bound on latency to something across the globe?

There is no difference at the hardware level between being 'close to' and 'having a low-latency connection to', as I already explained. And to the extent that having those connections matter, miners already have them. In particular, in Ethereum, due to the money you can make by frontrunning transactions to hack/exploit them ('miner exploitable value'), HFT Ethereum miners/stakers invest heavily in having a lot of interconnected low-latency Sybils nodes so they can see unconfirmed transactions as quickly as possible, compute a maximally-exploitative block (eg. temporarily jacking up the price of a thing being purchased using a flash loan solely to rip off a specific transaction), and get that block committed before anyone can beat them to the same exploit. Having a lot of MEV is considered a bad thing and Ethereum types are spending increasing effort on approaches like commit-and-reveal to minimize MEV, which comes at the expense of users and makes them very unhappy. You could, I suppose, design a protocol which has extra MEV by designing transactions to be especially exploitable, but most people would consider that a bad thing...
Thank you for the detailed explanation. I understand that the incentives are already to have a maximally well-connected network with nodes between (latency-wise) geographically distant other nodes whenever that is feasible from an interconnect point.  Though thinking about it, it probably means that this burns not only compute but also network traffic.

There is an ongoing Petrov Day poll. I don't know if everyone on LW is being polled.

The best way to honor Stanislav Petrov is to arrange systems not to need Stanislav Petrov.

This is super rough and unrefined, but there's something that I want to think and write about. It's an epistemic failure mode that I think is quite important. It's pretty related to Reversed Stupidity is Not Intelligence. It goes something like this.

You think 1. Alice thinks 2. In your head, you think to yourself:

Gosh, Alice is so dumb. I understand why she thinks 2. It's because A, B, C, D and E. But she just doesn't see F. If she did, she'd think 1 instead of 2.

Then you run into other people being like:

Gosh, Bob is so dumb. I understand why he thinks 1.

... (read more)
Load More