I'm not really sure how shortform stuff could be implemented either, but I have a suggestion on how it can be used: jokes!

Seriously. If you look at Scott's writing, for example, one of the things which makes it so gripping is the liberal use of amusing phrasing, and mildly comedic exaggerations. Not the sort of thing that makes you actually laugh, but just the sort of thing that is mildly amusing. And, I believe he specifically recommended it in his blog post on writing advice. He didn't phrase his reasoning quite like this, but I think of it as little bits of positive reinforcement to keep your system 1 happy while your system 2 does the analytic thinking stuff to digest the piece.

Now, obviously this could go overboard, since memetics dictates that short, likeable things will get upvoted faster than long, thoughtful things, outcompeting them. But, I don't think we as a community are currently at risk of that, especially with the moderation techniques described in the OP.

And, I don't mean random normal "guy walks into a bar" jokes. I mean the sort of thing that you see in the comments on old LW posts, or on Weird Sun Twitter. Jokes about Trolley Problems and Dust Specks and Newcomb-like problems and negative Utilitarians. "Should Pascal accept a mugging at all, if there's even a tiny chance of another mugger with a better offer?" Or maybe "In the future, when we're all mind-uploads, instead of arguing about the simulation argument we'll worry about being mortals in base-level reality. Yes, we'd have lots of memories of altering the simulation, but puny biological brains are error-prone, and hallucinate things all the time."

I think a lot of the reason social media is so addictive is the random dopamine injections. People could go to more targeted websites for more of the same humor, but those get old quickly. The random mix of serious info intertwined with joke memes provides novelty and works well together. The ideal for a more intellectual community should probably be more like 90-99% serious stuff, with enough fun stuff mixed in to avoid akrasia kicking in and pulling us toward a more concentrated source.

The implementation implications would be to present short-form stuff between long-form stuff, to break things up and give readers a quick break.

Note to self, in case I come back to this problem: the Vienna Circle fits the bill.

:)

Honestly, there are a bunch of links I don't click, because the 2 or 3 word titles aren't descriptive enough. I'm a big fan of the community norm on more technically minded subreddits, where you can usually find a summary in one of the top couple comments.

So, I'm doing what I can to encourage this here. But mostly, I thought it was important on the AI front, and wanted to give a summary which more people would actually read and discuss.

Here are some thoughts on the viability of Brain Computer Interfaces. I know nothing, and am just doing my usual reality checks and initial exploration of random ideas, so please let me know if I'm making any dumb assumptions.

They seem to prefer devices in the blood vessels, due to the low invasiveness. The two specific form factors mentioned are stents and neural dust. Whatever was chosen would have to fit in the larger blood vessels, or flow freely through all of them. Just for fun, let's choose the second, much narrower constraint, and play with some numbers.

Wikipedia says white blood cells can be up to 30 μm in diameter. (Also, apparently there are multiple kinds of white blood cells. TIL.) I'd guess that we wouldn't want our neural dust to be any larger than that if we want to be able to give it to someone and be able to reverse the procedure later without any surgery. The injection should be fine, but if you wanted to filter these things back out of your blood, you'd have to do something like giving blood, but with a magnet or something to filter out the neural dust. So, what could we cram into 30 μm?

Well, my first hit when searching "transistors per square mm" is an article titled "Intel Now Packs 100 Million Transistors in Each Square Millimeter", so let's go with that. I realize Elon's ~10 year time horizon would give us another ~6 Moore's law doublings, but if they did an entire run of a special chip just for this, then maybe they don't want to pay top dollar for state of the art equipent, so let's stick with 100m/mm^2. That'd give us on the order of 10k-100k transistors to work with, if we filled the entire area with transistors and nothing else.

But, looking at most electronics, they are more than just a chip. Arduinos and cellphones and motherboards may be built around a chip, but the chip itself has a relatively small footprint on the larger PCB. So, I'm probably missing something which would be incredibly obvious to someone with more hardware experience. (Is all the other stuff just for interfacing with other components and power supplies? In principle, could most of it be done within the chip, if you were willing to do a dedicated manufacturing run just for that one divice, rather than making more modular and flexible chips which can be encorporate into a range of devices?)

If we assume it'd be powered and transmit data electromagnetically, it'd also need an antenna, and an induction coil. I have a hunch that both of these suffer from issues with the square-cube law, so maybe that's a bad idea. The neural dust article mentioned that the (mm scale) devices both reported information and received power ultrasonically, so maybe the square-cube law is the reason. (If not, we might also run into the diffraction limit, and not have any wavelengths of light which were short enough to effect antenas that size, but still long enough to penetrate the skull without ionizing atoms.)

I like the idea of ultrasonic stuff because acoustic waves travel through tissue without depositing much energy. So, you get around the absorption problem photons have, and don't have to literally x-ray anyone's brain. Also, cranial ultrasounds are already a thing for infants, although they have to switch to transcranial Doppler for adults, because our skulls have hardened. Nearby pieces of neural dust would be monitoring the same neurons, and so would give off their signals at about the same time, boosting the signal but maybe smearing it out a little in time.

So, let's play with some numbers for piezoelectric devices instead. (I assume that's what their ultrasonic neural dust must be using, at least. They are switching between electricity and motion somehow, and piezoelectrichttps are the name for the solid state way of doing that. I can't picture them having tiny speakers with electromagnets on flexible speaker cones. The Wikipedia page on transducers doesn't mention other options.)

Quartz crystals are already used for timing in electronics, so maybe the semiconductor industry already has the ability to make transducers if they wanted to. (I'd be surprised if they didn't, since quartz is just ccrystaline silicon dioxide. Maybe they can't get the atomic lattice into the right orientation consistently, though.) If you couldn't transmit and receive simultaneously without interfering, you'd need a tiny capacitor to store energy for at least 1 cycle. I don't know how small quartz crystals could be made, or whether size is even the limiting factor. Maybe sufficiently small piezoelectric can't even put out strong enough pulses to be detectable on an ultrasound, or require too much power to be safely delivered ultrasonically? I don't know, but I'd have to play with a bunch of numbers to get a good feel.

I don't really know where to start, when discussing monitoring neuron firings. Could it be done electromagnetically, since they should make an instantaneous electromagnetic field? Or would the signal be too weak near a blood vessel? Apparently each neuron firing changes the concentration of Na, K, Cl, and Ca in the surrounding blood. Could one of these be monitored? Maybe spectrally, with a tiny LED of the appropriate wavelength, and a photo detector? I think such things are miniturizeable in principle, but I'm not sure we can make them with existing semiconductor manufacturing techniques, so the R&D would be expensive. We probably don't have anything which emits at the exact wavelength we need for spectroscopy though, and even if we did, I bet the LED would need voltage levels which would be hard to deliver without adding a voltage transformer or whatever the DC equivalent is.

Or, can we dump all the fancy electronics all together? Could we do something as simple as a clay particle (tiny rock) coated with a dispersent or other Surfactant, so that changes in the surrounding chemistry cause the collapse of the double layer), making the clay particles to flocculate together? Would such clumps of clay particles be large enough and have high enough density to show up on an ultrasound or other divice? Obviously this wouldn't let us force a neuron to fire, but it might be a cheap way of detecting them.

Maybe the electronics could be added later, if modifying surface charge and chemistry is enough to make a neuron fire. Neurotransmitrers affect neuron firings somehow, if I usnderstand correctly, so maybe chain a bunch of neurotransmitters to some neural dust as functional groups on the end of polymer chains, then change surface charge to make the chains scrunch up or fan out?

I only know just enough about any of this to get myself into trouble, so if it doesn't look like I know what I'm talking about, I probably don't.

(Sorry to spam comments. I'm separating questions out to keep the discussion tidy.)

The article only touches on it briefly, but suggests faster AI takeoff are worse, but "fast" is only relative to the fastest human minds.

Has there been much examination of the benefits of slow takeoff scenarios, or takeoffs that happen after human enhancements become available? I vaguely recall a MIRI fundraiser saying that they would start putting marginal resources toward investigating a possible post-Age of EM takeoff, but I have no idea if they got to that funding goal.

Personally, I don't see Brain-Computer Interfaces as useful for AI takeoffs, at least in the near term. We can type ~100 words per minute, but it takes more than 400 minutes to write a 40,000 word novel. So, we aren't actually I/O bound, as Elon believes. We're limited by the number of neurons devoted to a given task.

Early BCIs might make some tasks much faster, like long division. Since some other tasks really are I/O bound, they'd help some with those. But, we wouldn't be able to fully keep up with AI unless we had full-fledged upgrades to all our cognative architecture.

So, is almost keeping up with AI likely to be useful, or are slow takeoff just as bad? Are the odds of throwing together a FAI in the equivalent of a month any better than in a day? What % of those pannicked emergency FAI activities could be speed up by better computer user interfaces/text editors, personal assistants, a device that zapped your brain every time it detected Akrasia setting in, or by a RAM upgrade to the brain's working-memory?

(sorry to spam. I'm separating questions out to keep the discussion tidy.)

TL;DR of the article:

This piece describes a lot of why Elon Musk wanted to start Neurolink, and how Brain-Computer Interfaces (BCIs) currently work, and how they might be implemented in the future. It's a really, really broad article, and aims for breadth while still having enough depth to be useful. If you already have a grasp of evolution of the brain, Dual Process Theory, parts of the brain, how neurons fire, etc. you can skip those parts, as I have below.

AI is dangerous, because it could achieve superhuman abilities and operate at superhuman speeds. The intelligence gap would be much smaller if we also had access to such abilities. Therefore, we should attempt this if possible.

This might be possible, despite how extremely limited and highly invasive existing BCIs are. Opening the skull is obviously way too invasive for most people, but the blood vessels offer a possible minimally invasive solution. They are essentially a highway which goes directly to every neuron in the brain. Current methods monitor at most ~100 neurons, or have low temporal resolution. 1,000,000 neurons is probably the tipping point, where it would stop being an alternative to a keyboard/screen input/outputs, and start being transformative.

Neuralink is exploring many possibilities, and probably won't narrow to just one any time soon. However, options might include "neural dust", or stints in the blood vessels. Just as dies have made fine cell structures visible under microscopes, and genetically engineering bioluminescent genes into living animals has made cells glow when active, Neuralink would need a way for such a device to detect individual neuron firings on a large scale.

To do this, the inserts themselves only need to be able to:

1. React differently to electrical discharge associated with a nearby neurons firing, or to other changes associated with neurons firing, like sodium and potassium levels.

2. Have that difference be detectable from outside the skull. (I'd divide this into active methods, like emitting light in a wavenelgth which penetrates the skull, or passive changes in properties detectable from the outside, like radioactive isotopes which cluster together based on variables in blood flow.)

(The piece doesn't make this distiction, but I thought it would be useful for better discussion and understanding.)

Neuralink, of course, hasn't narrowed the specifics down very much (and will probably pivot several times, in my opinion). However, they will start out offering something largely similar to the sorts of BCIs available to people with paralysis or sensory problems. Elon hopes that if everything goes smoothly, in a decade they would have something which could provide a useful feature to someone without such disabilities, if the FDA would allow it.

They also hope to be able to eventually influence neural firings, so that we could supply information to the brain, rather than just reading information out. This would require something which could be influenced from the outside, and then influence nearby neurons. We can already put an electric field through the whole brain, to minimize seizures, but for meaningful inputs this would also have to be done at the neuron leven.

Why you should read it anyway:

It's >35,000 words. (For comparison, the cutoff for "short novel" is 40,000.) That said, it's a good read, and I recommend it if you want to understand why Elon Musk might think a BCI might increase our odds of surviving an AI takeoff scenario.

A lot of it is still hand-waving, and doesn't make it clear that we don't necessarily need full self-replicating autonomous nanobots or whatever. Since it doesn't provide a specific architecture, but just surveys what might be possible, I think it's easy to give an uncharatable reading. I've tried to steel-man the phrasing here, but I think if we focus on tangible, near-term concepts, it can be illustrative of what is possible.

If you read this with a critical eye, you'll just note that they haven't narrowed down to one architecture yet, and complain that their lack-of-an-architecture can't possibly work. The point is to convince lay people that this might even be possible, not to convince them that Neurolink will succeed, but the comments I've seen so far have just been skepticism of Neurolink.

Instead, I'd encourage you to read with an eye toward what could be done with a stint or neural dust, and then critically examine the more tangible challenge of how small each of those possible capabilities could be made. What could be done passively? What could be done if inductively powered? How small of blood vessels could various devices fit through? Will those shrink with Moore's law, or are they physics-constrained?

Such questions will generate the possible concrete architectures which you can then apply a critical lens to. Don't bother reading if you just want to be critical of the exploratory activity itself. It won't even put up a fight.

TL;DR: What are some movements you would put in the same reference class as the Rationality movement? Did they also spend significant effort trying not to be wrong?

Context: I've been thinking about SSC's Yes, We have noticed the skulls. They point out that aspiring Rationalists are well aware of the flaws in straw Vulcans, and actively try to avoid making such mistakes. More generally, most movements are well aware of the criticisms of at least the last similar movement, since those are the criticisms they are constantly defending against.

However, searching "previous " in the comments doesn't turn up any actual exemples.

Full question: I'd like to know if anyone has suggestions for how to go about doing reference class forcasting to get an outside view on whether the Rationality movement has any better chance of succeeding at it's goals than other, similar movements. (Will EA have a massive impact? Are we crackpots about Cryonics, or actually ahead of the curve? More generally, how much weight should I give to the Inside View, when the Outside View suggests we're all wrong?)

The best approach I see is to look at past movements. I'm only really aware of Logical Positivism, and maybe Aristotle's Lyceum, and I have a vague idea that something similar probably happened in the enlightenment, but don't know the names of any smaller schools of thought which were active in the broader movement. Only the most influential movements are remembered though, so are there good examples from the past ~century or so?

And, how self-critical were these groups? Every group has disagreements over the path forward, but were they also critical of their own foundations? Did they only discuss criticisms made by others, and make only shallow, knee-jerk criticisms, or did they actively seek out deep flaws? When intellectual winds shifted, and their ideas became less popular, was it because of criticisms that came from within the group, or from the outside? How advanced and well-tested were the methodologies used? Were any methodologies better-tested than Prediction Markets, or better grounded than Bayes' theorem?

Motive: I think on average, I use about a 50/50 mix of outside and inside view, although I vary this a lot based on the specific thing at hand. However, if the Logical Positivists not only noticed the previous skull, but the entire skull pile, and put a lot of effort into escaping the skull-pile paradigm, then I'd probably be much less certain that this time we finally did.

I'm not so sure. Would your underlying intuition be the same if the torture and death was the result of passive inaction, rather than of deliberate action? I think in that case, the torture and death would make only a small difference in how good or bad we judged the world to be.

For example, consider a corporate culture with so much of this dominance hierarchy that it has a high suicide rate.

Also:

Moloch whose buildings are judgment! ... Lacklove and manless in Moloch! ... Moloch who frightened me out of my natural ecstasy!

... Real holy laughter in the river! They saw it all! the wild eyes! the holy yells! They bade farewell! They jumped off the roof! to solitude! waving!

— Meditations on Moloch/Howl

Doesn't seem like a difference of kind, and maybe not even of degree. (The suicide rate is a couple percent, and higher in industrialized countries if I recall. What percent of the citizens of Oceania are tortured to death? ~2%?) I think 1984 is mainly shocking because of status quo bias. (But I haven't read it, so I'm probably missing some stronger points against that world.)

Most of the badness seems to be from the general state of both worlds, rather than from the occasional person tortured to death on the side. That's just the tip of the iceberg. It's a small, but obvious, part of much deeper problems. That's why EA doesn't use suicide rate or incarceration rate as their primary metrics to optimize for. They're just symptoms.

I'd add that it also starts to formalise the phenomenon where one's best judgement oscillates back and forth with each layer of an argument. It's not clear what to do when something seems a strong net positive, then a strong negative, then a strong positive again after more consideration. If the value of information is high, but it's difficult to make any headway, what should we even do?

This is especially common for complex problems like xrisk. It also makes us extremely prone to bias, since we by default question conclusions we don't like more than ones we do.

This is really sad. I'm sorry to hear things didn't work out, but I'm still left wondering why not.

I guess I was really hoping for a couple thousand+ word post-mortem, describing the history of the project, and which hypotheses you tested, with a thorough explanation of the results.

If you weren't getting enough math input, why do you think that throwing more people at the problem wouldn't generate better content? Just having a bunch of links to the most intuitive and elegant explanations, gathered in one place, would be a huge help to both readers and writers. Students trying to learn are already doing this through blind googling, so the marginal work to drop the links is low.

Pulling all the info together into a good explanation still requires one dedicated person, but perhaps that task can be broken down into chunks too. Like, once one version is written, translating it for non-mathy people should be relatively easy. Same for condensing things for mathy people.

But, why wouldn't adding more mathematicians mean a few would be good at and interested in writing new articles? Where did you do outreach? What did you do? There are entire communities, scattered across the web, who exist to try and learn and teach math. Have you tried partnering with any of them, or recruiting members?

If not, why do you think it won't work? Do you see promising alternative approaches, or are good explanations impossible even in principle?

Sorry for the flood of questions. I've just been waiting with baited breath for Arbital to stop pushing me away and start pulling people in. I even linked some people, but felt guilty about it for putting a strain on your overloaded servers before you were ready for the general public.