Shortform Content [Beta]
I've worked as a professional programmer for nine years now. I think in at some point a few years ago, it actually began to erode my sense of agency working with computers. At a certain point I became less interested in hobby programming. This was 100% a healthy thing, I started doing things like dancing a lot of contact improv and rock climbing. But when I largely stopped hobby programming, almost all my programming experience was coming from working on production systems. Writing production code is slow. I've routinely had the experience of one or two li... (read more)
Research Agenda Base Rates and Forecasting
An uninvestigated crux of the AI doom debate seems to be pessimism regarding current AI research agendas. For instance, I feel rather positive about ELK's prospects, but in trying to put some numbers on this feeling, I realized I have no sense of base rates for research program's success, nor their average time horizon. I can't seem to think of any relevant Metaculus questions either.
What could be some relevant reference classes for AI safety research program's success odds? Seems most similar to disciplines ... (read more)
When is an event surprising enough that I should be confused?
Today, I was reading Mistakes with Conservation of Expected Evidence. For some reason, I was under the impression that the post was written by Rohin Shah; but it turns out it was written by Abram Demski.
In retrospect, I should have been surprised that "Rohin" kept talking about what Eliezer says in the Sequences. I wouldn't have guessed that Rohin was that "culturally rationalist" or that he would be that interested in what Eliezer wrote in the sequences. And indeed, I was updating that Rohi... (read more)
Surprise and confusion are two different things[1], but surprise usually goes along with confusion. I think it's a good rationalist skill-to-cultivate to use "surprise" as a trigger to practice noticing confusion, because you don't get many opportunities to do that. I think for most people this is worth doing for minor surprises, not so much because you're that likely to need to do a major update, but because it's just good mental hygiene/practice.
- ^
Surprise is "an unlikely thing happened." Confusion is "a thing I don't have a good explanation for happened.
... (read more)Science fiction books have to tell interesting stories, and interesting stories are about humans or human-like entities. We can enjoy stories about aliens or robots as long as those aliens and robots are still approximately human-sized, human-shaped, human-intelligence, and doing human-type things. A Star Wars in which all of the X-Wings were combat drones wouldn’t have done anything for us. So when I accuse something of being science-fiction-ish, I mean bending over backwards – and ignoring the evidence – in order to give basically human-shaped beings a c
Science resists surveillance (dramatically more detailed record keeping) because real science is embarrassing.
reproduce ability helps with the critical path but not necessarily with all the thrown out side data in cases where it turns out that auxiliary hypotheses were also of interest.
In a world that is truly and completely post-scarcity there would be no need for making tradeoffs.
Normally when we think about a post-scarcity future we think in terms of physical resources like minerals and food and real estate because for many people these are the limiting resources.
But the world is wealthy enough that some people already have access to this kind of post-scarcity. That is, they have enough money that they are not effectively limited in access to physical resources. If they need food, shelter, clothing, materiel, etc. they can get it in s... (read more)
There will be always a way to ruin post-scarcity, if humanity reproduces exponentially. Unless some new laws of physics are discovered that would allow unlimited exponential growth. Or maybe future legislation will make reproduction the only remaining scarce thing. As people currently get richer, they have fewer babies on average, but the reason is that we live in (from historical perspective) unprecedented luxury that we now take for granted, and need to give up a part of it when taking care of kids. Post-scarcity robotic nannies could easily revert this ... (read more)
epistemic status: speculation backed by ~no reading
Asteroid mining is under-valued as an application of better rocket technology: clear economic advantage (rare metals that on earth have mostly accumulated in the core), no gravity wells to overcome, high potential for automation.
Compare to space colonisation: No economic advantage (instead probably a giant money-sink & a titanic public good), health problems galore (or planetary gravity wells humans want to get into & out of).
A potential problem with asteroid mining would be getting the mined mater... (read more)
- The energy input is absolutely trivial compared to sunlight; climate change is driven by changing radiation balances not energy inputs (on anything like this scale).
- You'd want to do a lot of aerobraking just to keep your metals easy-to-collect instead of dispersing them across the landscape. Expect regulatory issues anyway, since you've just built a dispenser of expensive rods from god.
Coping with being average: local group. If you want to produce some content like writing, you're very likely to take popular writers as a baseline. But in times of the Internet, those are super-high-performers from the far end of the distribution. You and I are just not as good them (I'm 99% sure the reader is not in top 1%).
This might make you lose motivation. You will produce less then your reference point. Your content will feel medicore. You will get orders of magnitude smaller audience.
Idea how to cope better: instead of taking the whole internet, tak... (read more)
Another technique is to compare yourself to your past self.
I'm often dissatisfied with my writing. But when I look back at stuff that I wrote six months ago, I can't help but notice how much better I've become.
The caveat here is that comparing myself to people like Scott Alexander gives me some direction. Comparing myself to an earlier version of myself doesn't give me that direction. Instead, it gives me a sort of energy/courage to keep on going.
Shortform #66 Waht, where did 7 months go?
Working mostly, actually. So that was good, and I'm still enjoying my hospital technology related job quite a bit, but I've been sorely missing having other aspects of my life than work (despite enjoying the work a lot).
I experienced a truly wonderful thing tonight, and that was a Norfolk Rationalists meeting with great conversations between excellent people, we went for five & a half hours and only stopped because we all got too sleepy.
Re: Shortform #65: no, I probably will not go to medical school. I am happi... (read more)
AGI will probably be deployed by a Moral Maze
Moral Mazes is my favorite management book ever, because instead of "how to be a good manager" it's about "empirical observations of large-scale organizational dynamics involving management".
I wish someone would write an updated version -- a lot has changed (though a lot has stayed the same) since the research for the book was done in the early 1980s.
My take (and the author's take) is that any company of nontrivial size begins to take on the characteristics of a moral maze. It seems to be a pretty good nul... (read more)
The essence of systems thinking: Every persistent biological or cultural structure exists because of a positive feedback loop. Sometimes it’s hard to see. But to understand the structure, you must understand the loop.
— Kevin Simler
Positive loops by themselves are unstable, yes, and both are needed for stability, but positive loops are primary. Without a positive loop, there would be nothing for a negative loop to stabilize.
Kevin Simler goes on to explain that he was talking about biological and social phenomena and he's not sure about physical phenomena.
Why do planets exist? You might think that gravity is a positive feedback mechanism. The larger a planet gets, the more it attracts other mass. I think that this is correct for black holes, but not quite correct for gravity in the Newtonian regime: a planet attracts other matter, but this gives it the energy to fly away again. I think a more fundamental mechanism is electro-magnetism making matter "sticky." It gives matter in... (read more)
In the early 1900s the Smithsonian Institution published a book each year, which mostly just described their organizational and budget updates. But they each also contained a General Appendix at the end, which seems to have served a function analogous to the modern "Edge" essays—reflections by scientists of the time on key questions of interest. For example, the 1929 book includes essays speculating about what "life" and "light" are, how insects fly, etc.
"Do or Do Not: There is No Try"
Like all short proverbs each word is doing a lot of work and you can completely flip the meaning by switching between reasonable definitions.
I think "there is no try" often means "I want to gesture at this but am not going to make a real attempt" in sentences like "I'll try to get to the gym tomorrow" and "I'll try to work on my math homework tonight".
"there is no try" means "I am going to make an attempt at this but it's not guaranteed to succeed" in sentences like "I'm going to try to bench 400 tomorrow", "I'm t... (read more)
OOOOH it's maybe encapsulated in "I'll try to do action" vs "I'm trying this action"
I find myself, just as a random guy, deeply impressed at the operational competence of airports and hospitals. Any good books about that sort of thing?
It is pretty impressive that they function as well as they do, but seeing how the sausage is made (at least in hospitals) does detract from it quite substantially. You get to see not only how an enormous number of battle hardened processes prevent a lot of lethal screw-ups, but also how also how sometimes the very same processes cause serious and very occasionally lethal screw-ups.
It doesn't help that hospitals seem to be universally run with about 90% of the resources they need to function reasonably effectively. This is possibly because there is relentle... (read more)
Rationality exercise: Take a set of Wikipedia articles on topics which trainees are somewhat familiar with, and then randomly select a small number of claims to negate (negating the immediate context as well, so that you can't just syntactically discover which claims were negated).
... (read more)By the time they are born, infants can recognize and have a preference for their mother's voice suggesting some prenatal development of auditory perception.
-> modified to
Contrary to early theories, newborn infants are not particularly adept at picking out the
I remember the magazine I read as a kid (Geolino) had a section like this (something like 7 news stories from around the World and one is wrong). It's german only, though I'd guess a similar thing to exist in english media?
A bright tomorrow
Original: https://brain.louis030195.com/Philosophy/Poetry/A+bright+tomorrow
When faced with choices, we often fall prey
To cognitive biases that can lead us astray;
We're overconfident, or we're too dismissive,
And end up making suboptimal decisions.
If we're to make the best choices we can;
By being mindful of the way we think,
We can avoid these pitfalls and make better plans.
In life, there will be many grieving times;
But if we keep our wits about us, we'll find
That we can mourn our losses and still move on,
Toward the bright tomorro... (read more)
Trying out LW shortfrom as a platform. Mostly planning to publish thoughts related to my experience with Logan's naturalism. I have not taken the course, so I am almost certainly doing something of my own stumbling inspired by their essays.
One thing that has recently struck me:
- The focus on the "natural" as a way to avoid fooling yourself by observing human-created phenomena and guessing rather than observing (paraphrasing from here)
This makes a lot of sense, but I do it differently. Another way to avoid fooling yourself is to have a clear marker of success... (read more)
A map displaying the prerequisites of the areas of mathematics relevant to CS/ML:
A dashed line means this prerequisite is helpful but not a hard requirement.
I’ve updated the diagram to indicate nice to have prerequisites with dashed lines following your remarks.
Showerthought: what's the simplest way to tell that the human body is less than 50% efficient at converting chemical energy to mechanical work via running? I think it's that running uphill makes you warmer than running downhill at the same speed.
When running up a hill at mechanical power p and efficiency f, you have to exert p/f total power and so p(1/f - 1) is dissipated as heat. When running down the hill you convert p to heat. p(1/f - 1) > p implies that f > 0.5.
Maybe this story is wrong somehow. I'm pretty sure your body has no way of recovering your potential energy on the way down; I'd expect most of the waste heat to go in your joints and muscles but maybe some of it goes into your shoes.
Running barefoot will produce the same observations, right? So any waste heat going into your shoes is probably a small amount.
Causal prediction markets.
Prediction markets (and prediction tournaments more generally) may be useful for telling us not only what will happen, but which actions will achieve our goals. One proposal for getting prediction markets to help with this is to get users to make conditional predictions. For example, we can ask the question "if Biden wins the election, GDP will be higher than if Trump wins" and use that as evidence about who to elect, and so on. But conditional predictions only predict the effect of an action if the event (e.g. who is elected) is ... (read more)
I see the main problem with these schemes as one of liquidity and collateral. Generally it's difficult to set up prediction markets where you condition on unlikely events (like a market on what's essentially a regression discontinuity) because assets that only pay out a small fraction of the time are not attractive to most buyers: if they don't hold a huge diversified portfolio of claims it means they must tie up a large amount of cash in collateral for small expected returns, and in those situations the cash can often get higher returns elsewhere so peopl... (read more)