This post was funded by a grant from Ben Pace.
Ben Pace asks:
To what extent have ideas and scientific discoveries gotten harder to find?Related: Why are there no gentlemen scientists any more — i.e. rich people who make novel scientific discoveries? Like Fermat and Pascal.Some theories have been put forward by Scott Alexander and Holden Karnofsky. Maybe they’re right, maybe not.
To what extent have ideas and scientific discoveries gotten harder to find?
Related: Why are there no gentlemen scientists any more — i.e. rich people who make novel scientific discoveries? Like Fermat and Pascal.
Some theories have been put forward by Scott Alexander and Holden Karnofsky. Maybe they’re right, maybe not.
Scott Alexander treats scientific ideas as nonrenewable.
Imagine scientists venturing off in some research direction. At the dawn of history, they don’t need to venture very far before discovering a new truth. As time goes on, they need to go further and further.
Holden Karnofsky believes the pattern applies to both art and science.
The broad theme is that across a variety of areas in both art and science, we see a form of "innovation stagnation": the best-regarded figures are disproportionately from long ago, and our era seems to "punch below its weight" when considering the rise in population, education, etc. Since the patterns look fairly similar for art and science, and both are forms of innovation, I think it's worth thinking about potential common factors.
I have a different perspective.
The most important scientific discoveries are those which are the most general and the most useful. Physics is the most general of sciences. Physics is basically solved.
There are unsolved problems in physics. Dark matter remains a mystery. Quantum mechanics has yet to be unified with general relativity. But the holes in physics don't matter. You don't need a Grand Unified Theory of quantum relativity to build a Mars base or a fusion reactor. All you need is today's physics plus a whole lot of engineering.
The recent discoveries in physics like quantum computing and the photograph of a black hole aren't really discoveries about the fundamental Laws of Physics. They're technological achievements.
All of the rest of science is just applied physics too. One could argue that biology, chemistry and so on are just footnotes to Einstein. The fundamental laws of the universe are (for all practical purposes) known. The remaining questions are:
No physicist will ever again make a discovery (in physics) as impactful as the great 20th century physicists. All the important fruit has been picked.
But that doesn't mean science has been exhausted. It just means physics has been exhausted. Biology is advancing fast. I'll never get tired of this graph about how, since 2007, biotechnology has advanced faster than computer technology ever did.
As recently as 2015, Nick Lane published a book that might have solved the origin of life. And biology isn't even the most exciting frontier.
In Contra Hoel, I talked about machine learning as feeling different from some other scientific fields: there are frequent exciting new discoveries. This shouldn’t be surprising. Physics is stagnant because Newton and Einstein already got all the cool results. But Newton and Einstein didn’t have TPUs so they couldn’t discover things about machine learning.―The Low-Hanging Fruit Argument: Models And Predictions by Scott Alexander
In Contra Hoel, I talked about machine learning as feeling different from some other scientific fields: there are frequent exciting new discoveries. This shouldn’t be surprising. Physics is stagnant because Newton and Einstein already got all the cool results. But Newton and Einstein didn’t have TPUs so they couldn’t discover things about machine learning.
―The Low-Hanging Fruit Argument: Models And Predictions by Scott Alexander
Do discoveries in machine learning count as science or technology? If we use the strictest definition of "science" then machine learning counts as "technology". But machine learning is also informing our understanding of the human mind. Psychology definitely counts as "science". "Untangling how intelligence works" is the most important scientific problem of our age.
Ambitious people go to where the low-hanging fruit is. I think it would be a mistake for an ambitious young person to study physics when machine learning has so many more new developments. (I say this as someone who did study physics.)
Solving the problem of AGI would eclipse everything else humanity has ever achieved. We are making faster progress toward AGI than at any point in history. So why does it feel like the low-hanging fruit has already been picked?
Because the fruit that has been picked is public knowledge but the fruit that hasn't been picked is not public knowledge. Unless you are an expert on the vanguard of a pioneering field, you cannot see the unpicked fruit.
I think the natural universe is like a game of chess. We have figured out the rules of our universe (like →F=m→a and E=mc2). But that was just the beginning. Now our challenge is figuring out how to play the game.
It is as if our civilization has just invented writing. (Which, on a cosmological scale, it has.) That was the boring part. Now is the time to write.
Why are there no gentlemen scientists any more — i.e. rich people who make novel scientific discoveries? Like Fermat and Pascal.
The greatest pure scientific discoveries have been exhausted. Many technologies are still invented by people doing things for fun. Steve Wozniak built his first computer for fun. Linus Torvolds, the inventor of Linux, literally titled his autobiography "Just for Fun".
I have lots of ideas. How many ideas? I recently posted a list of some of my ideas for blog posts. The number of ideas I have vastly outnumbers the number of ideas I will ever find the time to write about. This isn't because I don't have enough time to write. It's because writing generates ideas faster than I can write about them. I could write for a hundred lifetimes and never run out of things to say.
The phenomenon of "doing things gives you ideas for things to do" isn't limited to writing. It happens in business too. Starting companies takes many years. Few people can start more than a couple companies in a lifetime. I got three ideas for companies to start while running my first company. Starting companies gives you ideas for new companies (and variations on existing companies).
Brandon Sanderson has more ideas for novels then he can write in a lifetime. I would be astonished of artists like Vanripper or Telepurte didn't generate ideas faster than they can draw.
If you don't draw then you probably have few ideas for things to draw. If you don't write novels then you probably have few ideas for novels to write.
My favorite book about getting ideas is Steal Like an Artist: 10 Things Nobody Told You About Being Creative by Austin Kleon. Kleon's core premise is that to truly make an apple pie from scratch you must first create the universe. Nobody is original. Everybody copies. "Originality" is copying without getting caught.
Everything good I have ever written is a mix of other peoples' art. Bayeswatch (a dystopic science fiction series about AI alignment) is a cross between I, Robot by Isaac Asimov and Men in Black (1997). Predictive Coding has been Unified with Backpropagation is a summary of someone else's research paper. Russia has Invaded Ukraine was the curated opinions of the experts on Foreign Affairs.
The more ideas other people publish the more there is available for you to remix and adapt. Thanks to the Internet, there is more material for you to steal than ever before.
Holden Karnofsky asks:
Say that Beethoven was the greatest musician of all time (at least in some particular significant sense - see below for some caveats). Why has there been no one better in the last ~200 years - despite a vastly larger world population, highly democratized technology for writing and producing music, and a higher share of the population with education, basic nutrition, and other preconditions for becoming a great musician? In brief, where's today's Beethoven?
Beethoven is impressive, sure. But being impressive and being good are different things.
It can be hard to separate the things you like from the things you're impressed with. One trick is to ignore presentation. Whenever I see a painting impressively hung in a museum, I ask myself: how much would I pay for this if I found it at a garage sale, dirty and frameless, and with no idea who painted it? If you walk around a museum trying this experiment, you'll find you get some truly startling results. Don't ignore this data point just because it's an outlier.Another way to figure out what you like is to look at what you enjoy as guilty pleasures. Many things people like, especially if they're young and ambitious, they like largely for the feeling of virtue in liking them. 99% of people reading Ulysses are thinking "I'm reading Ulysses" as they do it. A guilty pleasure is at least a pure one. What do you read when you don't feel up to being virtuous? What kind of book do you read and feel sad that there's only half of it left, instead of being impressed that you're half way through? That's what you really like.―Copy What You Like by Paul Graham
It can be hard to separate the things you like from the things you're impressed with. One trick is to ignore presentation. Whenever I see a painting impressively hung in a museum, I ask myself: how much would I pay for this if I found it at a garage sale, dirty and frameless, and with no idea who painted it? If you walk around a museum trying this experiment, you'll find you get some truly startling results. Don't ignore this data point just because it's an outlier.
Another way to figure out what you like is to look at what you enjoy as guilty pleasures. Many things people like, especially if they're young and ambitious, they like largely for the feeling of virtue in liking them. 99% of people reading Ulysses are thinking "I'm reading Ulysses" as they do it. A guilty pleasure is at least a pure one. What do you read when you don't feel up to being virtuous? What kind of book do you read and feel sad that there's only half of it left, instead of being impressed that you're half way through? That's what you really like.
―Copy What You Like by Paul Graham
Beethoven wasn't the greatest musician of all time. If he was then modern people would be listening to Beethoven all the time. Instead, here is what I have actually been listening to. These are the last ten songs in my YouTube history.
Four of them are covers. Two are others remixes. None of them are Beethoven. None of them were written in the age of Beethoven. None of them are even remixes of work written in the age of Beethoven.
Master marketer Tim Ferris believes it's easier to create a category than to conquer someone else's category. Beethoven's work is prestigious because Beethoven was early. People listen to Beethoven because Beethoven is prestigious. But Beethoven's music is not actually better than the music getting published today.
Early work tends to be worse because artisans tend to improve over time due to competition and theft. Prestige, meanwhile, tends to increase over time (if you are remembered). Thus, we live in a world where old stuff (or, at least, the old stuff good enough to be preserved) is both highly-prestigious and yet not as good as the new stuff.
An aspiring author once got into an argument with epic fantasy author Jim Butcher. The aspiring author complained that he didn't have any good ideas. Jim Butcher claimed ideas don't matter. Jim Butcher could write about anything. Jim Butcher asked the aspiring author for the aspiring author's two worst ideas. The aspiring author replied "Pokémon" and "the Lost Roman Legion".
So Jim Butcher wrote an epic fantasy novel about Pokémon and the Lost Roman Legion.
It's called Codex Alera.
It was supposed to be a trilogy.
The fourth novel in the six-book series ranked #17 on the New York Times Best Seller List.
Nabokov is less popular and more prestigious than JK Rowling, and I prefer reading him, and get more pleasure out of doing so. I wouldn’t jump to the conclusion that people who say they prefer Beethoven to nightcore are lying to themselves. People’s tastes really do differ quite a lot.
I also think “only listen to/read/watch whatever gives you most units of pleasure per minute” is a meme that discourages people from seeking out a wide range of experiences. It’s suspiciously “wireheady”. If life was just guilty pleasures it would be a lot more boring. Better to take the risk of listening to jazz for a while, before realizing you’ve only been pretending to like it, then to just listen to pop music all the time and miss out on the possibly of having a new kind of experience.
Yeah, the argument "I don't listen to Beethoven, therefore Beethoven is not that great and people only pretend to like him for social reasons" seems really weak to me.
Physics is basically solved.
This echoes the sentiment of many prominent scientists in the late 1800s. All that was left was to resolve a few nagging irregularities.
I think we have gotten better at this and our exploration of reality has been both deeper and wider, meaning there are not that many low hanging fruits available. Now I'm not a physicist but I'm a mathematician, and in maths I don't expect us to find big gaps in our theories like they did in the 19th century, because we have gotten much more rigorous due to those early pitfalls.
(also I'd argue that there is not much we can do, engineering wise, due to relativity/quantum mechanic - for example the GPS uses general relativity, but in a world where Newtonian mechanic is true, we could still have made the GPS and it would in fact have been easier).
Such nagging irregularities included things like "how does the Sun shine". Current unexplained phenomena are much more exotic.
Exactly this. The rest, those little irregularities, at the time didn't matter, because we didn't know what we didn't know.
No? The Laws Underlying The Physics of Everyday Life Are Completely Understood. Seriously, The Laws Underlying The Physics of Everyday Life Really Are Completely Understood. That was not true 100 years ago, and that is true now. This is not actually a controversial statement.
Physics is very far from solved.
For example there are several proposals for resolving the dark matter problem, advanced by reputable people, and yet would still upend practically all knowledge of the universe derived so far, if true.
I am doubtful. Can you give an example of one such proposal, and its practical consequences? (Note that the claim is that physics is solved for all practical purposes, not that it is completely solved.)
Well several variants of vacuum energies, vacuum fluctuations, etc.
Practical consequence if true is that conservation of energy no longer holds as we understand it today. i.e. there could be a potentially accessible source of infinite energy.
Obviously not in the foreseeable future but it would go from strictly impossible to not so, thus implicating many things.
And since conservation of energy is just about the most fundamental theory there is, I can't any imagine any thing else that could have a larger practical consequence.