What is the world made of? The obvious answer is matter and the forms it takes: Atoms,molecules, organisms, machines, buildings, and galaxies.  Yet, there exists another set of structures, intangible, more abstract, but  just as real. These structures are not made of matter but of possibilities, the arrangement of all the things that could be.

You can view these as landscapes, vast terrains composed of all the possible forms a thing might take. Imagine,  for example, the landscape of all possible eyes, a vast terrain  covered by every eye that could possibly exist. If you envision this as a surreal land scattered with eyes of every imaginable shape and form, you wouldn’t be far off.  We can even consider abstract concepts, like the landscapes that span all possible ways to drive a car  or even write a poem. For every concept you can consider, there exists a structure  that contains every variation of this thing  that can ever be, all collected together and arranged in some abstract space.

I call these structures landscapes for a reason: Just like physical lands, they have contours:  valleys, hills, ridges, ravines, and even streams. The things that are, the things that exist, are flows running through these landscapes.  Just like a stream of water flowing through the surface, the trajectory of the things that exist is guided by the gradient of the landscape on which it moves.

Describing the flow of existence through the landscape of possibilities might sound  poetic, but these landscapes are real and  very practical. Scientists and engineers use them in numerous fields, from predicting how proteins evolve or how molecules arrange themselves, to teaching computers to drive cars and understand language. And  in these landscapes, there is a strange phenomena that counters our most basic intuition and yet  occurs again and again in different domains. Ever-ascending trails that cross these landscapes, covering vast distances and connecting remote regions that  by every common sense should have been unreachable.  The existence of these trails remains largely unexplained, and yet they have massive implications for technology, life, the universe, and everything.

The first example of  these strange landscapes comes from  the evolution of life. The theory of evolution appears at once obvious and implausible. The first assertion is almost trivial: living creatures pass on their traits to their offspring, but with small random variations. In a natural environment, some variations provide an advantage for survival and reproduction and thus become more common.

Evolution's second assertion is that,  given enough time, these small variations would accumulate, leading to the emergence of completely new features, organs, and species. Every creature, every organ in nature has emerged solely from this process. In contrast to the first assertion, this claim seems highly implausible. To understand why, let's return to the landscape of possibilities.

Consider, for example, the landscape of all eyes. This vast land contains all eyes that could possibly exist, functional or dysfunctional, from the eye of an eagle to a simple light-sensitive tissue. Every eye that has ever existed, as well as infinite others that never were.

In this landscape, the more similar two eyes are, the closer they are to each other. The "altitude" of this landscape represents the advantage a specific eye provides its owner. On this terrain, the eagle's eye sits on one of the tallest peaks, surrounded by eyes of similar acuity. A vast distance away, a dysfunctional, sightless tissue lies in one of the lowest valleys. What the theory of evolution tells us is that a continuous trail leads from the simple, blind tissue to the eagle's eye, where every step along the path offers an improvement (or at least no degradation). A path from the lowest valley to the highest peak in a complex landscape that contains no downward slopes. From a functional perspective, this seems wildly improbable. The eye, like any complex device, depends on many interdependent components, finetuned to one another. The idea of constructing such a device in tiny increments, with each step yielding a functional improvement or even a working device, seems highly unlikely from an engineering standpoint.

We can also consider this from a topological perspective. Imagine trying to cross a real landscape while following the rule that you are only allowed to walk uphill and are forbidden from ever stepping down, and very soon, you are bound to find yourself stuck on a big rock or minor elevation with no way to go but down. Yet, the theory of evolution makes this claim not just for the eye, but for every organ and every creature in nature. How unlikely it is that these kinds of trails would exist for so many different organs with completely different functions and structures, across so many different creatures and life forms. Biology has largely evaded this question by focusing on empirical evidence. On this level, evolution has been a massive success, with  vast amounts of experimental and fossil evidence making the theory almost universally accepted. And yet, the question of why these evolutionary trails should exist at all, let alone be so abundant, remains largely unanswered.

Evolution of life is just one of the fields in which these trails appear.  Another surprising example comes from what is perhaps the biggest technological revolution of our time: artificial intelligence (AI). The goal of AI  is to create systems capable of thinking, learning, and solving complex tasks. Many of these tasks, like vision or language understanding, are trivial to humans but have proven extremely challenging for computers. Attempting to tackle the task by analyzing the problem and writing the rules that define the solution have proven futile for even the simplest tasks. As it turns out, even for trivial tasks like walking or grasping, our ability to perform the task does not translate to understanding how the task is done.

To understand  why this problem is so hard, let's go back to the landscapes of possibilities. Consider, for example, the landscape of all possible strategies for playing chess. This landscape will be made of all the chess strategies that can ever  be conceived. Once again, in this terrain, the more similar two strategies are, the shorter the distance between them, and the better a strategy is, the higher its altitude.  The landscape will have peaks and valleys, and the peaks will  be the strategies of the  best chess player that could ever exist. The valleys will contain the worst chess strategies. This is an immense landscape, for the number of chess strategies is infinite, and all but a tiny fraction are terrible. Hence, it will have a vast area of low valleys and sparse tiny peaks.  Trying to guess the right solution to this infinite landscape is hopeless, like blindly dropping a pin from space and hoping it will fall on the top of Mount Everest. We can, of course, find a chess grandmaster and ask them to write their strategy into computer code, but as it turns out, none of them can.

And chess is a relatively simple game with only a handful of possible moves. Imagine the landscapes of all the ways you can drive a car, diagnose a disease, or write a poem.  The possibilities landscapes for each of these tasks are so vast that, for any practical purposes, it's infinite, containing large amounts of low valleys and  tiny peaks of true intelligence and skill.

For this reason, AI  seems for decades to lag hopelessly beyond basic human intelligence.  The main breakthrough that transformed AI from science fiction to the reality of ChatGPT and self-driving cars is based on a method named gradient descent. The approach is simple:  Given a task, start with a random solution, test it, to see how it works (it will work badly). Next, change each of the parameters of the solution by a tiny amount and see if it improves the solution if it does, keep this change. Repeat this tweaking again and again, adding up the changes, and after a large number of tiny steps, you will get a solution that can match and even exceed the best human expert (like in the case of chess).  This method is like a blind man traveling complex landscapes by touching the ground around him and always choosing  the direction of the highest upward slope.   Once again, this approach is both trivial and implausible. Trivial because random tweaking of existing solutions and keeping the changes that work is an obvious way to improve anything. And yet it's easy to see why this should  never work. Try to improve anything in the real world by only small tweaks, and you will  end up with a slightly better system, but rarely, if ever, a completely new method. Similarly try to try to climb a mountain or cross a real terrain always stepping  upward, and you will end up a few steps away, standing on a minor bump with no way to go but down.

For that reason, while gradient descent methods were known long before the first computers, only a few believe it can work beyond  simple optimization tasks. The idea that a near-human level of reasoning can emerge from complete randomness using only tiny gradual changes, each leading to a tiny improvement, would have been considered impossible up to a few years ago.

And yet it works not only to specific cases like language or vision, but to every field, from playing chess or any other game, to driving a car, predicting the structure of proteins, or diagnosing diseases and inventing new materials. All learned simply by starting from one random solution and gradually climbing upward in tiny steps,  never going downward and never getting stuck along the way.

Once again, the question  of why so many different tasks and in so many different domains will have possibilities landscapes that share these trails remains unanswered.   How is it possible that we can travel from the earliest organisms to high level life forms on a single unbroken trail through an unimaginably  complex landscape? How is it possible that there is a single continuous trail leading from random matrices of numbers to intelligent AI capable of human-level reasoning?

The existence of the trails  is far from being a consensus; there are many who believe that the evolution of life was just a lucky coincidence, one bound to happen in an infinite universe, and that modern AI, as impressive as it might seem, can't actually learn much, let alone everything.

However, if we choose to accept these trails as the universal property, then the implications for these are immense. The evolution of life, society,  and technology can be considered as one flow moving through a landscape of possible worlds, following an ever climbing trail that seems to be boundless.