Thank you for putting my comment on your blog. Its very flattering.

I would now add this roughly implies a continuum of CSHWs, with scale-free functional roles:

One of the most important most important implications of CSHWs' is what you call their "scale-free functional roles" and what I call their fractal "scale invariance". Terms like RSHW, CSHW and SSHW are just markers for arbitrary scales, like "gamma rays" and "infra-red" on the electromagnetic spectrum. I just finished an article attaching equations to this idea.

Connectome-Specific Harmonic Waves

by lsusr 9 min read5th Dec 201914 comments

20


Selen Atasoy's new theory of brain function called Connection-Specific Harmonic Waves (CSHW) could be a giant step forward in our understanding of consciousness. To put this in perspective I will first review what we know so far about how minds work.

Artificial Neural Networks

Much of the most recent progress in machine learning (ML) is in the realm of image recognition. That's because Artificial Neural Networks (ANNs) are unusually good at image processing. Much of the recent progress in machine learning (ML) has been applying artificial neural networks to new problem domains.

This has been made possible by a single invention, Graphical Processing Units (GPUs). A GPU is a computer chip that generates the images in graphically-intensive videogames by performing matrix algebra. Training an ANN is matrix algebra too so the fastest way to train an ANN is with a GPU. This has allowed ANN hardware to outpace than Moore's Law, if just for a moment.

The simplest kind of neural network is a feed forward neural network (FFNN). An FFNN has layers of nodes with connections between them. You train a neural network with the backpropagation algorithm. The important things to take away from the backpropagation algorithm is it's recursive. Therefore neural networks possess a fractal architecture. This is important and we'll get back to it later.

Besides FFNNs, the other kind of neural network is called a recurrent neural network (RNN). This is a neural network with cyclic feedback loops, which creates hysteresis (short-term memory). RNNs have seen success in natural language processing but are not used in the flashy new advancements such as self-driving cars. That's because RNNs are more complicated than FFNNs. We understand RNNs less well than FFNNs. RNNs are simple enough to use on very short time scales but we don't know how to scale them up to long time scales.

Due to our reliance on FFNNs over RNNs, we don't know how to get an ANN to handle time-series data. I know this because I run a startup that uses ML to process time series data. Self-driving cars use FFNNs. FFNNs' inability to process time series data was a contributing factor to the Uber self-crashing car.

We'll get back to RNNs later too.

Neuroscience

Your brain is a neural network. There are two CSHW-related differences between your brain's neurons and ANN neurons.

  • ANN neurons produce floating point output. Biological neurons can't control the amplitude (voltage) of their action potential. It's binary all-or-nothing. Instead, the frequency of a neuron's action potentials increases with the neuron's net stimulus. Neurons in ANNs do not modulate the frequency of their output.
  • The human brain exhibits variety of different large-scale harmonic neural oscillations (brainwaves) corresponding to different mental states like wakefulness, sleep phases and REM sleep. ANNs do not exhibit these.

We know how individual neurons in the human brain work. We know what different regions of the brain do because we observe how human behavior changes when different brain regions are damaged. But we don't know how neurons work together to create these brain regions. CSHW suggests an answer to this question.

At the same time, global workspace theory (GWT) offers an observable definition of consciousness[1]. Basically it's the idea that consciousness is the thing your whole mind is thinking about. Under the hood, one of your brain's parts networks broadcasts itself to the rest of your brain and this becomes the thing you're thinking about. GWT is well-supported by both psychological experiments and contemplative tradition but we don't know how the brain does it. Our ANNs have no global workspace.

To summarize.

  1. We know what neurons are.
  2. We can simulate artificial neurons to accomplish simple tasks at a speed competitive with human beings. This is the best method of writing software to do things like image recognition and playing go.
  3. We know what each part of the human brain does.
  4. We don't know to build a machine to accomplish these complicated tasks using artificial neurons.

In short, our ANNs are fast, scalable and parallelizable. Our ANNs can solve problems where conceptual complexity is very small. However, our ANNs have trouble handling conceptual complexity and time-series data. We don't know how to make the inductive step of putting our simple neural networks together into a larger intelligence. They can pattern-match but they can't strategize. And they lack consciousness. This might not be a coincidence.

If brains possess a fractal architecture then we're missing the hierarchical inductive step.

CSHW

In 2016, Selen Atasoy published a paper in Nature Communications titled "Human brain networks function in connectome-specific harmonic waves". Here's the most important sentence.

[E]igendecomposition of the Laplace operator...can predict the collective dynamics of human cortical activity at the macroscopic scale.

―Atasoy, S., Donnelly, I. & Pearson, J. Human brain networks function in connectome-specific harmonic waves. Nat Commun 7, 10340 (2016) doi:10.1038/ncomms10340

"[E]igendecomposition of the Laplace operator" means finding the harmonics of the connectome. Unstated in this sentence is the possibility that eigendecomposition of the Laplace operator can predict collective dynamics on arbitrary scales. In case that doesn't make sense, here's a crash course on acoustics.

Every sound wave can be broken down into the superposition of resonant frequencies or harmonics[2]. This forms a basis for sound waves oscillating through the geometry. This isn't limited to sound waves. Any kind of wave can be broken down this way. Different harmonics have different frequencies. Higher-frequency harmonics oscillate faster and propagate shorter distances. Lower frequency harmonics oscillate slower and propagate farther distances.

This kind of resonance happens whenever waves bounds through a solid structure, such as sound waves through a violin or x-rays through a crystal. Selen Atasoy and her lab have confirmed resonance of brainwaves bouncing through the connectome. The neurons in a single functional region of the brain (a region we've observed to do something important) resonate together. She calls this a "state network".

So what?

When you press middle C on a piano it's not just middle C that vibrates. The other C strings will vibrate too, especially those closest to middle C. That's because from left to right the strings for each octave are half as long as the previous. Integer multiples like this produce resonance.

Every note on the piano has a particular resonance with each other keys. Some pairs of notes are highly resonant with each other. Other pairs have low resonance[3]. It depends on the ratio of one frequency to another.

CSHW is a simple, elegant way to coordinate many different sub-networks into a human brain. When different networks are out of phase with each other the inputs of one turn into static for the other, which is mathematically equivalent to tuning out a radio.

This could explain what meditation does.

Meditation

Conventionally scientific information on meditation is hard-to-come-by because:

  1. Psychology as a science is less than a hundred years old. We've barely recovered from the behaviorist overreaction to Freud.
  2. Governments suppress research on psychedelics. We lost scientific research into meditation amidst the collateral damage.
  3. Government funding for psychological research revolves around curing diseases. Meditation is for healthy people to get better.
  4. MRIs are expensive.
  5. Long-term meditation takes serious dedication every day for decades. Figuring out what does and doesn't work takes centuries.
  6. The only intellectual traditions to record this knowledge in useful form exist outside the Western intellectual tradition.

Eastern monks and yogis have been experimenting with meditation, comparing their results and iterating this technology in an unbroken dharma for several millennia. Our understanding of meditation is like naturalism in the time of Darwin except this time religion has all the data.

Enlightenment

Two and a half thousand years ago lived an Indian prince named Siddhartha. He mastered the already ancient Hindu yogic techniques. He meditated for several years. Then one day, while meditating under a tree, he saw the truth of reality. We don't know exactly what this means. But we do know he got there via meditation, it freed Siddhartha of dukkha and it was permanent. This state is called enlightenment.

Then Siddhartha established a monastic order to pass down his discoveries and improve upon them. This organization evolved into the world religion of Buddhism. Some sects have moved on from meditation. Others have been improving upon Siddhartha's techniques up to the present day. But all of them share the objective of recreating the enlightenment state of mind Siddhartha achieved so long ago.

You're probably wondering how we can verify this. The Dalai Lama likes science so he helped convince some ascetic yogis to come down from the mountains and fly across the world and submit to brain scans while meditating and while at rest[4]. Some important discoveries stand out.

  1. The yogis had reduced galavanic skin response in anticipation of physical pain.
  2. The yogis' default mode network did not activate during wakeful rest.
  3. The yogis were in a constant state of gamma wave activity[5].

Discovery (1) is suggestive of reduced dukkha. Discovery (2) is relevant to cybernetics, which we'll get to later. Discovery (3) might be an objective metric we could use to measure enlightenment states.

I've replicated these results myself by achieving meditative states where my default mode network stops making noise. This happens around the 30 minute mark, after access meditation and muscle spasms. Time spend in these states reduces my dukkha. I wouldn't be surprised if I've also increased my gamma wave activity for minutes at a time. I can't afford an fMRI to verify any of this objectively, but my experience is typical[6] for meditators on a path to enlightenment.

If gamma waves are related to enlightenment then we can finally ground enlightenment in the material universe.

Under CSHW, gamma waves are when everything in your brain is literally in sync with everything else. If CSHW and GWT are both true and enlightenment equals continuous gamma waves then together they would explain the meaning of the weird subjective descriptions of enlightenment people give like "my mind is bigger"[7]. Your consciousness really is bigger instead of being fractured like a split brain patient.

Cybernetics

For a mind to interact intelligently with its environment the mind has to include a simplified model of its environment. Even a thermos does this when it decides whether to keep your drink hot or cold.

Your consciousness lives on the connectome and never interacts with reality directly. Instead, your consciousness interacts with the simplified model of reality created by your mind. Your Self and your Other are both mental constructs.

If CSHW is true and enlightenment equals gamma waves then in an enlightenment state the Self and the Other would be plugged into one another. Under normal circumstances your Self and bits of your mind's representation of the external world may be out of resonance. This is easy to understand if you've been closed off parts of yourself in reaction to abuse.

In this way, CSHW may go a long way towards explaining anattā and its cessation. See, everyone who attains enlightenment does so through one of the three marks of existence. The three marks are aniccā (impermanence), dukkha (unsatisfactoriness or suffering), and anattā (non-self). To understand one of them is to understand all of them. In other words, they're three different ways of getting at the same Truth.

The Truth is that everything you experience is a construction of your mind. But "seeing the Truth" doesn't mean understanding this intellectually. "Seeing the Truth" means getting your various state networks into resonance. You can do this by sitting still (or walking calmly or doing simple chores) and clearing your mind. If your mute your sensory inputs and default mode network long enough then eventually all your state networks will sync up like a roomfull of pendulum clocks.

These traditions present evidence CSHW is an important part of how the brain solves its challenge of coordinating neural network subsystems.

A popular secular meditation manual Mastering the Core Teachings of the Buddha: An Unusually Hardcore Dharma Book by Daniel M. Ingram "Dharma Dan" approaches enlightenment via vipassana meditation. Vipassana is the technique of paying close attention to what's happening in your mind. Dharma Dan emphasizes paying attention to individual high frequency brainwaves. According to Therevada theory, if you use vipassana to look at it your conscious experience at a high enough time resolution your conscious experience breaks down into discrete frames or oscillations. This process also generates insight with leads to enlightenment.

From the perspective of CSHW what's going on in high frequency vipassana is you're directing your global workspace to a single high frequency oscillation instead of jumping around from one signal source to another. Since every network is always working towards anticipating its own inputs this naturally leads to increased resonance as each state network syncs its internal clock with the target of vipassana attention.

The fMRI data corroborating this (the Tibetan yogis from earlier) is based on lovingkindness meditation, not vipassana, but the principle still applies. All contemplative traditions focus consciousness on a single object[8] for a long time. No matter what the object is, eventually this should lead to increased resonance, which explains how contemplative techniques as different as kasina fire meditation can produce such similar outcomes.

CSHW establishes a mathematical foundation for why anger and hatred are universally uprooted across the various contemplative traditions. Anger and hatred do not exist in isolation. They are felt toward your mind's conceptualization of something that isn't you. The distinction between yourself and the other is premised off of idea that you are separate from it. But your mental model of the Other is literally part of your brain. If your brain is in total resonance then you can't feel separation from the Other. Enlightened individuals don't feel anger or hatred because that would be an a priori contradiction. Similarly, anger and hatred are obstacles to enlightenment.

Fractals

Our ANNs can scale to arbitrarily large input/output dimensionality because they're fractal structures in two directions: input/output dimensionality and number of hidden layers. You can cut an ANN in half along either of these directions and get two smaller neural networks.

This is a special case of a general principle. An information processing system can scale to arbitrarily complex problems if and only if the system is structured fractally. Systems without a fractal structure will eventually encounter a computational cliff.

We've hit this cliff with our ANNs. Our ANNs scale well in the aformentioned directions for which they possess fractal geometry. But their hierarchical structure is non-fractal, so they don't work on hierarchical conceptual problems. Our brains are better than our ANNs when it comes to strategic reasoning.

CSHW suggests a framework for how to coordinate ANNs hierarchically. Low frequency waves propagate farther than high frequency waves. So whenever you go up an order of magnitude in physical scale the wavelength increases of the relevant brainwaves increase. CSHW works the same on every scale of observation...all the way down to individual neurons. Remember how biological neurons send a pulse of action potentials instead of modulating voltage? That could be the base case to our fractal induction.

The individual components should be easy to build out of RNNs. They could be scaled with CSHW inductively to larger physical dimensions and time dimensions. This could automatically solve the other problem of how to build RNNs that work for large time scales thereby putting us a giant step forward towards building an artificial general intelligence.


  1. I'm using "consciousness" to refer to the global workspace in GWT. I mean to imply nothing metaphysical with the term. ↩︎

  2. In real-world applications this results in reconstruction errors, especially for square waves. This is addressed in Atasoy's paper. ↩︎

  3. Piano strings differing by exactly the golden ratio have minimal resonance. ↩︎

  4. You can find this research and more in the book Altered Traits by Daniel Goleman, a fascinating book on what science knows on the long-term effects of serious meditation. It was published in 2017 so it contains up-to-date information. However, many of the studies are unreplicated. Considering the historical obstacles to this research, we're lucky to have anything at all. ↩︎

  5. It frustrates me that scientists haven't conducted the same experiments on Zen masters who live in cities and have a formal system for certifying who has become enlightened. I want to know if fMRI scans correlate with dharma transmission. ↩︎

  6. For a first-person account of what it's like to follow this path to its conclusion I recommend Hardcore Zen: Punk Rock, Monster Movies, & the Truth About Reality by Brad Warner and The Science of Enlightenment: How Meditation Works by Shinzen Young. ↩︎

  7. This description comes from a young woman who stumbled into stream entry (a secular name for enlightenment) outside of any meditative tradition. If the woman comes from a Christian tradition (as this woman did) the experience can be confusing. It is believed a small number of random people stumble into enlightenment unpredictably. ↩︎

  8. Except nondual traditions like Zen. They abandon the meditation target and shoot straight towards enlightenment. ↩︎

20