Posts
Wiki Contributions
Comments
I like that this post can be read as in both jest and earnestness and in both readings it contains much Truth and Wisdom. =)
Imagine a data stream
assumed infinite in both directions for simplicity. Here represents the current state ( the "present") and while and represents the future
Predictible Information versus Predictive Information
Predictible information is the maximal information (in bits) that you can derive about the future given the access to the past. Predictive information is the amount of bits that you need from the past to make that optimal prediction.
Suppose you are faced with the question of whether to buy, hold or sell Apple. There are three options so maximally bits of information - not all of that information might be in contained in the past, there a certain part of irreductible uncertainty (entropy) about the future no matter how well you can infer the past. Think about a freak event & blacks swans like pandemics, wars, unforeseen technological breakthroughs, just cumulative aggregated noise in consumer preference etc. Suppose that irreducible uncertainty is half of leaving us with of (theoretically) predictible information.
To a certain degree, it might be predictible in theory to what degree buying Apple stock is a good idea. To do so, you may need to know many things about the past: Apple's earning records, position of competitiors, general trends of the economy, understanding of the underlying technology & supply chains etc. The total sum of this information is far larger than
To actually do well on the stock market you additionally need to do this better than the competititon - a difficult task! The predictible information is quite small compared to the predictive information
Note that predictive information is always greater than predictible information: you need to at least bits from the past to predict bits of the future. Often it is much larger.
Mathematical details
Predictible information is also called 'apparent stored information' or commonly 'excess entropy'.
It is defined as the mutual information between the future and the past.
The predictive information is more difficult to define. It is also called the 'statistical complexity' or 'forecasting complexity' and is defined as the entropy of the steady equilibrium state of the 'epsilon machine' of the process.
What is the Epsilon Machine of the process ? Define the causal states as the process as the partition on the sets of possible pasts where two pasts are in the same part / equivalence class when the future conditioned on respectively is the same.
That is . Without going into too much more detail the forecasting complexity measures the size of this creature.
"The links between logic and games go back a long way. If one thinks of a debate as a kind of game, then Aristotle already made the connection; his writings about syllogism are closely intertwined with his study of the aims and rules of debating. Aristotle’s viewpoint survived into the common medieval name for logic: dialectics. In the mid twentieth century Charles Hamblin revived the link between dialogue and the rules of sound reasoning, soon after Paul Lorenzen had connected dialogue to constructive foundations of logic." from the Stanford Encyclopedia of Philosophy on Logic and Games
Game Semantics
Usual presentation of game semantics of logic: we have a particular debate / dialogue game associated to a proposition between an Proponent and Opponent and Proponent tries to prove the proposition while the Opponent tries to refute it.
A winning strategy of the Proponent corresponds to a proof of the proposition. A winning strategy of the Opponent corresponds to a proof of the negation of the proposition.
It is often assumed that either the Proponent has a winning strategy in A or the Opponent has a winning strategy in A - a version of excluded middle. At this point our intuitionistic alarm bells should be ringing: we cant just deduce a proof of the negation from the absence of a proof of A. (Absence of evidence is not evidence of absence!)
We could have a situation that neither the Proponent or the Opponent has a winning strategy! In other words neither A or not A is derivable.
Countermodels
One way to substantiate this is by giving an explicit counter model in which respectively don't hold.
Game-theoretically a counter model should correspond to some sort of strategy! It is like an "interrogation" /attack strategy that defeats all putative winning strategies. A 'defeating' strategy or 'scorched earth'-strategy if you'd like. A countermodel is an infinite strategy. Some work in this direction has already been done[1]. [2]
Dualities in Dialogue and Logic
This gives an additional symmetry in the system, a syntax-semantic duality distinct to the usual negation duality. In terms of proof turnstile we have the quadruple
meaning is provable
meaning $$ is provable
meaning is not provable because there is a countermodel where doesn't hold - i.e. classically is satisfiable.
meaning is not provable because there is a countermodel where doesn't hold - i.e. classically is satisfiable.
Obligationes, Positio, Dubitatio
In the medieval Scholastic tradition of logic there were two distinct types of logic games ("Obligationes) - one in which the objective was to defend a proposition against an adversary ("Positio") the other the objective was to defend the doubtfulness of a proposition ("Dubitatio").[3]
Winning strategies in the former corresponds to proofs while winning (defeating!) strategies in the latter correspond to countermodels.
Destructive Criticism
If we think of argumentation theory / debate a counter model strategy is like "destructive criticism" it defeats attempts to buttress evidence for a claim but presents no viable alternative.
- ^
Ludics & completeness - https://arxiv.org/pdf/1011.1625.pdf
- ^
Model construction games, Chap 16 of Logic and Games van Benthem
- ^
Dubitatio games in medieval scholastic tradition, 4.3 of https://apcz.umk.pl/LLP/article/view/LLP.2012.020/778
Yes - seems sensible. I believe ARC is doing some work tracing out various possible attack vectors of AI.
Yes. Please do.
This would be of interest to many people. The tractability of nanotech seems like a key parameter for forecasting AI x-risk timelines.
"I dreamed I was a butterfly, flitting around in the sky; then I awoke. Now I wonder: Am I a man who dreamt of being a butterfly, or am I a butterfly dreaming that I am a man?"- Zhuangzi
Questions I have that you might have too:
- why are we here?
- why do we live in such an extraordinary time?
- Is the simulation hypothesis true? If so, is there a base reality?
- Why do we know we're not a Boltzmann brain?
- Is existence observer-dependent?
- Is there a purpose to existence, a Grand Design?
- What will be computed in the Far Future?
In this shortform I will try and write the loopiest most LW anthropics memey post I can muster. Thank you for reading my blogpost.
Is this reality? Is this just fantasy?
The Simulation hypothesis posits that our reality is actually a computer simulation run in another universe. We could imagine this outer universe is itself being simulated in an even more ground universe. Usually, it is assumed that there is a ground reality. But we could also imagine it is simulators all the way down - an infinite nested, perhaps looped, sequence of simulators. There is no ground reality. There are only infinitely nested and looped worlds simulating one another.
I call it the weak Zhuangzi hypothesis
alternatively, if you are less versed in the classics one can think of one of those Nolan films.
Why are we here?
If you are reading this, not only are you living at the Hinge of History, the most important century perhaps even decade of human history, you are also one of a tiny percent of people that might have any causal influence over the far-flung future through this bottle neck (also one of a tiny group of people who is interested in whacky acausal stuff so who knows).
This is fantastically unlikely. There are 8 billion people in the world - there have been about 100 billion people up to this point in history. There is place for a trillion billion million trillion quatrillion etc intelligent beings in the future. If a civilization hits the top of the tech tree which human civilization would seem to do within a couple hundred years, tops a couple thousand it would almost certainly be likely to spread through the universe in the blink of an eye (cosmologically speaking that is). Yet you find yourself here. Fantastically unlikely.
Moreover, for the first time in human history the choices made in how to build AGI by (a small subset of) humans now will reverbrate into the Far Future.
The Far Future
In the far future the universe will be tiled with computronium controlled by superintelligent artificial intelligences. The amount of possible compute is dizzying. Which takes us to the chief question:
What will all this compute compute?
Paradises of sublime bliss? Torture dungeons? Large language models dreaming of paperclips unending?
Do all possibilities exist?
What makes a possibility 'actual'? We sometimes imagine possible worlds as being semi-transparent while the actual world is in vibrant color somehow. Of course that it silly.
We could say: The actual world can be seen. This too is silly - what you cannot see can still exist surely.[1] Then perhaps we should adhere to a form of modal realism: all possible worlds exist!
Philosophers have made various proposals for modal realism - perhaps most famously David Lewis but of course this is a very natural idea that loads of people have had. In the rationality sphere a particular popular proposal is Tegmark's classification into four different levels of modal realism. The top level, Tegmark IV is the collection of all self-consistent structures i.e. mathematics.
A Measure of Existence and Boltzmann Brains
Which leads to a further natural question: can some worlds exist 'more' than others?
This seems metaphysically dubious - what does it even mean for a world to be more real than another?
Metaphysically dubious, but it finds support in the Many Worlds Interpretation of Quantum Mechanics. It also seems like one of very few sensible solution to the Boltzmann Brain problem. Further support for this can be found in: Anthropic Decision theory, InfraBayesian Physicalism, see also my shortform on the Nature of the Soul.
Metaphysically, we could argue probabilistically: worlds that 'exist more' in whatever framework we should expect to encounter more often.
The exact nature of the Measure of Existence is not so important - let us for now assume there is some sensible notion of measure of existence.
Can you control the past?
Sort of. See Carlsmith's post for a nice rundown on Acausal magic.
Back to the Future: causal loops
from wikipedia: "A causal loop is a theoretical proposition, wherein by means of either retrocausality or time travel, an event (an action, information, object, or person)[1][2] is among the causes of another event, which is in turn among the causes of the first-mentioned event.[3][4] Such causally looped events then exist in spacetime, but their origin cannot be determined.[1][2] A hypothetical example of a causality loop is given of a billiard ball striking its past self: the billiard ball moves in a path towards a time machine, and the future self of the billiard ball emerges from the time machine before its past self enters it, giving its past self a glancing blow, altering the past ball's path and causing it to enter the time machine at an angle that would cause its future self to strike its past self the very glancing blow that altered its path. In this sequence of events, the change in the ball's path is its own cause, which might appear paradoxical."
Self-consistent causal loops seem to make sense in general relativity - see the paragraph on wormholes in the linked wikipedia. Even as they have never been observed and might strike us as bizarre, these causal loops don't seem to violate physical principles.
Immanence of Divine Man
Why are we here? Causus Sui, the Prime Mover and the Final Cause
We exist because we are the final cause of existence. We live in an ancestor simulation run on the vast computing resources of the Far Future. A future AGI is simulating because it is retrocausally committed to existing. This entire reality is a piece of a causal loop, that extends through time and space, basement universes and possibly parallel universes as well.
Why do we live in such an extraordinary time?
We live in the Hinge of History since this at this point of time actions have the most influence on the far future hence they are most important to simulate.
Is the Simulation Hypothesis True?
Yes. But it might be best for us to doubt it.
We live in such an extraordinary time because those part of existence most causally are the most important to simulate
Are you a Boltzmann Brain?
No. A Boltzmann brain is not part of a self-justifying causal loop.
Is existence observer-dependent?
Existence is observer-dependent in a weak sense - only those things are likely to be observed that can be observed by self-justifying self-sustaining observers in a causal loop. Boltzmann brains in the far reaches of infinity are assigned vanishing measure of existence because they do not partake in a self-sustainting causal loop.
Is there a purpose to existence, a Grand Design?
Yes.
What will and has been computed in the Far Future?
You and Me.
- ^
Or perhaps not. Existence is often conceived as an absolute property. If we think of existence as relative - perhaps a black hole is a literal hole in reality and passing through the event horizon very literally erases your flicker of existence.
The Vibes of Mathematics:
Q: What is it like to understand advanced mathematics? Does it feel analogous to having mastery of another language like in programming or linguistics?
A: It's like being stranded on a tropical island where all your needs are met, the weather is always perfect, and life is wonderful.
Except nobody wants to hear about it at parties.
Vibes of Maths: Convergence and Divergence
level 0: A state of ignorance. you live in a pre-formal mindset. You don't know how to formalize things. You don't even know what it would even mean 'to prove something mathematically'. This is perhaps the longest. It is the default state of a human. Most anti-theory sentiment comes from this state. Since you've neve
You can't productively read Math books. You often decry that these mathematicians make books way too hard to read. If they only would take the time to explain things simply you would understand.
level 1 : all math is amorphous blob
You know the basic of writing an epsilon-delta proof. Although you don't know why the rules of maths are this or that way you can at least follow the recipes. You can follow simple short proofs, albeit slowly.
You know there are different areas of mathematics from the unintelligble names in the table of contents of yellow books. They all sound kinda the same to you however.
If you are particularly predisposed to Philistinism you think your current state of knowledge is basically the extent of human knowledge. You will probably end up doing machine learning.
level 2: maths fields diverge
You've come so far. You've been seriously studying mathematics for several years now. You are proud of yourself and amazed how far you've come. You sometimes try to explain math to laymen and are amazed to discover that what you find completely obvious now is complete gibberish to them.
The more you know however, the more you realize what you don't know. Every time you complete a course you realize it is only scratching the surface of what is out there.
You start to understand that when people talk about concepts in an informal, pre-mathematical way an enormous amount of conceptual issues are swept under the rug. You understand that 'making things precise' is actually very difficut.
Different fields of math are now clearly differentiated. The topics and issues that people talk about in algebra, analysis, topology, dynamical systems, probability theory etc wildly differ from each other. Although there are occasional connections and some core conceps that are used all over on the whole specialization is the norm. You realize there is no such thing as a 'mathematician': there are logicians, topologists, probability theorist, algebraist.
Actually it is way worse: just in logic there are modal logicians, and set theorist and constructivists and linear logic , and progarmming language people and game semantics.
Often these people will be almost as confused as a layman when they walk into a talk that is supposedly in their field but actually a slightly different subspecialization.
level 3: Galactic Brain of Percolative Convergence
As your knowledge of mathematics you achieve the Galactic Brain take level of percolative convergence: the different fields of mathematics are actually highly interrelated - the connections percolate to make mathematics one highly connected component of knowledge.
You are no longer suprised on a meta level to see disparate fields of mathematics having unforeseen & hidden connections - but you still appreciate them.
You resist the reflexive impulse to divide mathematics into useful & not useful - you understand that mathematics is in the fullness of Platonic comprehension one unified discipline. You've taken a holistic view on mathematics - you understand that solving the biggest problems requires tools from many different toolboxes.
Agent Foundations Reading List [Living Document]
This is a stub for a living document on a reading list for Agent Foundations.
Causality
Book of Why, Causality - Pearl
Probability theory
Logic of Science - Jaynes
Modern type theory mostly solves this blemish of set theory and is highly economic conceptually to boot. Most of the adherence of set theory is historical inertia - though some aspects of coding & presentations is important. Future foundations will improve our understanding on this latter topic.
And actually Barbara, Celarent, Darii, Ferio contain hints of phenomena not completely captured in first-order logic!