Abstract

Formal modeling tools are powerful but inaccessible. They require users to master rigid syntax and specialized thinking. We present a prototype exploring Live Theory - a design paradigm where digital infrastructure sensitively adapts to human intention, instead of the other way around.

Our platform enables researchers to build, modify, and integrate simple causal models through intuitive interactions. Users can i) “translate” models between different ontological frameworks, ii) explore “threads”; AI-suggested extensions that understand the modeling context, and iii) “steal” relevant mechanisms from other models. Alongside this, we discuss the notion of composition we are sensitively scaling which continues with our current work.

Through implementing these post-formal design patterns (i.e. patterns which can exploit context beyond the operative formalisms), we demonstrate how AI can make rigorous modeling admit more intuitive expressions. By treating AI as attentive infrastructure rather than autonomous agent, we outline a path towards digital ecosystems that amplify human reasoning and agency instead of supplanting it.

Acknowledgements

Many thanks are due to the wonderful people involved in initiating, developing, and continuing this work. This writing is an individual articulation of collective efforts, and so is frightfully under-representative of our time together. Even so, I hope that the other Autostructures AISC10 participants feel a deep camaraderie with the content here, as it wouldn’t have happened without them.

I have benefitted massively from the care and attention of my team - Aayush, Sofi, and Sahil - as we envisioned and assembled the prototype into existence and beyond. In wrestling with the writing process, Jayson, Aditya Adiga and Aditya Prasad have offered much energy toward refinement and clarity. Many more involved with our burgeoning Groundless approach have also left fingerprints on this. To name a few, Abram, TJ, and Matt concretized abstract issues, while Keith helped me structure interviews. Along with multiple friends eager to endure semi-structured user tests; all shall be thanked and remunerated appropriately once the credit assignment problem is solved.

Introduction

New technological scaling paradigms are upon us - pre-training, post-training, and inference-time scaling, the distribution of moderate intelligence, and others. Increasing AI coherence and reliability gives us an opportunity to begin growing a digital ecosystem of attentive infrastructure. 

In the spirit of Sahil’s earlier work articulating aspects of so-called Live Theory, we present our prototype platform for fluidly modifying and translating formal causal models. Among the possible paradigm choices, this represents a subtly attentive alternative to traditional software design methods. Our intention is to illustrate a future of intuitive user interaction, backgrounded by formal digital structures that automatically adjust to user needs, with the central purpose of facilitating meaningful exchanges. 

In what follows, we illustrate a simple example of some design methods that go beyond our familiar formal digital experiences toward postformal design methods. In formal systems, users must adapt their thinking to predefined structures. In postformal systems, the infrastructure adapts to human intention; translating fluid thinking into formal structures only where necessary. More detail on postformal exchanges of meaning follow later, but for now, the scene should be set with concrete examples.

https://imgs.xkcd.com/comics/washing_machine_settings.png 

Within our strange design paradigm, our prototype begins to illustrate possibilities for scaling the composition of models. In particular, we investigate how cheap intelligence facilitates the arrangement of formal structures by interpreting intuitive instruction. This allows the structure of formal models to begin competently attuning to the creative and scientific vision of the modeler.

Our prototype platform is hosted here. A continuation of this work enables the handcrafted composing (and sharing) of functional views based on long-form documents. For the eager, express your interest in participating in our grand and subtle experiments using this form, or jump to our trailer of the workflow!

Prototype Design

Although the process by which we converged on various decisions ranged from the regimented to the whimsical, we outline some key considerations here. These decisions affected both the prototype’s current state as well as the aspects of Live Machinery we hope to elucidate. Since it may be inevitable that all content will become liquid potential, our central task consisted of determining which structural elements must persist in a digital platform; insofar as implementing anything requires adherence to some structures. Despite being able to break grammar rules frequently with no misunderstanding, we still require some structure in grammar to make precise a particular meaning.

Since fluidizing insensitive structure is our task, how might browsing through formal models contribute to this? In the limit, we would like to visualize the various potentially composable model states. This is especially difficult given that an arbitrarily complex composition procedure is still an open research direction, but we can at least begin to define what brings us closer to this goal^[1]. 

To the extent that it’s currently plausible, we prioritize operating directly with a model’s afforded insights, as opposed to the object level artefacts like its code. Likewise, we aim to insert anticipatory inferences in the places where a user recognizing a good contribution is more appropriate than generating one. If we can generate a model simplification, addition, or larger composition that fulfills the user’s intentions, this adds fluidity to their experience.

Let’s see the current state in more detail and connect this to the broader vision.

Postformal Design Patterns

The Topos Institute has been developing formal conceptual modeling methods from which we draw considerable inspiration. Here, modelling simply refers to creating artifacts for understanding some (conceptual) phenomena in the world. To this end, our prototype visualizes three postformal design patterns. We have restricted ourselves to causal loop diagrams for this prototyping process, but are excited to explore the broader horizons for meaning-facilitation.

Pattern 1: Translate

Translate is our simplest pattern - allowing a researcher to interpret a different model in their ontology. For example, a psychologist engaging with some neuroscience models might be able to exploit some mechanism’s usefulness if there is sufficient similarity between particular functional regions. Translate bears the responsibility of carrying the meaning from one field to another in a way that is sensitive to the considerations of both. 

Pattern 2: Threads

Threads act as sensitive future modifications to the current model. Their function is to approximate the current intuition that a researcher is trying to model and gently offer single-step updates that are richly context-aware. If the researcher has some partially-formed direction in mind, there’s a free-form input to help steer the thread suggestions.

Under the hood, threads are constituted by a precise description of the working model and a structured series of inferences which interrogate it and reason about what modifications befit the model’s state. It then suggests ways to clarify, expand, or reconfigure accordingly. Eventually, this process would be made more sensitive to the researcher’s context and workflow and the complete library of models.

Pattern 3: Steal

Stealing allows a researcher to selectively compose aspects of another causal model with theirs. Users can specify what interests them about the new model and how they imagine a synchronization happening. In the background, we compose a linkage of the two models by connecting the relevant nodes in a manner that emphasizes the selected behaviour. By selecting models to compose based on intuitions about how the two behaviors might mix holistically, we begin to enable more postformal stealing (like an artist).

Composing in Patterns

As a process, some aspects of composition remain elusive. Even still, our three design patterns aim to begin scaling the composition process directly. In particular, note that:

i) Translate acts as a form of context-switching, by merging semantic features of one model’s ontology into another. This involves essentially no structural composing at this point, but could later.

ii) Threads act as further inspiration for either clarity or more details by removing and adding nodes, or restructuring accordingly. These sparks of composition are only minor step-wise edits to the overall behaviour, but they are tuned around expected user preferences.

iii) Stealing acts as composing some chosen components together. This is a skillful and ongoing endeavor which is only provably achievable within rigid contexts. As such, it is currently automatically performed with accuracy which depends on the scale of the operative causal models. Therefore, this accuracy can be expected to scale with increases in intelligence.

It is subtle, but there are two key model-to-model relations shown here: i) the interfacing edges (red arrows), and ii) the shared alphabet. As a result, we have the early whispers of composition in two complementary senses - roughly along syntactic versus semantic lines. 

First, there is the combining of two models, the linkages of which must be composed to suit the intent of the connection. Composition here can be read as a syntactic property of the network, facilitated within a carefully constructed arena. Broadly, composition as combining models constitutes the building of a formal structural mechanism around an intended holistic behaviour. 

Second, we must deal with the arbitrariness of changing contexts such that there is a coherent semantic preservation throughout. More simply, one model might be able to use the mechanism of another without needing a particular relation between model contexts. Here, we compose the insights of one model into the domain of another. Broadly, this constitutes attention to the model's local context of application, from which domain-specific insights are discovered to be inclusively generalizable. 

Okay, but what does this have to do with formal and postformal things exactly?

Patterns of Sensitivity

To contrast the three similar terms in one breath, a (pre-formal) model ‘c’ exists in a yet-to-be-formalized state and can be given a formal structural bridge to another model, in accordance with some theoretician's postformal intuition. Once the formal bridge is enacted, this helps to verify the intuition as appropriately applicable.

Importantly, as we relinquish our dependence on insensitive structure, we are empowered to engage more of our intuitions with helpful, sensitive structure. One doesn’t need to be an expert in causal loop modeling to be able to think in causal terms. We naturally identify causal relations, and the diagrams help us extend that inclination by expanding the scope and precision of our causal links.

Broadly, we want an ecosystem where researchers can i) create structures (models) that are true to their inspiration, ii) share them with people non-modularly, and iii) find inspiration from the aesthetics and ideas of others and iv) fluidly engage with them without constraints imposed by insensitive structure. Creating such an ecosystem requires both a delicate touch and a rigorous grounding so that ideas can be conceived and crafted naturally, while they are constituted and transformed properly.

Balancing these opposing design incentives comes at the cost of a slow and thoughtful approach; one which is rooted in precise formal machinery, and which aims at sensitive infrastructure.

Formal Approaches

World models can be represented with category theory, which provides an opportunity to leverage its precise mathematical nature to perform various compositions. Specifically, causal loop diagrams can be viewed as a mathematical structure (called a double theory). Such a representation allows for the arrangement of meta-mathematical arguments; precisely-structured comparisons of the suitability of different mathematical toolkits along various axiomatic criteria.

Regardless, there are multiple notions of composition in the landscape of categories, though we won’t go into detail about what categories are lest we delve into notation that shrouds the main point^[2]. For our purposes, category theory provides a rich toolset to perform compositions in a formal, systematic manner. New methods are being discovered that allow a wider perspective on composition to show itself (across dynamical systems, in various ways).

Although powerful, these formal approaches are notably constrained; they are brittle and cumbersome. The structures in question often need to be carefully and inflexibly crafted to adhere to certain conditions that make them difficult to apply to real-world situations. We need a means of sensibly adapting these formal approaches around the nuance of reality, centered on the meaning communicated behind the notation. For this, we need post-formal design.

Enlivened Machinery

This project has been an exploration into a new design philosophy oriented around intelligent machinery. If increasingly adaptive systems continue to pervade, we need a design paradigm that maintains the core features of our meaning-making ecosystem. To this end, we present some reflections from our work.

Our goal has been to help “inspire this ecosystem of research which can competently caretake the widescale-exchange of adaptive artefacts”. First though, let's give a sketch of this paradigm and then identify some themes noticeable throughout.

The Paradigmatic Shape of Live Theory:

To navigate this design space, we can turn to a spatial metaphor in the form of a diagram. If we can assume that highly adaptive machinery will become pervasive, we can extrapolate broader features of the resulting ecosystem. To articulate a meaningful exchange within a highly-generative space, we can describe the process of actualizing a potential.

Here, we can imagine two peers interacting to create a website, where the source is a user and the target is the site host. Instead of a traditional static site, highly adaptive systems allow partially-specified websites to take the place of unchanging code, with the unfinished elements arriving just in time to meet the user as they arrive. To generate the optimal interface requires rich contextual information from both peers. As the source peer conveys their preferences and makes decisions, this narrows the range of relevant interfaces as they intersect with the intended communications of the host.

1. The aim of our technology should be to facilitate meaningfulness, not approximate it. Live Machinery facilitates meaning-making between peers by serving as infrastructure rather than simulacrum; as the sensitive background substantiator of ideas instead of mind-imitator in the foreground. This distinction is crucial: approximation aims for an artificial interlocutor, while facilitation creates a medium for synchronicity.
    
2. Facilitating potential meaning between two living beings requires careful actualization. In this scheme, a meaningful exchange between living beings exists first as potential - ideas not yet fully formed, intuitions not yet articulated - which itself is regarded as a first class entity. Our goal is to work with this potential, providing just enough structure to help actualize its meaning without constraining and distorting it.
    
3. Actualizing a meaningful exchange means finding a method of connecting these potentials in the manner most agreeable to both parties involved. At a technical level, this would be performed by using the minimal inference power required to produce the intended structure. This preserves the agency of the human actors while reducing the risk of overreach or overeagerness.

So, what are we to do with this idea of meaning-facilitation? As the future of technology brings advanced inference capabilities to every flash drive, what can be done with this power? Structuring an understanding of this environment is far from trivial, but there are a few related principles for good design whose conjunction lays the foundation for a distributed intelligent ecosystem.

Postformal Exchanges of Meaning

One counterintuitive aspect of designing Live Machinery is in reconciling its meaning-facilitation aim with the traditional iterative design cycle. With regards to our prototype, the potential idea of a modeler is met with a just-in-time inference of what would be meaningful within the context of their causal loop model. Paradoxically, we must both converge on an interface design, and enable the design to take on a multitude of forms.

As the potential is actuated through both a user’s deliberate choices and the subtle inferences made possible through increasing sensitivity, this spatial navigation framework gives us a helpful reference for envisioning possible actuations and how to nurture them. It provides a landscape for interpreting where to establish deterministic boundaries on the creative scope involved in translating from the informal to the formal^[3]. If we can identify which inferences require which cognitive processes, we can encourage surprising, synchronicity-spirited actuations with minimal compute (or, generally bureaucratic) cost.

In this paradigm, we can begin to talk about our decisions in terms of the clarifications they afford at the designed artefact level, while exploiting two directions of inference power to converge on a final user interface through anticipated intermediary structuring.

Parts of a Postformal Ecosystem

To articulate the wider vision of a postformal digital ecosystem, we can position this project within the other attempts to build out snippets of Live Theory. This work is in the middle of a chain of processes dealing with formalisms in a similar manner.

1. Extraction: To begin, we can extract formal structures from informal expression. For example, when colleagues discuss a complex system verbally, key causal relationships might be identified and extracted as formal elements. In the case of formal models, this could be a multimedia transcript of a conversation or causal loop model specifying any given system mechanism. 
    
2. Composition: Once extracted, these formal elements can be composed into coherent models. However, unlike composition in the domain-specific sense which requires rigorous adherence to precisely defined rules, postformal composition aims at fluidity for the user, especially in spaces where testing your intuition of formal transformations can be done quickly and cheaply.
    
3. Modification: Rather than requiring users to make changes within rigid constraints, this ecosystem would enable users to express modifications in natural ways, with the system translating these intentions into the appropriate formal adjustments. This represents a postformal version of our traditional mathematical operators, insofar as these modifications map rigorously-defined elements to underspecified, but valid alternate constructions.
    
4. Distribution: Finally, these models become truly valuable when they can be shared, combined, and evolved collectively. Our prototype explores new norms for model sharing that preserve provenance while encouraging creative recombination through the library of enlivened models. Our future goals also include a more robust credit assignment tree of model inspiration ancestry. 
    

More specifically, once we have i) extracted some formal structures from the world, we can then make sensible ii) compositions of them such that we can make meaningful arrangements of their contents. From those composed artefacts, we can begin scaling coherent iii) modifications of the underlying formalism through postformal instruction and human discernment. Meanwhile, artefacts in interpretable interim states can be iv) distributed through enlivened forums in accordance with v) local norms – from trivial conventions to complex customs.

Each of these extracting-, composing-, modifying-, distributing-, and norm-adapting steps can be accomplished within the peer-to-peer approach, rather than in a traditional sequential pipeline. Just as different themes in a conversation are relevant for different listeners, the contextual dialogue each listener might wish to extract could match their interests directly. As such, the form the particular step takes depends entirely upon which peers are involved and what meaning they intend to communicate.

This project has focused on visualizing the process of intelligently composing these models.

Future Directions and Limitations

We can extend a few of our prototype’s features to help exemplify the principles of Live Machinery. As a benefit of the paradigm’s commitments, it seems that our current limitations are positioned to strongly benefit from the foreseeable trend of increasing model adaptivity. In designing directly for the future, we can consider how some prototype features could scale in response to this trend. Each feature and its associated potential future version exemplify various means of making the prototype more live.

Although these features are not yet the focal point of our current work, you can see some similarity in our trailer hinting at what’s to come!

Model Library Semantics

Future users might explore the library of models in new ways. Beyond browsing a flat catalog, we can enable more sensible navigation through the conceptual territory. Currently, we can search models by their title and node names, but we could imagine sifting through the library by more nuanced characteristics captured within an on-the-spot structural understanding. Searching through models according to their structure could involve measuring their proportion of balancing or reinforcing loops as indicators of overall homeostasis or specific behaviors.

With semantically richer causal loop diagrams comes more information on their modelling accuracy and predictive power. The rising precision and speed of inference capabilities promise better relative structural understanding. This enables the most relevant connections to be centered; perhaps represented as navigation through visual semantic graphs. As such, edits adding loops to the starting conditions that either balance or reinforce the model dynamics can be more easily understood in terms of their overall effects.

Browsing Conceptual Spaces

To navigate through the branching possible model states created as options arise, we can envision more meaningful thread-wise composition. At the moment, our steal and thread patterns aim to bring an identifiable contribution to the working model from some inspired mechanism. In the future, the richer semantics and faster inference capabilities could deliver on-demand simulations of anticipated compositions, including coherence checks for meaningful arrangements. For both our steal and threads patterns, choosing various compositions moves us through the aforementioned cones of potential toward actuating the meaningful model being envisioned. The extent of each inferential step - in terms of novelty, added accuracy, or another measure - plays some comparable role toward realizing the actuation. As we interpret and choose among these steps, what does it mean to be informed as we choose between the branching possibilities?

Multiple tradeoffs present themselves when considering how to make a conceptual space browsable, such as ensuring there is enough parseable information which is neither too dense nor uninformative. Alongside the scaling of browsable conceptual spaces, we could likewise scale the navigable empirics of the potential-to-actuation process to maintain our capacity for oversight. In pursuing this, we expect that scaling the search processes of meaning facilitation will depend on a deep appreciation of the subtle inference components that have been made algorithmically feasible.

Interactivity for Intuitive Actuation

Finally, we could ingrain both subtle and more explicit interactivity techniques to evoke napkin-sketch-level inspiration. Whereas currently the doodling nature of the Loopy implementation allows minor edits, this could be enhanced both subtly and explicitly. Subtly, we could extend the diagrammatic aspect of the interface by dissolving restrictive distinctions between formal and informal input methods. Written text could be read as both notation and instruction and its functioning can be interpreted within the context of the entire workspace. This would smoothen the user interactions with what hopefully eventually appears to be an intelligent page, or some spirit of understanding within it. More explicitly, we could enable postformal adjustments of the interface itself. Capturing custom macros by exploiting inferable functionality can streamline the process of shaping the modelling environment around the workflow of the user. In essence, we aim to scale fluid interactivity.

As with any designed process, there is a complex ebb and flow of considerations. Ahead lies the multitude of possible choices to explore. The path forward is shaped by a modeler's inspiration and success criteria. How can we use the widespread adaptivity of moderate machine intelligence to take helpful steps toward the natural endpoints of our ideas?

Snags on the Path to Liquidity 

While the transition toward liquid content is underway, there remain some key obstacles to fluidizing what has been made formal. One key technique enlivening this new paradigm is made possible through the transformer architecture of (large) language models. However, current LLMs are plagued by unreliable outputs. This may not be a critical issue when writing in natural language, but it can be especially difficult to build flexible software pipelines whose components rely on LLMs for their structure.

Critically though, this contingency is directly met with our expectation that frontier intelligent machinery will increase in capabilities and reliability. As such, we have begun scaling those aspects of user interaction which are otherwise thought not to scale. To be more specific, consider the fundamental challenges of composition. Stealing functional model components means that they must be integrated in a way that is truthful to its inspiration. As such, the complexity of compositions available correlates with the underlying LLM capabilities.

1. ^{^}

    There are multiple instances where composition is a solved process given sets of domain-dependent constraints such as functions, certain dynamical systems, or simple causal models. However, the fact composition persists as both property and process throughout various mathematical and artistic realms is a centerpiece of the human-centered craftsmanship we want to emphasize. 

2. ^{^}

    For the interested, much of the basics of category theory can be found in this text, or this article. 

3. ^{^}

    Sampling over this bounded cross-section of interface options also tracks a description of the variability permitted by this inference procedure over many trials, wherein a tighter distribution of interface samples presumably means more inferences were required to group the resulting actuations, as opposed to a wider range implying fewer inferential stepping-stones preceding it. This plays an important role in deciding how and which anticipated inferences to represent when providing automatic recommendations.