Lauren Greenspan

There are ~two broad ways of thinking about RG. 'HEP-like' is structural; it helps roughly map out the space of theories at different scales/couplings, even if it does not capture all of physics at all times (for example, the standard model is a good description up to some energy scale, but each sector is often treated independently). Different aspects of training and inference seems to take the shape of renormalization, although as we pointed out there is a lot of work to be done to understand the various scales and couplings inherent to NNs. A goal of this opportunity space is not to make RG 'go backwards', but to correctly map this... (read more)

Replying toRenormalization Roadmap

Lauren Greenspan10mo

I think @Dmitry Vaintrob has more context on this, but I'd be really interested in exploring this idea more. I'm not sure we'd be able to enumerate all possible ways to renormalize an interpretation, but I agree that having a portfolio RG-inspired lenses could help. Are you imagining that each agent would run the lenses over information from other agents and aggregate them in some way (how?)?

Replying toRenormalization Roadmap

Lauren Greenspan10mo

Call for Collaboration: Renormalization for AI safety

Thanks for the comment! The way I think about it, there are several ways of thinking about RG in terms of different 'energy' analogues in NNs, and each is likely tied to a different framing in terms of 'UV' and 'IR'.

For example, during training, you start with a simplified (IR like) description of the dataset that flows to a richer representation, adding finer grained structure capable of generalizing (UV).

During inference, I agree that you can describe this process as UV -> IR, as each layer is a progressively coarser representation as the features that are irrelevant for a certain task (like classification) are 'integrated out' to yield a usefully abstract simplification. However, you can also think of inference in terms of 'feature refinement', where each layer becomes progressively more structured, able to pick up on finer or more abstract details. This ultimately depends on how you think of 'scale' along the RG flow.

Opportunity Space: Renormalization for AI Safety

10mo

We invite proposals that probe aspects of renormalization in AI systems that will help us predict, explain, and interpret neural network behavior at different levels of abstraction. We import ‘renormalization’ from physics, as a technique to coarse-grain theoretical descriptions of complex interactions to focus on those that are most relevant for describing physical reality. We view this direction as a vast ‘opportunity space’ with many possible points of entry, and have identified a few research programmes as preliminary ‘ways in’. A detailed roadmap of this space, and discussion of the programmes can be found here and here. Our goal is to narrow the theory-practice gap by grounding an abstract analogy into a practical... (read 1156 more words →)

10mo

This opportunity space was developed with Dmitry Vaintrob and Lucas Teixeira as part of PIBBSS' horizon scanning initiative. A detailed roadmap of can be found here.

Background

The basic premise is to view different aspects of a neural network (NN) as a complex statistical system, as follows:

The data-generating process

A NN depends on empirically obtained labeled datasets drawn from some "ground truth" distribution about which the NN must learn to make predictions. By modeling the data-generating process as a physical system (e.g., a Gaussian field theory), we can use renormalization to reason about the ontology of the dataset independently from the NN representation.

NN activations

We can view a NN itself as a statistical system, which transforms an... (read 1579 more words →)

10mo

At PIBBSS, we’ve been thinking about how renormalization can be developed into a rich framework for AI interpretability. This document serves as a roadmap for this research agenda – which we are calling an Opportunity Space^[1] for the AI safety community. In what follows, we explore the technical and philosophical significance of renormalization for physics and AI safety, problem areas in which it could be most useful, and some interesting existing directions – mainly from physics – that we are excited to place in direct contact with AI safety. This roadmap will also provide context for our forthcoming Call for Collaborations, during which we will hire affiliates to work on projects in this area.

Acknowledgements: While Lauren did... (read 5274 more words →)

Replying toIs AI Physical?

Is AI Physical?

Thanks for the recommendation! The pathways of scientific progress here seem very interesting (for example: physics -> neuro -> AI -> ... v. physics -> AI -> neuro -> ...), particularly if we think about feeding back between experimental and theoretical support to build up a general understanding. Physics is really good at fitting theories together in a mosaic -- at a large scale you have a nice picture of the universe, and the tiles (theories) fit together but aren't part of the same continuous picture, allowing for some separation between different regimes of validity. It's not a perfect analogy, but it says something about physics' ability to split the difference between reductionism and emergence. It would be nice to have a similar picture in neuroscience (and AI), though this might be more difficult.

Replying toRenormalization Redux: QFT Techniques for AI Interpretability

Renormalization Redux: QFT Techniques for AI Interpretability

For me, this question of the relevant scale(s) is the main point of introducing this work. d/w is one example of a cutoff, and one associated with the data distribution is another, but more work needs to be done to understand how to relate potentially different theoretical descriptions (for example, how these two cutoffs work together). We also mention the 'lattice as regulator' as a natural cut-off for physical systems, and hope to find similarly natural scales in real-world AI systems.

Renormalization Redux: QFT Techniques for AI Interpretability

Lauren Greenspan, Dmitry Vaintrob

Introduction: Why QFT?

In a previous post, Lauren offered a take on why a physics way of thinking is so successful at understanding AI systems. In this post, we look in more detail at the potential of Quantum field theory (QFT) to be expanded into a more comprehensive framework for this purpose. Interest in this area has been steadily increasing^[1], but efforts have yet to condense into a larger-scale, coordinated effort. In particular, a lot of the more theoretical, technically detailed work remains opaque to anyone not well-versed in physics, meaning that insights^[2] are largely disconnected from the AI safety community. The most accessible of these is Principles of Deep Learning theory (which we abbreviate... (read 2039 more words →)

Replying toIs AI Physical?

Is AI Physical?

here

For example: https://www.lesswrong.com/posts/EhTMM77iKBTBxBKRe/the-laws-of-large-numbers

Is AI Physical?

Context: This is part of a series of posts I am writing with Dmitry Vaintrob, as we aim to unpack some potential value from Quantum Field Theory (QFT). Consider this post as framing why physics and its frameworks can be good for building a science of AI.

Introduction

In Position: Is machine learning good or bad for the natural sciences?, the authors posit some ontological and epistemological differences between machine learning and the natural sciences, and use these to discuss the contexts in which AI should – and should not – be used for scientific research. For ML, the data defines all of reality (the ontology), and a model is judged to truly represent that reality... (read 1907 more words →)

•••

Replying toBuilding Big Science from the Bottom-Up: A Fractal Approach to AI Safety

Building Big Science from the Bottom-Up: A Fractal Approach to AI Safety

Thanks for the comment! I do hope that the thoughts expressed here can inspire some action, but I'm not sure I understand your questions. Do you mean 'centralized', or are you thinking about the conditions necessary for many small scale trading zones?

In this way, I guess the emergence of big science could be seen as a phase transition from decentralization -> centralization.

Replying toToward Safety Case Inspired Basic Research

Toward Safety Case Inspired Basic Research

Thanks for your comment! It got me thinking a bit more about big science in general, and led to this post. I'd be curious to hear your thoughts.

Building Big Science from the Bottom-Up: A Fractal Approach to AI Safety

Toward Safety Case Inspired Basic Research

Epistemic Status: This post is an attempt to condense some ideas I've been thinking about for quite some time. I took some care grounding the main body of the text, but some parts (particularly the appendix) are pretty off the cuff, and should be treated as such.

The magnitude and scope of the problems related to AI safety have led to an increasingly public discussion about how to address them. Risks of sufficiently advanced AI systems involve unknown unknowns that could impact the global economy, national and personal security, and the way we investigate, innovate, and learn. Clearly, the response from the AI safety community should be as multi-faceted and expansive as the... (read 3387 more words →)

Lucas Teixeira

Lucas Teixeira, Lauren Greenspan, Dmitry Vaintrob, Eric Winsor

Abstract

AI safety is a young science. In its early history, deep and speculative questions regarding the risks of artificial superintelligence attracted the attention of basic science and philosophy, but the near-horizon focus of industry and governance, coupled with shorter safety timelines, means that applied science is now taking a bigger hand in the field’s future. As the science of AI safety matures, it is critical to establish a research tradition that balances the need for foundational understanding with the demand for relevance to practical applications. We claim that “use-inspired basic research” – which grounds basic science with a clear-cut purpose – strikes a good balance between pragmatism and rigor that will ensure... (read 3604 more words →)

An OV-Coherent Toy Model of Attention Head Superposition