LESSWRONG
LW

harsimony — LessWrong

Replying toA Simple Method for Accelerating Grokking

A Simple Method for Accelerating Grokking

This is interesting! I think it would be worthwhile to check the robustness of your conclusions by focusing on e.g. modular addition and trying larger/smaller models or changing the weight on the regularization term.

The spikes can be understood using centralizing flows, you may want to plot the top Hessian eigenvalues to clean the plot up:

https://centralflows.github.io/

Feedback on the plots: I would find them easier to read if training and validation were the same line. I.e. dashed lines at beginning for train, switches to solid lines for validation so you can see the full path of the model.

Some links you might find useful: https://www.lesswrong.com/posts/jusq6kyZ6XSrtW3Bf/paper-summary-omnigrok-grokking-beyond-algorithmic-data

https://openreview.net/pdf?id=9XFSbDPmdW

Replying toAnother Cost Disease? We are all capitalists now

harsimony1mo

Another Cost Disease? We are all capitalists now

One thing to recognize is that the cost of renting capital is the real interest rate. So for people owning capital, a higher real interest rate is good. But, as more and more people have capital to loan out, real interest rates should fall (because people compete). This isn't a cost disease per se.

An intro to the Tensor Economics blog

harsimony

4mo

The Tensor Economics blog covers the economics of producing text from language models at scale.

The posts themselves are wonderfully detailed but somewhat overwhelming. I want to provide a summary of their work that might act as a guide. Then I’ll tie in some complimentary data from SemiAnalysis.

Note that I am presenting their posts in a different order to make it flow better. Each section selects a small amount from each of the original posts. Ideally, this post gives you the scaffolding to read their posts in full, they have a lot more insight.

Preludes

Time is money

Roughly 50% of AI data center spending is on hardware. Other CapEx^[1] grows in proportion to hardware costs^[2].

Hardware depreciates... (read 3437 more words →)

Replying toRL-as-a-Service will outcompete AGI companies (and that's good)

harsimony5mo

RL-as-a-Service will outcompete AGI companies (and that's good)

I would count your consulting service as RLaaS essentially. I'll admit, RLaaS is a buzzword that obscures a lot. "Have AI researchers and domain experts iterate on current AI models until they are performant at a particular task" would be more accurate. Things I think this model will apply to:

Anything involving robots. Consider the journey to self driving cars with lots of human data collection, updating the hardware, cleaning the dataset, and tweaking algorithms. Any physical manipulation task that has to be economically competitive will need a lot of input from experts. Factory managers will need robots that operate under idiosyncratic requirements. It'll take time to iron out the kinks.
To a lesser

harsimony5mo

RL-as-a-Service will outcompete AGI companies (and that's good)

These are good points. I'm uncertain about what models will form the foundation of RLaaS. But I think your point about where the task-specific data teams are working is more important. Off the top of my head, I think there's 3 bins:

For a lot of programming tasks, big AI companies already have lots of expertise and users in-house, so I expect them to dominate production of code generation.
For some tasks like writing marketing copy, LLM's are already good enough at this. There's no business training models further here.
Most interesting are tasks that require lots of tacit knowledge or iteration. For example, getting to self-driving cars required a decade plus of iterating on

harsimony

5mo

Companies drive AI development today. There's two stories you could tell about the mission of an AI company:

AGI: AI labs will stop at nothing short of Artificial General Intelligence. With enough training and iteration AI will develop a general ability to solve any (feasible) task. We can leverage this general intelligence to solve any problem, including how to make a profit.

Reinforcement Learning-as-a-Service (RLaaS)^[1]: AI labs have an established process for training language models to attain high performance on clean datasets. By painstakingly creating benchmarks for problems of interest, they can solve any given problem with RL leveraging language models as a general-purpose prior. This is essentially a version of the CAIS model.

... (read 683 more words →)

Replying toQualified rights for AI agents

harsimony5mo

Qualified rights for AI agents

You may be interested in this series, especially the post on "three prong bundle theory": https://www.greaterwrong.com/s/EA2uNqKjmu2NzFhRx

One good framing is to consider the rights digital minds need in order to participate in a market economy. They need property rights, freedom of speech, freedom of association, and so on. By being able to participate in market exchange, digital minds may prefer to be part of society rather than fight against it. Comparative advantage is a particularly good reason to cooperate with others.

Market rights: https://splittinginfinity.substack.com/p/markets-dont-work-without-individual

My comment on personhood and the value of being punish-able: https://www.lesswrong.com/posts/4m2MTPass3Ri2zZ43/legal-personhood-three-prong-bundle-theory#bpavKPBwJbCJtv8QA

The Data Scaling Hypothesis

harsimony

6mo

This idea is already implicit in most people's thinking, but I wanted something I could link to.

I propose we use the data scaling hypothesis to refer to the claim:

The quality and quantity of data used to train an AI model is by far the most important determinant of model performance.

It's a subset of the Scaling Hypothesis, which encompasses other inputs like architecture, parameters, and compute. Though the scaling hypothesis already places substantial weight on data.

By data, I mean any sort of information external to the model. This includes data in the traditional sense (images, text, spreadsheets, etc.) and things like the reward an RL agent gets in an environment or verifiable answers... (read 266 more words →)

Replying toLegal Personhood - Three Prong Bundle Theory

harsimony6mo

Legal Personhood - Three Prong Bundle Theory

Good points on an important topic. Thank you for this series.

One thing I'd like to point out is that receiving this form of personhood is highly valuable. In other words, being punishable makes you more trustworthy and safer to engage with. So AI's and digital minds might voluntarily construct methods by which they can be punished for wrongdoing.

This right-to-be-sued is an important legal right. I covered some discussion on twitter about this in point 3 here: https://splittinginfinity.substack.com/p/links-15

The key quote from Issac King here:

If the laws are enforced on everyone equally, then people can safely interact with each other, knowing that they have recourse if they are wronged. But when one particular group of

... (read more)

Nicely written proposal, thank you.

In truth, I'm quite concerned with such a proposal being implemented. Part of this is, as you mentioned, the risk of lock in. In particular, a global entity with a monopoly on violence is alarmingly close to global authoritarianism. This is an existential risk all on its own.

Such an entity would have to:

Limit access to space (to avoid self-sustaining expansion faster than everyone else).
Limit/monitor the uses of AI and AI training.
Possess a credible monopoly on violence.
Surveil enough to prevent surprise attacks and overthrow attempts.

Considering the risk that such a system is corrupted or misaligned or permanent, I feel better about a future that emphasized freedom and acceleration of... (read more)

Replying toOn METR’s AI Coding RCT

harsimony7mo

On METR’s AI Coding RCT

I feel like people are dismissing this study out of hand without updating appropriately. If there's at least a chance that this result replicates, that should shift our opinions somewhat.

First, a few reasons why the common counterarguments aren't strong enough to dismiss the study:

I've been seeing arguments against this result based on vibes or claims that the next generation of LLM's will overturn this result. But that is directly contradicted by the results of this study, people's feelings are poor indicators of actual productivity.
On Cursor experience, I think Joel Becker had a reasonable response here. Essentially, many of the coders had tried cursor, had some experience with it, and had a lot

harsimony9mo

Sleep need reduction therapies

Yeah I think sleep probably serves other roles, I just don't see why those roles require 7 hours of sleep rather than say 5 hours.

I do agree that basic research is what will actually get sleep need reduction therapies to work at scale. I'm hoping that citizen science and discussion of the topic will encourage more work on this.

Replying toSleep need reduction therapies

harsimony9mo

Sleep need reduction therapies

Oh I see. So the hypothesis is "In a healthy animal, stress is a highly-informative signal that inhibits risk-taking. Sleep ensures the stress system continues to inhibit risk taking appropriately."

Makes sense. It's consistent with sleep deprivation raising the level of cortisol and the brain developing a tolerance to high levels of certain hormones.

Sleep need reduction therapies

harsimony

9mo

EDIT: the funding proposal I mentioned at the end is out! See here for more.

None of this is medical advice.

5 AM on a Saturday and I can’t go back to sleep. It’s not the first time, so I get up to write, I might as well use the time I’m given. My hangover is to blame, even a little alcohol the night before changes my sleep cycle.

You’d think it would ruin my day, but in some ways I feel better. I’m more alert, more anxious, more motivated. It’s like being more alive. Being more awake for only one day has me asking questions. Why did sleep evolve? Can we sleep less?

Why did

... (read 2859 more words →)

•••

On AI Scaling

harsimony

I’ve avoided talking about AI, mostly because everyone is talking about it. I think the implications of AI scaling were clear a while ago, and I mostly said my piece back then.

It’s time to take another crack at it.

Scaling laws

Scaling laws are a wonderful innovation^[1]. With large enough training runs, performance is often a smooth function of the inputs you used for training. This is true over many orders of magnitude change in the inputs.

Major inputs include:

Money
Compute
Training time
Dataset size
Dataset quality
Number of parameters
Training algorithm
Researcher hours

Many of these factors are related. Money gets spent buying more compute and researchers. Total compute is a function of training time.

Let’s look at the relationship between compute, dataset... (read 2205 more words →)

The economics of space tethers

harsimony

Some code for this post can be found here.

Space tethers take the old, defunct space elevator concept and shorten it. Rockets can fly up to a dangling hook in the sky and then climb to a higher orbit. If the tether rotates, it can act like a catapult, providing a significant boost in a location where providing thrust is expensive. Kurzgesagt has a nice explainer and ToughSF has a great piece explaining the mechanics and some applications.

Tethers make it cheaper to explore space, but how much cheaper? Let’s look at the benefits.

Tether materials and characteristic velocity

The key performance metric for the tether material is the characteristic velocity:

$V_{c} = \sqrt{\frac{2 T}{ρ}}$

Where T is the tensile strength... (read 2044 more words →)

Modifying Jones' "AI Dilemma" Model

harsimony

Code for some of the equations and plots can be found here.

Charles I. Jones’ paper The A.I. Dilemma: Growth versus Existential Risk tackles the question of whether or not to pursue AI development in the face of existential risk. I found the approach both interesting and accessible and I recommend it to people interested in understanding these sorts of problems.

I want to illustrate the model and suggest a few modifications. For the purposes of this post, we’re going to assume society can make a collective decision about AI (a big if!). We’re also going to assume that those decisions are made from the perspective of a total utilitarian social planner.

To start, we’re... (read 1599 more words →)

Some Thoughts On Using Auctions For Land Valuation

harsimony

Land value taxes are an attractive approach to public finance. However, some people argue that the difficulty of assessing land values makes land value taxation impractical. While I think assessment by governmental officials typically is good enough for practical purposes, it’s worth thinking about ways to do even better. I believe a promising avenue for this is auctions so I’d like to stimulate discussion on using them^[1]. This article will accomplish that by outlining a specific land valuing auction system. These ideas are tentative, not certain, so I welcome critiques, comments, questions, and suggestions from readers to help refine them.

Auction Theory

Thinking about auctions all started when I wondered how we could solve... (read 2507 more words →)

Current AI Models Seem Sufficient for Low-Risk, Beneficial AI

harsimony

Note: Mostly opinion/speculation, hoping to encourage further discussion.

Summary: Current AI models have demonstrated amazing capabilities. It seems feasible to turn them into usable products without much additional scaling. These models are probably safe, and have the potential to do a lot of good. I tentatively argue that their deployment should be encouraged, with reservations about accelerating risky AI development.

Current AI has potential

It feels like there is a new AI breakthrough every week. Driven by advances in model architecture, training, and scaling, we now have models that can write code, answer questions, control robots, make pictures, produce songs, play Minecraft, and more.

Of course, a cool demo is far from a product that can... (read 551 more words →)

Massive Scaling Should be Frowned Upon

harsimony

Summary: It seems at least possible that scaling AI systems (broadly construed) could create dangerously powerful agents. I consider methods to discourage groups from massively scaling AI systems with little regard for safety. Examining cultural, regulatory, and technological interventions, cultural approaches seem best suited to this goal in the near term.

Scale

For the purposes of this post, I am going to lump several things together when I talk about "scale". Some of the highly-scalable inputs to an AI system include:

Number of parameters
Training time
Total memory
Total compute
Dataset size
Total cost
Number of researchers
Total research effort

I'm not particularly concerned with the specific way that these resources can be deployed to increase AI capability, rather, it's important that there are... (read 1354 more words →)

LESSWRONG
LW

LESSWRONG
LW

harsimony

Sleep need reduction therapies

The economics of space tethers

Why I'm Optimistic About Near-Term AI Risk

Georgism in Space

harsimony

harsimony

An intro to the Tensor Economics blog

RL-as-a-Service will outcompete AGI companies (and that's good)

The Data Scaling Hypothesis

Sleep need reduction therapies

On AI Scaling

The economics of space tethers

Modifying Jones' "AI Dilemma" Model

harsimony

Sleep need reduction therapies

The economics of space tethers

Why I'm Optimistic About Near-Term AI Risk

Georgism in Space

harsimony

harsimony

An intro to the Tensor Economics blog

RL-as-a-Service will outcompete AGI companies (and that's good)

The Data Scaling Hypothesis

Sleep need reduction therapies

On AI Scaling

The economics of space tethers

Modifying Jones' "AI Dilemma" Model

Preludes

Time is money

Why did

Scaling laws

Tether materials and characteristic velocity

Auction Theory

Current AI has potential

Scale