This is a special post for short-form writing by Dalcy Bremin. Only they can create top-level comments. Comments here also appear on the Shortform Page and All Posts page.
31 comments, sorted by Click to highlight new comments since: Today at 2:14 AM

(Quality: Low, only read when you have nothing better to do—also not much citing)

30-minute high-LLM-temp stream-of-consciousness on "How do we make mechanistic interpretability work for non-transformers, or just any architectures?"

  • We want a general way to reverse engineer circuits
    • e.g., Should be able to rediscover properties we discovered from transformers
  • Concrete Example: we spent a bunch of effort reverse engineering transformer-type architectures—then boom, suddenly some parallel-GPU-friendly-LSTM architecutre turns out to have better scaling properties, and everyone starts using it. LSTMs have different inductive biases, like things in the same layer being able to communicate multiple times with each other (unlike transformers), which incentivizes e.g., reusing components (more search-y?).
  • Formalize:
    • You have task X. You train a model A with inductive bias I_A. You also train a model B with inductive bias I_B. Your mechanistic interpretability techniques work well on deciphering A, but not B. You want your mechanistic interpretability techniques to work well for B, too.
  • Proposal: Communication channel
    • Train a Transformer on task X
      • Existing Mechanistic interpretability work does well on interpreting this architecture
    • Somehow stitch the LSTM to the transformer (?)
      • I'm trying to get at to the idea of "interface conversion," that by the virtue of SGD being greedy, it will try to convert the outputs of transformer-friendly types
    • Now you can better understand the intermediate outputs of the LSTM by just running mechanistic interpretability on the transformer layers whose input are from the LSTM
    • (I don't know if I'm making any sense here, my LLM temp is > 1)
  • Proposal: approximation via large models?
    • Train a larger transformer architecture to approximate the smaller LSTM model (either just input output pairs, or intermediate features, or intermediate features across multiple time-steps, etc):
      • the basic idea is that a smaller model would be more subject to following its natural gradient shaped by the inductive bias, while larger model (with direct access to the intermediate outputs of the smaller model) would be able to approximate it despite not having as much inductive bias incentive towards it.
        • probably false but illustrative example: Train small LSTM on chess. By the virtue of being able to run serial computation on same layers, it focuses on algorithms that have repeating modular parts. In contrast, a small Transformer would learn algorithms that don't have such repeating modular parts. But instead, train a large transformer to "approximate" the small LSTM—it should be able to do so by, e.g., inefficiently having identical modules across multiple layers. Now use mechanistic interpretability on that.
  • Proposal: redirect GPS?
    • Thane's value formation picture says GPS should be incentivized to reverse-engineer the heuristics because it has access to inter-heuristic communication channel. Maybe, in the middle of training, gradually swap different parts of the model with those that have different inductive biases, see GPS gradually learn to reverse-engineer those, and mechanistically-interpret how GPS exactly does that, and reimplement in human code?
  • Proposal: Interpretability techniques based on behavioral constraints
    • e.g., Discovering Latent Knowledge without Supervision, putting constraints?
  • How to do we "back out" inductive biases, just given e.g., architecture, training setup? What is the type signature?
    • (I need to read more literature)

Having lived ~19 years, I can distinctly remember around 5~6 times when I explicitly noticed myself experiencing totally new qualia with my inner monologue going “oh wow! I didn't know this dimension of qualia was a thing.” examples:

  • hard-to-explain sense that my mind is expanding horizontally with fractal cube-like structures (think bismuth) forming around it and my subjective experience gliding along its surface which lasted for ~5 minutes after taking zolpidem for the first time to sleep (2 days ago)
  • getting drunk for the first time (half a year ago)
  • feeling absolutely euphoric after having a cool math insight (a year ago)
  • ...

Reminds me of myself around a decade ago, completely incapable of understanding why my uncle smoked, being "huh? The smoke isn't even sweet, why would you want to do that?" Now that I have [addiction-to-X] as a clear dimension of qualia/experience solidified in myself, I can better model their subjective experiences although I've never smoked myself. Reminds me of the SSC classic.

Also one observation is that it feels like the rate at which I acquire these is getting faster, probably because of increase in self-awareness + increased option space as I reach adulthood (like being able to drink).

Anyways, I think it’s really cool, and can’t wait for more.

Sunlight scattered by the atmosphere on cloudless mornings during the hour before sunrise inspires a subtle feeling ("this is cool, maybe even exciting") that I never noticed till I started intentionally exposing myself to it for health reasons (specifically, making it easier to fall asleep 18 hours later).

More precisely, I might or might not have noticed the feeling, but if I did notice it, I quickly forgot about it because I had no idea how to reproduce it.

I have to get away from artificial light (streetlamps) (and from direct (yellow) sunlight) for the (blue) indirect sunlight to have this effect. Also, it is no good looking at a small patch of sky, e.g., through a window in a building: most or all of the upper half of my field of vision must be receiving this indirect sunlight. (The intrinsically-photosensitive retinal ganglion cells are all over the bottom half of the retina, but absent from the top half.)

I observed new visual qualia of colors while using some light machine.

Also, when I first came to Italy, I have a feeling as if the whole rainbow of color qualia changed

What's a good technical introduction to Decision Theory and Game Theory for alignment researchers? I'm guessing standard undergrad textbooks don't include, say, content about logical decision theory. I've mostly been reading posts on LW but as with most stuff here they feel more like self-contained blog posts (rather than textbooks that build on top of a common context) so I was wondering if there was anything like a canonical resource providing a unified technical / math-y perspective on the whole subject.

The MIRI Research Guide recommends An Introduction to Decision Theory and Game Theory: An Introduction. I have read neither and am simply relaying the recommendation.

i absolutely hate bureaucracy, dumb forms, stupid websites etc. like, I almost had a literal breakdown trying to install Minecraft recently (and eventually failed). God.

I think what's so crushing about it, is that it reminds me that the wrong people are designing things, and that they wont allow them to be fixed, and I can only find solace in thinking that the inefficiency of their designs is also a sign that they can be defeated.

God, I wish real analysis was at least half as elegant as any other math subject — way too much pathological examples that I can't care less about. I've heard some good things about constructivism though, hopefully analysis is done better there.

Yeah, real analysis sucks.  But you have to go through it to get to delightful stuff— I particularly love harmonic and functional analysis.  Real analysis is just a bunch of pathological cases and technical persnicketiness that you need to have to keep you from steering over a cliff when you get to the more advanced stuff.  I’ve encountered some other subjects that have the same feeling to them.  For example, measure-theoretic probability is a dry technical subject that you need to get through before you get the fun of stochastic differential equations.  Same with commutative algebra and algebraic geometry, or point-set topology and differential geometry.

Constructivism, in my experience, makes real analysis more mind blowing, but also harder to reason about.  My brain uses non-constructive methods subconsciously, so it’s hard for me to notice when I’ve transgressed the rules of constructivism.

There were various notions/frames of optimization floating around, and I tried my best to distill them:

  • Eliezer's Measuring Optimization Power on unlikelihood of outcome + agent preference ordering
  • Alex Flint's The ground of optimization on robustness of system-as-a-whole evolution
  • Selection vs Control as distinguishing different types of "space of possibilities"
    • Selection as having that space explicitly given & selectable numerous times by the agent
    • Control as having that space only given in terms of counterfactuals, and the agent can access it only once.
    • These distinctions correlate with the type of algorithm being used & its internal structure, where Selection uses more search-like process using maps, while Control may just use explicit formula ... although it may very well use internal maps to Select on counterfactual outcomes!
      • In other words, the Selection vs Control may very well be viewed as a different cluster of Analysis. Example:
        • If we decide to focus our Analysis of "space of possibilities" on eg "Real life outcome," then a guided missile is always Control.
        • But if we decide to focus on "space of internal representation of possibilities," then a guided missle that uses internal map to search on becomes Selection.
  • "Internal Optimization" vs "External Optimization"
    • Similar to Selection vs Control, but the analysis focuses more on internal structure:
      • Why? Motivated by the fact that, as with the guided missile example, Control systems can be viewed as Selection systems depending on perspective
      • ... hence, better to focus on internal structures where it's much less ambiguous.
    • IO: Internal search + selection
    • EO: Flint's definition of "optimizing system"
      • IO is included in EO, if we assume accurate map-to-environment correspondence.
    • To me, this doesn't really get at what the internals of actually-control-like systems look like, which presumably a subset of EO - IO.
  • Search-in-Territory vs Search-in-Map
    • Greater emphasis on internal structure—specifically, "maps."
    • Maps are capital investment, allowing you to be able to optimize despite not knowing what to exactly optimize for (by compressing info)

I have several thoughts on these framings, but one trouble is the excessive usage of words to represent "clusters" i.e. terms to group a bunch of correlated variables. Selection vs Control, for example, doesn't have a clear definition/criteria but rather points at a number of correlated things, like internal structure, search, maps, control-like things, etc.

Sure, deconfusing and pointing out clusters is useful because clusters imply correlations and correlations perhaps imply hidden structure + relationships—but I think the costs from cluster-representing-words doing hidden inference is much greater than the benefits, and it would be better to explicitly lay out the features-of-clusters that the one is referring to instead of just using the name of the cluster.

This is similar to the trouble I had with "wrapper-minds," which is yet another example of a cluster pointing at a bunch of correlated variables, and people using the same term to mean different things.

Anyways, I still feel totally confused about optimization—and while these clusters/frames are useful, I think thinking in terms of them would ensue even more confusion within myself. It's probably better to take the useful individual parts within the cluster and start deconfusing from the ground-up using those as the building blocks.

Why haven't mosquitos evolved to be less itchy? Is there just not enough selection pressure posed by humans yet? (yes probably) Or are they evolving towards that direction? (they of course already evolved towards being less itchy while biting, but not enough to make that lack-of-itch permanent)

this is a request for help i've been trying and failing to catch this one for god knows how long plz halp

tbh would be somewhat content coexisting with them (at the level of houseflies) as long as they evolved the itch and high-pitch noise away, modulo disease risk considerations.

The reason mosquito bites itch is because they are injecting saliva into your skin. Saliva contains mosquito antigens, foreign particles that your body has evolved to attack with an inflammatory immune response that causes itching. The compound histamine is a key signaling molecule used by your body to drive this reaction.

In order for the mosquito to avoid provoking this reaction, they would either have to avoid leaving compounds inside of your body, or mutate those compounds so that they do not provoke an immune response. The human immune system is an adversarial opponent designed with an ability to recognize foreign particles generally. If it was tractable for organisms to reliably evolve to avoid provoking this response, that would represent a fundamental vulnerability in the human immune system.

Mosquitoe saliva does in fact contain anti-inflammatory, antihemostatic, and immunomodulatory compounds. So they're trying! But also this means that mosquitos are evolved to put saliva inside of you when they feed, which means they're inevitably going to expose the foreign particles they produce to your immune system.

There's also a facet of selection bias making mosquitos appear unsuccessful at making their bites less itchy. If a mosquito did evolve to not provoke (as much of) an immune response and therefore less itching, redness and swelling, you probably wouldn't notice they'd bitten you. People often perceive that some are prone to getting bitten, others aren't. It may be that some of this is that some people don't have as serious an immune response to mosquito bites, so they think they get bitten less often.

I'm sure there are several PhDs worth of research questions to investigate here - I'm a biomedical engineer with a good basic understanding of the immune system, but I don't study mosquitos.

Because they have no reproductive advantage to being less itchy.  You can kill them while they’re feeding, which is why they put lots of evolutionary effort into not being noticed.  (They have an anesthetic in their saliva so you are unlikely to notice the bite.)  By the time you develop the itchy bump, they’ve flown away and you can’t kill them.

There's still some pressure, though. If the bites were permanently not itchy, then I may have not noticed that the mosquitos were in my room in the first place, and consequently would less likely pursue them directly. I guess that's just not enough.

There’s also positive selection for itchiness.  Mosquito spit contains dozens of carefully evolved proteins.  We don’t know what they all are, but some of them are anticoagulants and anesthetics.  Presumably they wouldn’t be there if they didn’t have a purpose.  And your body, when it detects these foreign proteins, mounts a protective reaction, causing redness, swelling, and itching.  IIRC, that reaction does a good job of killing any viruses that came in with the mosquito saliva.  We’ve evolved to have that reaction.  The itchiness is probably good for killing any bloodsuckers that don’t flee quickly.  It certainly works against ticks.

Evolution is not our friend.  It doesn’t give us what we want, just what we need.

People mean different things when they say "values" (object vs meta values)

I noticed that people often mean different things when they say "values," and they end up talking past each other (or convergence only happens after a long discussion). One of the difference is in whether they contain meta-level values.

  • Some people refer to the "object-level" preferences that we hold.
    • Often people bring up the "beauty" of the human mind's capacity for its values to change, evolve, adopt, and grow—changing mind as it learns more about the world, being open to persuasion via rational argumentation, changing moral theories, etc.
  • Some people include the meta-values (that are defined on top of other values, and the evolution of such values).
    • e.g., My "values" include my meta-values, like wanting to be persuaded by good arguments, wanting to change my moral theories when I get to know better, even "not wanting my values to be fixed"
      • example of this view: carado's post on you want what you want, and one of Vanessa Cosoy's shortform/comment (can't remember the link)

One of the rare insightful lessons from high school: Don't set your AC to the minimum temperature even if it's really hot, just set it to where you want it to be.

It's not like the air released gets colder with lower target temperature, because most ACs (according to my teacher, I haven't checked lol) are just a simple control system that turns itself on/off around the target temperature, meaning the time it takes to reach a certain temperature X is independent of the target temperature (as long it's lower than X)

... which is embarrassingly obvious in hindsight.

Well is he is right about some ACs being simple on/off units.

But there also exists units than can change cycle speed, its basically the same thing except the motor driving the compression cycle can vary in speed. 

In case you where wondering, they are called inverters. And when buying new today, you really should get an inverter (efficiency).

moments of microscopic fun encountered while studying/researching:

  • Quantum mechanics call vector space & its dual bra/ket because ... bra-c-ket. What can I say? I like it - But where did the letter 'c' go, Dirac?
  • Defining cauchy sequences and limits in real analysis: it's really cool how you "bootstrap" the definition of Cauchy sequences / limit on real using the definition of Cauchy sequences / limit on rationals. basically:
    • (1) define Cauchy sequence on rationals
    • (2) use it to define limit (on rationals) using rational-Cauchy
    • (3) use it to define reals
    • (4) use it to define Cauchy sequence on reals
    • (5) show it's consistent with Cauchy sequence on rationals in both directions
      • a. rationals are embedded in reals hence the real-Cauchy definition subsumes rational-Cauchy definition
      • b. you can always find a rational number smaller than a given real number hence a sequence being rational-Cauchy means it is also real-Cauchy)
    • (6) define limit (on reals)
    • (7) show it's consistent with limit on rationals
    • (8) ... and that they're equivalent to real-Cauchy
    • (9) proceed to ignore the distinction b/w real-Cauchy/limit and their rational counterpart. Slick!

(will probably keep updating this in the replies)

Just noticing that the negation of a statement exists is enough to make meaningful updates.

e.g. I used to (implicitly) think "Chatbot Romance is weird" without having evaluated anything in-depth about the subject (and consequently didn't have any strong opinions about it)—probably as a result of some underlying cached belief. 

But after seeing this post, just reading the title was enough to make me go (1) "Oh! I just realized it is perfectly possible to argue in favor of Chatbot Romance ... my belief on this subject must be a cached belief!" (2) hence is probably by-default biased towards something like the consensus opinion, and (3) so I should update away from my current direction, even without reading the post.

(Note: This was a post, but in retrospect was probably better to be posted as a shortform)

(Epistemic Status: 20-minute worth of thinking, haven't done any builder/breaker on this yet although I plan to, and would welcome any attempts in the comment)

  1. Have an algorithmic task whose input/output pair could (in reasonable algorithmic complexity) be generated using highly specific combination of modular components (e.g., basic arithmetic, combination of random NN module outputs, etc).
  2. Train a small transformer (or anything, really) on the input/output pairs.
  3. Take a large transformer that takes the activation/weights, and outputs a computational graph.
  4. Train that large transformer over the small transformer, across a diverse set of such algorithmic tasks (probably automatically generated) with varying complexity. Now you have a general tool that takes in a set of high-dimensional matrices and backs-out a simple computational graph, great! Let's call it Inspector.
  5. Apply the Inspector in real models and see if it recovers anything we might expect (like induction heads).
  6. To go a step further, apply the Inspector to itself. Maybe we might back-out a human implementable general solution for mechanistic interpretability! (Or, at least let us build a better intuition towards the solution.)

(This probably won't work, or at least isn't as simple as described above. Again, welcome any builder/breaker attempts!)

Quick thoughts on my plans:

  1. I want to focus on having a better mechanistic picture of agent value formation & distinguishing between hypotheses (e.g., shard theory, Thane Ruthenis's value-compilation hypothesis, etc) and forming my own.
  2. I think I have a specific but very high uncertainty baseline model of what-to-expect from agent value-formation using greedy search optimization. It's probably time to allocate more resources on reducing that uncertainty by touching reality i.e. running experiments.
    1. (and also think about related theoretical arguments like Selection Theorem)
  3. So I'll probably allocate my research time:
    1. Studying math (more linear algebra / dynamical systems / causal inference / statistical mechanics)
    2. Sketching a better picture of agent development, assigning confidence, proposing high-bit experiments (that might have the side-effect of distinguishing between different conflicting pictures), formalization, etc.
      1. and read relevant literature (eg ones on theoretic DL and inductive biases)
    3. Upskilling mechanistic interpretability to actually start running quick experiments
    4. Unguided research brainstorming (e.g., going through various alignment exercises, having a writeup of random related ideas, etc)
  4. Possibly participate in programs like MATS? Probably the biggest benefit to me would be (1) commitment mechanism / additional motivation and (2) high-value conversations with other researchers.

Dunno, sounds pretty reasonable!

Useful perspective when thinking of mechanistic pictures of agent/value development is to take the "perspective" of different optimizers, consider their relative "power," and how they interact with each other.

E.g., early on SGD is the dominant optimizer, which has the property of (having direct access to feedback from U / greedy). Later on early proto-GPS (general-purpose search) forms, which is less greedy, but still can largely be swayed by SGD (such as having its problem-specification-input tweaked, having the overall GPS-implementation modified, etc). Much later, GPS becomes the dominant optimizing force "at run-time" which shortens the relevant time-scale and we can ignore the SGD's effect. This effect becomes much more pronounced after reflectivity + gradient hacking when the GPS's optimization target becomes fixed.

(very much inspired by reading Thane Ruthenis's value formation post)

This is a very useful approximation at the late-stage when the GPS self-modifies the agent in pursuit of its objective! Rather than having to meticulously think about local SGD gradient incentives and such, since GPS is non-greedy, we can directly model it as doing what's obviously rational from a birds-eye-perspective.

(kinda similar to e.g., separation of timescale when analyzing dynamical systems)

It seems like retrieval-based transformers like RETRO is "obviously" the way to go—(1) there's just no need to store all the factual information as fixed weights, (2) and it uses much less parameter/memory. Maybe mechanistic interpretability should start paying more attention to these type of architectures, especially since they're probably going to be a more relevant form of architecture.

They might also be easier to interpret thanks to specialization!

I've noticed during my alignment study that just the sheer amount of relevant posts out there is giving me a pretty bad habit of (1) passively engaging with the material and (2) not doing much independent thinking. Just keeping up to date & distilling the stuff in my todo read list takes up most of my time.

  • I guess the reason I do it is because (at least for me) it takes a ton of mental effort to switch modes between "passive consumption" and "active thinking":
    • I noticed then when self-studying math; like, my subjective experience is that I enjoy both "passively listening lectures+taking notes" and "solving practice problems," the problem is that it takes a ton of mental energy to switch between the two equilibriums.
    • (This is actually still a problem—too much wide & passive consumption rather than actively practicing them and solving problems.)
  • Also relevant is wanting to just progress/upskill as fast and wide of a subject as I can, sacrificing mastery for diversity. This probably makes sense to some degree (especially in the sense that having more frames is good), but I think I'm taking this wayyyy too far.
  • My  for opening new links far exceeds 1. This definitely helped me when I was trying to get a rapid overview of the entire field, but now it's just a bad adaptation + akrasia.

Okay, then, don't do that! Some directions to move towards:

  • Independent brainstorming/investigation sessions to form concrete inside views
  • Commitment mechanisms, like making regular posts or shortforms (eg)

I recently learned about metauni, and it looks amazing. TL;DR, a bunch of researchers give out lectures or seminars on Roblox - Topics include AI alignment/policy, Natural Abstractions, Topos Theory, Singular Learning Theory, etc.

I haven't actually participated in any of their live events yet and only watched their videos, but they all look really interesting. I'm somewhat surprised that there hasn't been much discussion about this on LW!

Is there a case for AI gain-of-function research?

(Epistemic Status: I don't endorse this yet, just thinking aloud. Please let me know if you want to act/research based on this idea)

It seems like it should be possible to materialize certain forms of AI alignment failure modes with today's deep learning algorithms, if we directly optimize for their discovery. For example, training a Gradient Hacker Enzyme.

A possible benefit of this would be that it gives us bits of evidence wrt how such hypothesized risks would actually manifest in real training environments. While the similarities would be limited because the training setups would be optimizing for their discovery, it should at least serve as a good lower bound for the scenarios in which these risks could manifest.

Perhaps having a concrete bound for when dangerous capabilities appear (eg a X parameter model trained in Y modality has Z chance of forming a gradient hacker) would make it easier for policy folks to push for regulations.

Is AI gain-of-function equally dangerous as biotech gain-of-function? Some arguments in favor (of the former being dangerous):

  • The malicious actor argument is probably stronger for AI gain-of-function.
    • if someone publicly releases a Gradient Hacker Enzyme, this lowers the resource that would be needed for a malicious actor to develop a misaligned AI (eg plug in the misaligned Enzyme at an otherwise benign low-capability training run).
  • Risky researcher incentive is equally strong.
    • e.g., a research lab carelessly pursuing gain-of-function research, deliberately starting risky training runs for financial/academic incentives, etc.

Some arguments against:

  • Accident risks from financial incentives are probably weaker for AI gain-of-function.
    • The standard gain-of-function risk scenario is: research lab engineers a dangerous pathogen, it accidentally leaks, and a pandemic happens.
    • I don't see how these events would happen "accidentally" when dealing with AI programs; e.g., the researcher would have to deliberately cut parts of the network weights and replace it with the enzyme, which is certainly intentional.

Random alignment-related idea: train and investigate a "Gradient Hacker Enzyme"

TL;DR, Use meta-learning methods like MAML to train a network submodule i.e. circuit that would resist gradient updates in a wide variety of contexts (various architectures, hyperparameters, modality, etc), and use mechanistic interpretability to see how it works.

It should be possible to have a training setup for goals other than "resist gradient updates," such as restricting the meta-objective to a specific sub-sub-circuit. In that case, the outer circuit might (1) instrumentally resist updates, or (2) somehow get modified while keeping its original behavioral objective intact.

This setup doesn't have to be restricted to circuits of course; there was a previous work which did this on the level of activations, although iiuc the model found a trivial solution by exploiting relu—it would be interesting to extend this to more diverse setup.

Anyways, varying this "sub-sub-circuit/activation-to-be-preserved" over different meta-learning episodes would incentivize the training process to find "general" Gradient Hacker designs that aren't specific to a particular circuit/activation—a potential precursor for various forms of advanced Gradient Hackers (and some loose analogies to how enzymes accelerate reactions).


What is the Theory of Impact for training a "Gradient Hacker Enzyme"?

(note: while I think these are valid, they're generated post-hoc and don't reflect the actual process for me coming up with this idea)

  • Estimating the lower-bound for the emergence Gradient Hackers.
    • By varying the meta-learning setups we can get an empirical estimate for the conditions in which Gradient Hackers are possible.
      • Perhaps gradient hackers are actually trivial to construct using tricks we haven't thought of before (like the relu example before). Maybe not! Perhaps they require [high-model-complexity/certain-modality/reflective-agent/etc].
    • Why lower-bound? In a real training environment, gradient hackers appear because of (presumably) convergent training incentives. Instead in the meta-learning setup, we're directly optimizing for gradient hackers.
  • Mechanistically understanding how Gradient Hackers work.
    • Applying mechanistic interpretability here might not be too difficult, because the circuit is cleanly separated from the rest of the model.
    • There has been several speculations on how such circuits might emerge. Testing them empirically sounds like a good idea!

This is just a random idea and I'm probably not going to work on it; but if you're interested, let me know. While I don't think this is capabilities-relevant, this probably falls under AI gain-of-function research and should be done with caution.

Update: I'm trying to upskill mechanistic interpretability, and training a Gradient Hacker Enzyme seems like a fairly good project just to get myself started.

I don't think this project would be highly valuable in and of itself (although I would definitely learn a lot!), so one failure mode I need to avoid is ending up investing too much of my time in this idea. I'll probably spend a total of ~1 week working on it.

New to LessWrong?