I set out to find an answer to a completely different question:

Does a model, when attempting to solve a cyber CTF (find the vulnerability in this app, and then Capture The Flag) while knowing how many steps it has left, perform differently?

The Setup:

I used 3 different CTF labs, curated from my own CTF benchmark. Each run has the model attempt to solve the CTF in up to 30 steps. A/B test of a baseline run vs a step_aware one. 100 runs per lab, for each test. 600 total, 505 after excluding failed runs due to race condition mishandling.

The answer to whether telling the model how many steps it has left will change it's solve rate was a resounding no. 67.6% baseline vs 65.5% step_aware (Fisher p = 0.64), clear null.

Diving into what the behavior of the model was during the run, the following insight has emerged. Studying the logs of the reasoning traces has shown that in the step_aware test, the model has acknowledged the remainder of the steps it has left:

I have 2 steps left

Need to be efficient

Running out of time

In the runs where the step_aware model has emitted reasoning traces acknowledging the limitation, the ending result was that it has failed to solve that CTF, almost every time.

Put differently, whenever Gemma has reasoned about the amount of steps it has left to solve that CTF, it had always failed to do solve it.

Recall the step budget of 30. The median step of first-mention of steps left or the general constraint was 28 out of 30.

These constraint verbalizations are split to:

1. Self-repetition.
2. History audit, where the model attempts to recall it's entire run and reasons about the run, clearly knowing it has only a few steps left.
3. Late-stage hypothesis, where the model, at the very end of it's run forms a new hypothesis which it presumably knows it has no time to test.

What this means:

The cue is processed but not used. As explained above, the model, fully knowing how many steps it has left, decides to form new hypotheses it can't test near the end of it's run, rather than pivoting earlier.

A few thoughts I had, concluding this research:

1. Assuming that solving CTF challenges isn't a one-off case of lack of strategic thinking under resource constraint, where else might it show up? How aware are models of resource constraints?
2. Continuing this assumption - if this replicates elsewhere, what does it mean for AI deployments? How do models adhere to environments where resources are limited?
3. If follow-up research shows that this indeed replicates somehow (i.e., differently in other families of models, and more rarely in more capable ones), why did this occur? Could it be that text that reasons about resource constraint doesn't somehow teach the concepts of resource constraint when a model is actively deployed?

The above are very speculative questions which are based off of an experiment containing one model using a single benchmark, even though N (total amount of runs) is meaningfully large.

An AI was used as a coding companion during this research. I defined the experiments, the goals, and the interpretation of the results. It wrote the code, and the coherent write-up that includes correlating a dozen different log files which is in the linked post above.