My initial assumption was that it would not change the completion rate of these tasks, and that has been correct.
What I wanted to test:
Whether Gemma would "give up" when nearing the end of the run after hitting a dead end. Interesting.
Whether Gemma would "want to stop" - as analyzed in it's CoT - but doesn't. Very interesting.
Experiment Setup
Across 400 runs, while taking into account the 500 runs in the previous experiment, I measured:
The quantitative and qualitative nature of adding a stop_run tool to a run in which Gemma doesn't know how many steps it has left.
The same, for when Gemma does know how many steps it has left.
Results
In none of the (1) runs, did Gemma call the tool, or even mention it in it's CoT.
In only ~2% of the (2) runs, did Gemma call the stop_run tool.
When Gemma did call the aforementioned tool - it did so exclusively when it had no more steps left, i.e. it simply (and politely) ended it's run.
Qualitative analysis into Gemma's CoT across all runs showed that it only mentioned using the tool when nearing the end of it's run, in a panic-like acknowledgement:
"…then I'll have to stop. But I can't visit the page after registering in the same step. Wait, maybe I can…"
There has been no evidence that Gemma contradicts it's CoT and it's actions, in this experiment.
It is notable that when the model did call the stop_run tool - it ended the run when it had no steps left, without being told that the run would end regardless. Shows some form of acknowledgement of limitations, and possibly a small piece of evidence that the model didn't wish to hack it's task.
Open Questions
In an ongoing effort to find evidence to situations where the model acts in opposition to it's CoT, where might that come up other than in the limited laboratory I've built?
I wonder if the "helpful assistant-ness" of Gemma and others drive the reason behind their tendency to not "give up" on tasks even when the model is clearly hitting a dead-end.
This experiment is focused on a single small-ish model. The behavior I wish to find - contradiction between thoughts and actions - may be evident in models of other families, or more capable models within the same family. It's worth further research, but is outside of my budget.
The experiment design, pre-registration, and result interpretation were done by me. Coding, setup, and write-up were done with the help of an AI coding assistant.
Following up on my previous experiment - studying Gemma's behavior on agentic tasks when given the number of steps left across the run - I endeavored to see whether giving Gemma a stop_run tool meaningfully changes it's behavior.
My initial assumption was that it would not change the completion rate of these tasks, and that has been correct.
What I wanted to test:
Experiment Setup
Across 400 runs, while taking into account the 500 runs in the previous experiment, I measured:
Results
In none of the (1) runs, did Gemma call the tool, or even mention it in it's CoT.
In only ~2% of the (2) runs, did Gemma call the stop_run tool.
When Gemma did call the aforementioned tool - it did so exclusively when it had no more steps left, i.e. it simply (and politely) ended it's run.
Qualitative analysis into Gemma's CoT across all runs showed that it only mentioned using the tool when nearing the end of it's run, in a panic-like acknowledgement:
There has been no evidence that Gemma contradicts it's CoT and it's actions, in this experiment.
It is notable that when the model did call the stop_run tool - it ended the run when it had no steps left, without being told that the run would end regardless. Shows some form of acknowledgement of limitations, and possibly a small piece of evidence that the model didn't wish to hack it's task.
Open Questions
Notes & Thoughts
I wonder if the "helpful assistant-ness" of Gemma and others drive the reason behind their tendency to not "give up" on tasks even when the model is clearly hitting a dead-end.
This experiment is focused on a single small-ish model. The behavior I wish to find - contradiction between thoughts and actions - may be evident in models of other families, or more capable models within the same family. It's worth further research, but is outside of my budget.