tl;dr: How does an agent decide whether it is worth doing work to get something, as opposed to not working and not getting it? I'm a neuroscientist and my lab did an experiment to explore this question. Rats did voluntary work to earn water in a closed economy, while the wage rate varied.  Rats showed characteristics of rational economic agents, which could be captured by a parsimonious utility model.  The main results are described below. Links to the original paper, data and code are provided here^[1]. 

The question

In the wild, animals generally have to work to get the things they need. Research on reward-based motivation has revealed a lot about how animals choose between two or more available (mutually exclusive) goal-directed actions, such as two different tasks that give food or water rewards. Other things being equal, animals will choose the actions that offer more reward, or require less effort, or deliver reward with shorter delay, or deliver reward with less uncertainty. Optimal Foraging Theory addresses another set of questions about explore-exploit trade-offs. When animals think the yield in their current foraging patch is sufficiently below the average yield in the global environment, they forego immediate foraging to go search for a better patch, taking into account how far the other patches are and how likely the yields in other patches have changed since they last checked.

But very little research has addressed the question of how animals decide whether to perform any of the available actions for some given reward, versus none of them; whether to keep foraging, as opposed to stopping. Perhaps it's simply homeostasis: they just keep doing whatever the best available option is, until they have enough. But the overall opportunity in the environment can vary drastically over time. The average yield of all available foraging patches, or the best-yielding of all available actions, can be lean in some years or seasons, rich in others. Or sometimes there is only one patch to forage in, or only one action that can yield the thing they need.  Even when there’s only one option on the table, animals still have the choice to take or refuse the option. How do animals decide when they have “enough”, and when it’s not worth working to get more?

The experiment

The rats lived in custom cages whose only source of water was a computer-controlled visual classification task. The rats could perform a “trial” at any time (request a visual stimulus and assign it to one of two classes), and receive a drop of water if they provide the correct answer. The rats succeeded only 80% of the time, which shows both that the task was difficult (well below 100% correct), and that they were trying (well above 50% chance). The rats did not do the task if there was no water reward or if they had unlimited free water, which shows that they only did trials because they were “paid”.  By this definition the task is considered work or “labor”^[2]. The expected amount of water per trial (probability of success x size of drop) is the “wage rate”. This represents – in a highly stylized way – the fact that animals must generally do something effortful or costly (such as forage) to get a resource they need (such as water). 

Every few weeks the size of the water-drops, and thus the wage rate, was randomly changed. This represents the fact that seasons or weather patterns can make a resource more or less abundant at different times. To ensure animal welfare, animals were checked daily; if any animal didn't consume enough water, lost weight, or showed any physical or behavioral signs of dehydration or distress it would have been removed from the study and given free water^[3]. However, the study only used drop sizes for which it had been previously determined that such interventions would not be required.

Within a few days after a change, rats adopted a new steady-state trial rate.  In nature, environmental conditions can last long enough that animals need to find a sustainable strategy in each condition. To ensure the observed strategies were sustainable, in the experiment the rats were monitored until it was verified that the number of trials, water consumed per day, and body weight were stable before changing to a new wage rate.

The observations

From reading the reward literature, one might think rats would be more motivated and therefore do more trials whenever the reward was larger. But that would be a bad strategy in nature. Animals have to increase the rate of  effortful goal-directed behavior (like water foraging) precisely when that behavior is least rewarded (as in a drought), if they are to meet their basic needs. Indeed, rats did more work when wages were low, compensating for the lower yield and always meeting their physiological needs. When the water-drops were very large, rats did very few trials per day.  In macroeconomic terms, this is called a backwards-bending labor curve, because the labor supply (willingness to perform work for pay) declined as wages increased. In human markets this is explained in terms of workers preferring to take wage increases at least partly in the form of increased leisure.  

It’s not surprising that rats would do as many trials as necessary to get the amount of water they needed. If this were the whole story, however, rats should have consumed the same total amount of water in all conditions, regardless of the size of the water drops. This was not observed.

Instead, rats were willing to work for much larger total amounts of water (up to 3x more) when the water was easier to get. In macroeconomic terms, this is called price elasticity of demand: consumers consume more of a commodity when it is cheaper. This finding was more surprising, because core survival needs like water are often assumed to be “inelastic”. Apparently when unlimited free water is available rats drink a lot more water than they actually need. Part of it is physiologically necessary, but most of it is optional or hedonic. But rats strategically self-limited their water consumption to just meet their basic needs when water was scarce, and indulged in optional extra water when it was abundant – while also taking it easy (doing less overall work). Pretty smart.

Taking inspiration from classical economic theory, a utility maximization model was proposed to quantitatively account for and normatively explain the rats’ total effort and total water consumption as a function of the water drop size. There are only two free parameters: one for a rat's satiety point for water, and one for its aversion to the work.  The model makes a number of testable predictions which the paper lays out in detail.

If one assumes the rat's choice to do a trial at any moment is a sigmoidal function of the instantaneous marginal utility, the model further qualitatively predicts when in time rats did the trials.^[4]  The shape of the marginal utility curve strikingly matched the activity profile of neurons known to be necessary and sufficient for driving water-consumption behavior in mice, providing a testable neural circuit hypothesis for how marginal utility of work for water might be computed.

Conclusion

Animals don't just choose between alternative ways to meet a need. They also regularly make the choice to refuse all available offers, even when unclaimed rewards are on the table. They decide this based on when they have “enough”. But their definition of “enough” is flexible and contextual – even for survival-essential resources.  When resources are scarce, they consume less and work more; in times of plenty they enjoy a bit of excess, but also take time off.

Related work and Future Directions

Generalization: The described study tested a highly domesticated strain of rats. One generalization of interest would be to see if the same results hold for wild rats, or if it is specific to species domesticated by humans. Another would be to see if the results hold for phylogenetically diverse species. Does rational economic utility maximization require a neocortex, or would the same results be found in birds? lizards? bees?  For the latter, one could further distinguish the individual decision-to-work from the hive-level organism decision-to-work. One would have to use a commodity other than water to test aquatic species but then: what about fish? shrimp? cephalopods? 

The overall approach used here could be applied to how animals decide whether and when it is worth doing work to gain other resources, like food.  It is speculated in the paper that the details would be different for food, because animals can store excess consumed energy as glycogen or fat, thereby buffering fluctuations in availability. By contrast, most animals including rats can't store much excess water in their tissues, and instead eliminate excess fluid as urine. Therefore, with food rats might to a greater extent rely on stored energy reserves to avoid excessive labor during food scarcity, and overconsume to replenish reserves during food abundance^[5].  

Given that excess water cannot be stored in body tissues, it is something of a mystery why the rats even want to consume excess water when they can. One testable hypothesis is that drinking extra water enables them to eat extra (dry) food, effectively trading a non-storable commodity for a storable one. This would be easy to test by measuring or limiting their food consumption in such an experiment.

Extensions: It would be quite interesting (but  complicated) to simultaneously model allocation of effort toward two or more competing goals, including how they interact dynamically. For example, the same experiment could be done where food pellets were also earned by another kind of labor, like lever-pressing. Then the rats could choose to work for water, work for food, or neither.  The utility functions would have to include cross-terms because the marginal utility of water increases with the amount of dry food consumed and vice versa.  The interpretation of marginal utility as an instantaneous determinant of motivation might be enough to recapitulate observed oscillation between bouts of drinking, bouts of eating, and other activities. 

Alternatively, if the cost for switching between tasks is significant, one might cast goal-switching as a generalization of patch-switching, and apply Optimal Foraging Theory.^[6]

Applications: In related work, it was shown that the observed gap between physiological need and hedonic drive for water can be leveraged to motivate animals to perform behavioral trials for fluid rewards without the need for water restriction. It turns out rats will consume exactly, but only, the physiologically necessary amount of water if it contains Citric Acid (CA), a harmless additive. Animals tested in the same environment described in the paper were provided with an unlimited supply of free CA water, and still performed substantial voluntary work for water rewards. This has been widely adopted as an animal welfare refinement in research settings.  From a theoretical standpoint it would be quite interesting to expand the utility  model to explain how rats distribute voluntary consumption between costly water and a free but inferior substitute commodity. If anyone has any ideas how to do the math, I'd be happy to share the data.

1. ^{^}

   For more details about the model, other extensions and predictions, alternative models, and a candidate neural mechanism, see the original paper, supplementary materials, data and code.

2. ^{^}

   This needs to be said because sometimes animals will play with toys or games just for fun; mice will even pay for the chance to run on a running wheel. So not all effort is labor.

3. ^{^}

   These study-specific checks were in addition to standard animal welfare measures, which included ample space, daily food replenishing, regular cage cleaning, a rodent-preferred light cycle, at least two toys, daily health observations, and and daily positive human contact whenever group housing was not possible. The baseline criterion for welfare is that animals are "bright, alert and responsive".

4. ^{^}

   It's a bit too complicated to fully explain here, but the model predicts that the marginal utility of doing one trial falls quite steeply with number of trials completed, correctly predicting that animals will dramatically slow down after just a couple dozen trials, even if the ingested water hasn't reached the bloodstream yet (which takes 10-20 min), and even though they may only have consumed 10% of the total water they need to fully restore hydration. This pattern was key for identifying a possible neural mechanism. See the paper for a more complete explanation.

5. ^{^}

   This might explain why humans need GLP-1 inhibitors: overconsumption when calories are abundant and cheap was likely adaptive for most of our evolutionary history; there hasn't been much time for selective pressure against the maladaptive consequences of obesity, and societal interventions supersede strictly biological selection pressures.

6. ^{^}

   All of the suggested generalizations and extensions were in fact proposed in a grant proposal, but the grant was not funded after a couple of rounds and the lab has moved on. So these experiments are not currently planned, at least in my lab.