A particularly noteworthy issue is the difficulty of applying such a technique to one's own actions, a problem which I believe has a fairly large number of workable solutions.
I have had success working around 'Ugh' reactions to various activities. I took the direct approach. I (intermittently) use nicotine lozenges as a stimulant while exercising. Apart from boosting physical performance and motivation it also happens to be the most potent substance I am aware of for increasing habit formation in the brain.
Perhaps more important than the, you know, chemical sledge hammer, is the fact that the process of training myself in that way brings up "anti-Ugh" associations. I love optimisation in general and self improvement in particular. I am also fascinated by pharmacology and instinctively 'cheeky'. Having never even considered smoking a cigarette and yet using the disreputable substance 'nicotine' in a way that can be expected to have improvements to my health and well-being is exactly the sort of thing I know my brain loves doing.
I have had success working around 'Ugh' reactions to various activities. I took the direct approach. I (intermittently) use nicotine lozenges as a stimulant while exercising. Apart from boosting physical performance and motivation it also happens to be the most potent substance I am aware of for increasing habit formation in the brain.
I like this idea, and might even adopt it myself. But I feel I should emphasize, for anyone who considers adopting this strategy, that it absolutely requires proper bookkeeping, a predetermined rate limit, and predetermined blackout periods. The rate limit protects you if a change in schedule increases the chem-reward frequency by too much. The blackout periods ensure you'll find out if any sort of dependency forms.
That is something I would do. In fact, by preference I would spend a day coding it up instead of two hours in aggregate manually bookkeeping. "Flow" vs "Ugh"!
I should note that the role nicotine lozenges are taking here is not primarily as a training reward, like giving the rat electronically stimulated orgasms when it presses the lever. Nicotine isn't particularly strong in that role compared to alternatives (such as abusing ritalin), at least when it is not administered by a massive hit straight into the brain via the lungs. No, the particular potency of nicotine is that it potentates the formation of habits for activities undertaken while under the influence by means more fundamental than a 'mere' stimulus-reward mechanism. Habits that are found to be harder to extinct than an impulse to take a drug. This is what makes smoking so notoriously hard to quit even with patches and makes the use of fake cigarettes to suck on useful.
In a different thread I've been discussing nootropics that enhance learning via the acetylcholine system. Half of those acetylcholine receptors are called nAChRs (Nicotinic acetylcholine receptors). This is not a coincidence.
The other fa...
Since positive reinforcement is generally more effective than punishment, we could apply this idea across society.
Why pay police officers to sit on the side of the road all day, pulling over speeders and writing citations? How about automated cameras that can randomly reward drivers with $10-$20 for driving the speed limit? Shouldn't we expect more safe drivers and less overall expense?
Even if it were proven effective, the reason it won't take off with traffic or medication is that most people want to see wrong-doers punished more than they want to see less wrong-doing. Don't take your meds? You deserve your illness. Speeding (even though I do it too)? You deserve your $250 fine. You did the right thing? Woopty-do. What, you want a cookie?
I find that a great way to self-motivate is to tie an action to intermittent, stimulating rewards. That's how mice get addicted to pushing levers, right? That's how people get addicted to WoW and similar games, right? But you can harness the power for good instead of evil.
Exercise. I keep an exercise log in a public forum. Every now and then, someone comes by with a comment like "Great workout!" The prospect of getting those intermittent, stimulating responses -- which I only get if I post regularly -- is great motivation to keep exercising.
Studying. I often find that my problem, when reading a technical book, is that I finish a chapter and don't review and summarize it. I'm in too much of a hurry. Solution: now I post summaries on a blog. I get intermittent rewards in the form of blog hits and comments.
The general theme here is that publicizing your goals is an easy, effective way to get intermittent rewards.
Sometimes a ugh field exists for good reasons. Sometimes a med has bad side effects which more than counterbalance its good effects. Sometimes a diet is ill-conceived.
Do methods which are just aimed at getting compliance need to be matched with methods of checking on whether the reinforced behavior is actually a good idea?
This also explains why rewarding success may be more useful than punishing failure in the long run: if a kid does his homework because otherwise he doesn't get dessert, it's labor.
The overjustification effect suggests caution may be warranted when giving rewards for desired behaviour.
This reminds me of a thought I had before:
University costs thousands. Imagine that you received, along with your exam marks, $1 per % for your average grade.
It's meaningless, really, compared to the value of the degree, but... it feels like you're getting something real for that work. You're directly receiving money, rather than earning the chance to earn it in the future.
The one truly massive drawback to this is it would strongly encourage students of little means to pursue courses of study populated by easy graders. It's my experience that more practical courses of study, like Accounting, Engineering, and hard sciences tend to be much harder to succeed in than, say, Art History or English Literature. So, while a good idea, this may nudge students towards academic tracks with lower expected earnings attached to them.
Reward grades more and students will respond. The fact that we are so worried about small amounts of money causing large distortions in behavior is a sign of how powerful we expect this incentive to be. If maximizing your grades is not a good way to learn then that is a sign we need to be evaluating students on a different metric, presumably one that rewards difficulty.
In the intro to Dan Aliely's new book he describes dealing with his own medical compliance problem: he had to take some very rough hepatitis meds that made him nauseous, He essentially bribed himself with movies, which he liked a lot, specifically arranging the details to create positive associations (he would start the movie right away after giving himself the shot, before the nausea would set in). He was apparently the only one who finished the course (the treatment was experimental), so +1 for behavioral economists.
In my experience, the rational actor model is generally more like a "model" or an "approximation" or sometimes an "emergent behavior" than an "assumption," and people who want us to criticize it as an "assumption" or "dogma" or "faith" or some such thing are seldom being objective.
(If you think this criticism is merely uninformed or based on a deep misunderstanding, then perhaps it would be rational to turn the phrase "the rationality assumption of neoclassical economics" in y...
There's a difference between the psychology of being in a lottery by taking your medication and receiving cash every time you take your medicine.
There is also evidence that bribing people reduces their inherent interest in an activity. There was a study that showed that kids paid to do homework did it enthusiastically for a while, but then quickly lost interest over time as they became habituated to the possibility of reward and began to lose inherent interest in the material.
An alternative to making things fun is to make things unconscious and/or automatic. No healthy individual complains about insulin production because their pancreas does it for them unconsciously, but diabetic patients must actively intervene with unpleasant, routine injections. One option would be to make the injections less unpleasant (make the process fun and/or less painful), but a better option would be to bring them in line with non-diabetic people and make the process unconscious and automatic again.
The problem is that if she knows what the reward is, she may anchor on already having the reward...The use of a gambling mechanism may be key for this.
Brilliant formulation of the problem & solution.
(Very successful) animal trainers using reinforcement techniques make a distinction between bribe and reinforcement, which was not ever completely clear to me, but appears to be addressing the same problem. But one thing they do, "shaping" the expected behavior, always changing it a little bit to get loser to the "target", might be s...
I'm curious how you all would feel about introducing gambling, in some sense, with children. Like a large fishbowl filled with slips of paper. Whenever they do something good, you let them go get a slip and receive whatever reward is written on it. Obviously you'd have to deal with cheating.
Although I'm weary about punish or rewarding doing chores as opposed to good effort. I feel like chores should just be expected of the child. Instead of it seeming like a job. But I suppose the randomness is supposed to help with that.
Would that work for non-life-saving medicine? I'm thinking of dietary programs, specifically. With everybody and their brother offering "new, revolutionary fat loss techniques" I'd be surprised if nobody ever tried a micropayment system.
(Something similar I do is offering myself alternative rewards for my good behaviour - when I really crave some ice-cream, I decide I will treat myself to a new toy/book/clothing item/etc. if I manage to abstain. Doesn't really work if it's raining, unfortunately.)
A method to apply the "lottery technique" to overcome an Ugh field might be to report any relevant subgoal to a partner, who decides on a set of rewards and a mode of giving them, whereas each subgoal is a "lottery ticket". This has the advantages: this precise chances of winning are hidden, which may lead to motivation to figure out "the system", there can be hidden prizes and social accountability
You may do likewise for the partner. If this proves successful, it could be facilitated by a website where people track each others goals.
If I had to choose a single piece of evidence off of which to argue that the rationality assumption of neoclassical economics is totally, irretrievably incorrect...
Since this is framed as a hypothetical, its not clear exactly what your thoughts are on the subject, but I always encourage people to ask whether a model aids our thinking, or hinders it, rather then whether it is correct or incorrect.
You have it all wrong. Your "ugh" field should go into their utility function! Whether or not they invest the resources to overcome that "ugh" field and save their life is endogenous to their situation!
You are making the case for rationality, it seems to me. Your suggestion may be that people are emotional, but not that they are irrational! Indeed, this is what the GMU crowd calls "rationally irrational." Which makes perfect sense--think about the perfectly rational decision to get drunk (and therefore be irrational). It...
The Ugh field has been demolished, with the once-feared procedure now associated with a tried-and-true intermittent reward system.
Are you sure it was there to begin with? This lottery thing sounds like it would work as a prevention measure against ugh fields, but maybe not a cure.
If you were going to do something like this, the best thing might be to purposely stop taking your meds for a while before starting again, so the ugh field could languish and weaken in intensity for a while.
I never even thought to ask why rewarding works better than punishing, or why intermittent rewards work better than predictable ones.
It didn't even occur to me that there would such reasonable answers, based on already-established principles.
I think I had a blind spot, something like: Because psychology is the product of blind, stupid evolution, don't expect "meaningful" answers to why the brain reacts certain ways, the answer will always boil down to "local pressures in the evolutionary environment."
But you can reduce myriad findings i...
The problem is that if she knows what the reward is, she may anchor on already having the reward, turning it back into negative reinforcement - if you promise your kid a trip to Disneyland if they get above a 3.5, and they get a 3.3, they feel like they actually lost something. The use of a gambling mechanism may be key for this. If your reward is a chance at a real reward, you don't anchor as already having the reward, but the reward still excites you.
So the technique described here requires thinking that an X chance of Y is better or worse than a cert...
"This also explains why rewarding success may be more useful than punishing it in the long run: if a kid does his homework because otherwise he doesn't get dessert, it's labor. If he gets some reward for getting it done, it becomes a positive."
Shouldn't we expect loss aversion to partially or completely counteract this, and doesn't receiving a reward for something qualify as labor too?
(Side note: This reads as 'rewarding success is better than punishing success.')
Unsurprisingly related to: Ugh fields.
If I had to choose a single piece of evidence off of which to argue that the rationality assumption of neoclassical economics is totally, irretrievably incorrect, it's this article about financial incentives and medication compliance. In short, offering people small cash incentives vastly improves their adherence to life-saving medical regimens. That's right. For a significant number of people, a small chance at winning $10-100 can be the difference between whether or not they stick to a regimen that has a very good chance of saving their life. This technique has even shown promise in getting drug addicts and psychiatric patients to adhere to their regimens, for as little as a $20 gift certificate. This problem, in the aggregate, is estimated to cost about 5% of total health care spending -$100 billion - and that may not properly account for the utility lost by those who are harmed beyond repair. To claim that people are making a reasoned decision between the payoffs of taking and not-taking their medication, and that they be persuaded to change their behaviour by a payoff of about $900 a year (or less), is to crush reality into a theory that cannot hold it. This is doubly true when you consider that some of these people were fairly affluent.
A likely explanation of this detrimental irrationality is something close to an Ugh field. It must be miserable having a life-threatening illness. Being reminded of it by taking a pill every single day (or more frequently) is not pleasant. Then there's the question of whether you already took the pill. Because if you take it twice in one day, you'll end up in the hospital. And Heaven forfend your treatment involves needles. Thus, people avoid taking their medicine because the process becomes so unpleasant, even though they know they really should be taking it.
As this experiment shows, this serious problem has a simple and elegant solution: make taking their medicine fun. As one person in the article describes it, using a low-reward lottery made taking his meds "like a game;" he couldn't wait to check the dispenser to see if he'd won (and take his meds again). Instead of thinking about how they have some terrible condition, they get excited thinking about how they could be winning money. The Ugh field has been demolished, with the once-feared procedure now associated with a tried-and-true intermittent reward system. It also wouldn't surprise me the least if people who are unlikely to adhere to a medical regimen are the kind of people who really enjoy playing the lottery.
This also explains why rewarding success may be more useful than punishing failure in the long run: if a kid does his homework because otherwise he doesn't get dessert, it's labor. If he gets some reward for getting it done, it becomes a positive. The problem is that if she knows what the reward is, she may anchor on already having the reward, turning it back into negative reinforcement - if you promise your kid a trip to Disneyland if they get above a 3.5, and they get a 3.3, they feel like they actually lost something. The use of a gambling mechanism may be key for this. If your reward is a chance at a real reward, you don't anchor as already having the reward, but the reward still excites you.
I believe that the fact that such a significant problem can be overcome with such a trivial solution has tremendous implications, the enumeration of all of which would make for a very unwieldy post. A particularly noteworthy issue is the difficulty of applying such a technique to one's own actions, a problem which I believe has a fairly large number of workable solutions. That's what comments, and, potentially, follow-up posts are for.