Isn't punishment negative reward? Shouldn't punishment have the same effect as reward but just the opposite sign?
One hypothesis:
When reward happens, the subject gets curious about exactly what they did to earn the reward; they refine their behavior to more efficiently obtain it (explore → exploit mode). But when punishment happens, the subject gets aversive towards the whole general area around the punishment; they take opportunities to avoid the situation associated with it. If getting in a scuffle at school gets me punished at home, that's a good reason to avoid school or home (or telling parents about school) rather than avoiding scuffles.
In the sufficiently general case, the punished person feels they are being punished for being who they are rather than for any specific action. "I can't be myself around you, because you don't like me." This seems like a chance to lead into some pretty nasty stuff.
One thing I wonder about:
What are good ways to reinforce someone for being honest about making mistakes? If kid comes home from school and has the choice of whether to confess that they got in trouble at school, or lie about it, then punishing them if they confess will deter confession. There's a trope that kids stop telling their parents what happens at school because they take their parents' reactions to be aversive or at least embarrassing. "What'd you do at school, dear?" "Oh, nothing ..."
What are good ways to reinforce someone for being honest about making mistakes?
Kazdin addresses this in so far as he recommends building close trustful communication in general - not only rewarding confessions.
Another issue with punishment is that it amounts to an attempt to replace a behavior with nothing (as was pointed out the the main article). Behaviors exists because they have a function, but mere elimination means the function is not addressed. Tends to be better to replace the behavior with something that has the same function. For instance, replace an unwanted behavior that functions to get attention with a wanted behavior and give that attention, so that the wanted behavior fulfills the existing function.
And as you point out, punishment tends to train avoidance of the parent and sneakiness.
In summary, punishment has bad side effects and it's not a tool for building up a system of wanted behaviors.
Interesting. Indeed "rational parenting" should be only "parenting". My first thought was to argue that "rational parenting" is becoming a construction (via usage in prior posts) of a certain (class of) parenting style. But that wouldn't be right, so I fixed it.
Perhaps 'Optimal Parenting' or 'Evidence-Based Parenting' or just a normal parenting guide without any rational buzzwords.
If I see a title with 'evidence-based' in it, I have a good guess at what sort of methodology and epistemology I expect to see in it: systematic review with a focus on randomized trials & meta-analyses. This gives me a good start at interpreting it (positive results should be taken seriously, but nulls are not very informative because typically there is little acceptable data; cost-benefit reasoning will likely not be included).
While if a title includes 'rational' in it...
Erm, yes I think so too, I don't know why I offered that. I'm retracting the whole comment, it seems unnecessary.
One thing you did not discuss is fading. Fade tokens to just use praise and positive attention. Fade constant praise to occasional random praise. Random reinforcement makes a habit more robust and less prone to extinction. (I think of fading as moving toward occasionally reminding the kid that the behavior is evidence of his good character, his virtues, but I a not sure that is explicit in Kazdin's book.)
A central idea is catch them being good and reinforce. Then, after a period of constant reinforcement, fade. If the target behavior does not happen then you can't just catch them doing it. So, use methods to get it started: simulation, prompting, token reward charts. (Also, you might shape if starting with some existing behavior.)
Prompting is a technique to get more compliance to commands. Get close to the kid, speak calmly, touch, don't use a question. Avoid prompting more than a few times (3 or 4) without compliance (nagging). Instead, come up with a different strategy.
Better yet, encourage them to implement their own mental token economy:
''changing cognitive representations of rewards (e.g. making long term rewards seem more concrete) and/or creating situations of “pre-commitment” (eliminating the option of changing one’s mind later) can reduce the preference for immediate reward seen in delay discounting.[146]''
Thoughts in relation to AI learning...
Parents tend to be a bit obsessed with setting limits. Setting limits is sometimes necessary, but parents tend rely too much on reacting to limit crossings. If you trained a robot by only reacting to limit crossings then the robot might well spend all it's time bouncing off the limit.
Think of a limit as a border on a region of acceptable behavior. The Kazdin method relies on incrementally (in small behavior shaping steps) drawing the child toward the optimal point in that region of acceptable behavior. If you train a robot this way, then the robot will tend stay sufficiently close to the optimal location, well away from the limits.
This reminds me of my significantly downvoted post about AI needing a caregiver.
I think there is something to be learned from 'training natural intelligences' who also try to break out of the box so to speak. But most people here either don't see the connection or consider it obviously wrong.
The link on "AI needing a caregiver" links to your profile and I can't find the post about AI needing a caregiver.
You mention inner motivation, competing children, conflict/violent outbreaks. I don't think you yet have a proper analysis of this violent behavior.
I don't think of competition as bad in and of itself. Kids can compete to improve in the direction you want them to improve and you can direct sometimes this process merely by directing your attention toward the preferred behavior.
The violent conflicts are probably not caused by inner motivation. It probably the kids motivating each other's behavior. The problem is that, like inner motivation, it's not your behavior that is reinforcing the conflict, so it can be a bit harder to address. The dynamics might be the Patterson Coercive Cycle, only between two siblings rather than a parent and child:
http://www.pendletonpsych.com/doc/parent-child-coercive-cycle.pdf
I think the usual approach is to separate the kids. Why do you say "reinforce separating them"? You probably are not reinforcing the conflict by separating them. It probably puts conflicts on extinction if you put them both in time out.
Otherwise, you'd have to make sure the aggression does not get it's reward. That is possible but seems hard. Or you could come up with alternative behaviors that had the same function as the aggression, but I don't have any good ideas on how to do that, but you could come up with competitions that are less likely to lead to aggression.
Also, this Kindness Chart, or something like it might help:
Inner motivation is a different issue, I think. Typically you change the behavior to a better one that fulfills the same inner motivation.
Kindness Chart.
The kindness chart starts to show some effects. There is a specific valued reward once the sun is full: All children may cuddle and sleep in the family bed. They don't go for it very actively yet. But at the very least it make kindness more visible and avilable to reflection. I immediately reward kindness when I notice it.
You mention inner motivation, competing children, conflict/violent outbreaks. I don't think you yet have a proper analysis of this violent behavior.
Indeed. I do not really know why they escalate when they do. I have the feeling that it results from inner motivation to lead and control by the older against the raising resistence of the younger one the one hand and different (incompatible?) emotion regulation of both - namely the older continuously raises the level while the younger handles it relatively smart and cool until he suddenly has enough. Then boom. Trouble is this is a slow process which can go on for a long time below the radar and sometimes not happen at all, e.g. if they don't touch problematic areas esp. competive ones.
I don't think of competition as bad in and of itself.
Neither do I. But their competition is not a friendly one.
But maybe that is an idea I can follow up on. Not trying to shape cooperation (as they do have a lot of that), but reinforcing positive competition. But probably I have to start with simulation to get that started.
It probably the kids motivating each other's behavior. ... so it can be a bit harder to address.
Exactly.
I think the usual approach is to separate the kids.
I had and have to use it often enough. But separating them doesn't help. Except for the moment. As Kazdin writes: It comes too late. Whatever the cause, both will think they won (or at least didn't lose) and thus got their reward. So any punishment (time out) has no effect.
Why do you say "reinforce separating them"?
The only effect it does have is that they are sparated and I do not want to reinforce separation beteween them. I recognize that they can and do learn a lot from their interaction. I don't want to alienate them of each other. And it shouldn't be necessary. They can play and cooperate for hours - if the agree on a topic. And are in not too bad a mood.
Kindness Chart.
I will try it. I will just have to look out that it doesn't degrade into just another competition and superficial kindness.
I still don't think you are reinforcing separation. You are not giving them a tangible or intangible reward when they separate. Also, I don't see that the mere act of separating them will alienate them from each other.
But I can see that it's plausible that there might be a better strategy than separating them.
If I separate them they immediately switch to other objectives (reading a book, playing lego...) and gain reward from that. It extinguishes the joint play.
I agree that it doesn't alienate them. For that I'd have to reward them for avoiding each other.
The term punishment kind of tricky in this context. Kazdin is a behaviorist writing a parenting book. In behaviorism, the term has a different meaning from it's typical use by parents.
In behaviorism, "positive" means adding something, and "negative" means taking something away. A reinforcement increases a behavior, a punishment decreases a behavior. So in behaviorism efficacy is built into the definition, if it does not decrease behavior then it's not a punishment.
In parenting, punishments are typically used incorrectly, overused, have bad side effects. And, most importantly, it's often not the best alternative because research has found better alternatives. Also, parents are reinforced to punish because they tend to be rewarded with short-term reinforcement, so it can be part of a vicious cycle. What parents consider a punishment might actually be reinforcement, for instance (1) yelling at your kid can make you look like a vanquished foe (2) if the kid can divert the parent into a predictable punishment mode, the parent might not follow through on something that the kid wanted to avoid even more.
If I recall correctly, Kazdin's does not tell parents to never use short-term grounding (or withdrawal of privilege), and that's a negative punishment in behaviorism. He says long-term grounding is no more effective and just causes resentment with no extra benefit.
The term punishment kind of tricky in this context.
Kazdin clarifies these terms in the book. In particular he splits the behavior-changing part from the social/informational part.
In parenting, punishments are typically used incorrectly,..
See the section about punishment disadvantages in the OP.
Kazdin's does not tell parents to never use short-term grounding....
Indeed he is quite balanced on this point. The section on withholding reinforcements could have been more elaborate.
This is a review of The Everyday Parenting Toolkit: The Kazdin Method for Easy, Step-by-step, Lasting Change for You and Your Child by Alan E, Kazdin (all phrases in quotes below are from this book if not otherwise indicated). I was pointed to this book by tadamsmars comment on Ignorance in Parenting.
This is a post in the sequence about parenting. I also see some cross relations to learning and cognitive sciences in general. Kazdins advice also is not only applicable to children but to adults as well if you read the book with a mind open to the backing research (Kazdin actually gives some such examples to illustrate the methods).
Define the positive behavior you do want. Communicate this clearly and provide events that make it likely to occur. Praise any occurrence of the positive behavior effusively. Think about and communicate consequences beforehand. Use mild and short punishments (if at all). Provide a healthy environment.
Antecedents are ways to set up the behavior you want. "They come before the behavior in question." These are mainly prompts, e.g. "Please come over here". Also helpful are setting events, e.g. sitting together (quietly) improves chances of requests to stay calm (this is related to priming). Challenges - "I bet you can't do X" - increase the likelihood of the behavior (though no reason why this works is given).
Key points:
Think about the behavior you do want. Specify it (actually write it down), but use only positives. Make it precise (smart). Start with a single behavior.
In case of problematic behaviors you want to reduce specify the positive opposite (this is a word you will read very often in the book) of the behavior. Do not address the negative behavior (that will just reinforce that; more below). Instead divert attention of the child toward positively reinforced opposite behaviors.
Break this down into doable steps (same as for procrastination advice) and reward each one.
Kazdin proposes three methods to get the behavior you want: Shaping of existing behavior, simulating for new behavior and jump starting for difficult cases.
Shaping means starting from an initially small behavior like cleaning up just one piece and rewarding that and as this behavior increases slowly increasing required behavior and/or reducing reward.
Simulation really means simulation, i.e. trying out the wanted behavior in a game-like situation where formally nothing is at stake for the child but it can nonetheless earn a reward. This allows for learning behavior which otherwise doesn't occur. Beside changing frequent behavior to new behavior this can also be use for to learn e.g. to stay away from danger by using simulated danger (example: medicine in reach of small children).
Jump starting means "finding early behaviors that can lead to this end behavior" e.g. assume you want to have someone at your LW meetup but he refuses, instead of plain inviting him you could invite him to your place and have some LessWronger present, or propose to have a walk with have on the way to the meetup where you part. Both can lead to him getting in touch with the group but he always has clear exit options.
Don't move too fast. Only reduce reward or increase load when the changed behavior occurs reliably. Provide reward for progress not for end results. "The most important concept: Practice the behavior".
Consequences are what comes after a behavior. Kazin stresses that positive reinforcement is scientifically proven to work much better then negative reinforcement (aka punishment).
He focusses on two types of positive consequences:
Praise and Attention
He describes how and when praise and attention should be given as a reward. "Turn to notice something a child did, smile, touch approvingly". He details to praise smaller children effusively, add physical contact and to give quieter more intimate praise for teens.
Points system
Also known as "token economy" is a well known method but Kazdin hones this to work reliably.
First you need to
The difference to normal points systems (one example he rigorously deconstructs) is mainly in the timing and splitting of the rewards and the integration into the larger method.
When to apply consequences:
Apply "ommediately after the behavior." "Convey exactly what [the reward] is given for." "Give high-quality reinforcer." And give praise often at least initially. "Don't improvise consequences." Don't reduce normal parenting attention to the child. Kazdin stresses that repeated immediate feedback to positive behavior works better than grand rewards.
In short: Punishment works. Immediately but not for long. The child doesn't change its behavior it just stops it for the duration of the punishment (which initially may be just a frown) but it adapts.
Insightful: The immediate albeit short-lived effect of punishment is actually reinforcing the punishing behavior in the parent because of the immediate positive effect!
Disadvantages of punishment
Methods
Key points
"Use sparingly." Give it calmly consistently immediately and brief to make it effective. "Done in isolation from others". Never punish with any activities you want you child to like.
"Don't waste your creativity on coming up with novel ways to punish misbehavior."
For me this all actually means: Hard to do right. Easier to avoid punishment altogether. I further reduced the punishments I used. I now almost only use mild aversive consequences (frown, mild reprimands), removal of attention and requirment of undoing damage done (but no forced apologies).
Other aspects of punishments:
"Don't believe that knowing and doing are necessarily related." "Knowing and understanding a rule by itself will not lead to your child changing the behavior."
Separate out the behavior changing part from other intended purposes like serving justice, instilling remorse, teach a moral lesson or send a message. Separating means to immediately punish (if you must) for the behavior changing effect and to address the other purposes independently such as to avoid ruining the behavior changing effect (this separation is not clearly spelled out by Kazdin).
Withhold attention (includes negative attention) aka ignorance is another form of negative consequence.
This works but it is slow. Initially the child may show the behavior even more (called "burst") and even later the extinguished behavior can come back spontaneously. Kazdin offers no explanation as to why but I think this is kind of Thompson sampling by the child to (subconsciously?) test whether the behavior might again work.
This method can be difficult if the reinforcement is not coming from the parent (but e.g. from class mates, siblings) or even worse from internal motivation like curiosity.
This can cause side effects like emotional responses and aversion like in the case of punishment. "Don't use in isolation."
With this the toolkit is complete. Kazdin goes on to provide practical usage examples of this toolkit in everyday situations for differently aged children (adding to to earlier examples used to illustrate the methods which I just summarized dryly above). These do not add no new concepts but may be helpful to translate the methods into real-life use.
The last part of the book is devoted to the overall family parenting environment. This contributes to the overall parenting success but is no specific method you apply in a specific situation. Much of this is actually covered on LessWrong here or there already. Nonetheless I will shortly summarize the sections:
These sections are all backed by research.
A very humane and down to earth approach based on solid research. Pragmatic even in allowing mild forms of punishment. Being mild and calm and ignoring behavior isn't showing weakness and using the complete set of methods is not exploitable as far as I can see. It appears that this is general advice that is just specialized for children. Kazdin sometimes gives real-life examples for older children and adults.
Actually Kazdin has enriched and sharpened my 'parenting toolkit' and changed my view on a few points. I have now mostly dropped again any punishment that had crept in over the years (disquieting how the immediate feedback of it manipulates you away from your ideals).
And I have changed my position on ignorance. I'd now write my reply differently. First I'd refer to this post :-) Second I'd accept that ignorance is OK - as long as you provide sufficient reinforcements of the positive opposites of the behavior. Third a frown to indicate that you disapprove is a kind of mild punishment that would be appropriate (albeit not in isolation).
1) How to deal with behavior that results from inner motivation. If you have two competing children who are inherently and strongly motivated to out-do the other, then I don't see the positive opposite. I don't want to reinforce separating them. And I haven't found a way to limit their conflict. They cooperate a lot, so reinforcement that doesn't help either. And their sudden violent outbreaks provide little leverage to reward.
2) What about natural or logical consequences that are part of many other (esp. newer) parenting methods? Kazdin says little about this and I wished for some advice here.
3) And at last a theoretical question:
Isn't punishment negative reward? Shouldn't punishment have the same effect as reward but just the opposite sign? Is this an asymmetry that applies to all reinforced learning algorithms or is this something special to biological learning? And if yes why?
Could be because negative reward just pushes away but in no specific direction whereas the 'positive opposite' also moves away but into a specific direction.
Does anything follow from this for AI learning? Should we instead of providing negative reward for areas to be avoided use positive reward for areas to be steered to?
EDIT: Removed redundant "rational" from title. Fixed breaks.
ADDED: See also http://en.wikipedia.org/wiki/Applied_behavior_analysis