Bryan didn't touch this subject with a 10-foot pole, but on the subject of why adults mostly don't retain anything they learn in school, it's worth mentioning that learning things is cognitively difficult and most people aren't all that smart. Why g Matters: The Complexity of Everyday Life by Linda S. Gottfredson is the most revealing thing I've read on this subject by far. Some choice quotes, although much of the info in this paper is in large tables:
If the 25th WPT percentile of applicants is used to estimate the minimum threshold for employability in an occupation, it suggests that virtually all occupations accommodate individuals down to IQ 110, but virtually none routinely accommodates individuals below IQ 80.... Lest IQ 80 seem an unreasonably high (i.e., exclusionary) threshold in hiring, it should be noted that the military is prohibited by law (except under a declaration of war) from enlisting recruits below that level (the 10th percentile). That law was enacted because of the extraordinarily high training costs and high rates of failure among such men during the mobilization of forces in World War II (Laurence & Ramsberger, 1991; Sticht et al., 1987; U.S. Department of the Army, 1965).
...
Table 3 provides the passing rates in the 1955 standardization sample of the WAIS for its 40 vocabulary items (Matarazzo, 1972, p. 514). All individuals tested were able to provide at least a tolerable definition of concrete items such as bed, ship, and penny, but passing rates dropped quickly for more abstract and nuanced concepts such as slice (94%), sentence (83%), domestic (65%), and obstruct (58%). Only half of this nationally representative sample of 16 to 65-year-olds could define the words “remorse,” “ reluctant,” and “calamity.” Fewer than one in five knew the words “ominous” and “tirade,” and only 5% could provide even a partial definition of “travesty.” None of these words is esoteric; anyone who has attended U. S . high schools or read national newspapers or magazines has surely encountered them. Vocabulary tests gauge the ease with which individuals have routinely caught on to new and more complex concepts they encounter in the general culture.
...
The Similarities subtest of the WAIS provides another example of how the manifest content of a test serves merely as a vehicle for creating differentially complex cognitive tasks. As shown in Table 4, all concepts in the subtest are well known; the most difficult test item uses the words “fly” and “tree.” What the test requires is for people to state one way in which the two concepts (say, orangebanana or table-chair) are similar. It thus requires people to abstract key attributes or uses for each, compare those attributes, and then judge which ones are similar. Passing rates drop quickly as relations between the items become more abstract. Over 90% of the WAIS standardization sample could identify one pertinent similarity between oranges and bananas, but only 69% could do so for eyes and ears. Fewer than half succeeded in giving a similarity between egg and seed, and only one quarter produced one similarity between praise and punishment.
...
Table 8 shows the percentages of White adults who are proficient at each of the five levels on the three NALS scales. Generally about 4% reach the highest level. Level 5 (376-500) signals an 80% probability, for example, of being able to summarize two ways that lawyers challenge prospective jurors (based on a passage discussing such practices) and to use a calculator to determine the total cost of carpet to cover a room (see Figure 2). Roughly another 20% of White adults reach Level 4 (326-375), where individuals can perform such tasks as restating an argument made in a lengthy news article and calculating the money needed to raise a child based on information stated in a news article. A total of about one third of White adults reach Level 3 (276-325), but no higher, which includes capabilities for writing a brief letter explaining an error in a credit card bill and using a flight schedule to plan travel arrangements. Level 2 proficiency (226-275) includes locating an intersection on a street map, entering background information on an application for a social security card, and determining the price difference between two show tickets. This level is reached but not exceeded by about 25% of Whites. Finally, one out of seven White adults functions routinely no higher than Level 1 (less than 225), which is limited to 80% proficiency in skills like locating an expiration date on a driver’s license and totaling a bank deposit. Individuals at Level 1 or 2 “are not likely to be able to perform the range of complex literacy tasks that the National Education Goals Panel considers important for competing successfully in a global economy and exercising fully the rights and responsibilities of citizenship” (Baldwin et al., 1995, p. 16).
The picture of the world I get from reading this paper is depressing as hell. I get a sense of a form of suffering that is politically difficult to talk about: suffering because you don't have enough cognitive resources to tackle the challenges the world gives you, including but not limited to the torture of schools attempting to teach you things that are both pointless and difficult for insane reasons. I also get a sense of the privilege - let's call it cognitive privilege - of being able to ignore this, because it's not a problem that you or the people you know have.
Tl;dr: the second half of the post conflates ability bias with signalling.
You’d think you could also do this simply by paying people purely on the basis of their educations (and perhaps seniority), like a government or a union shop where ability isn’t relevant. The problem is you’d still need to fix that more able people more often attend, and conditional on going more often finish.
That is the obvious explanation, and is completely consistent with everyday experience. Sure, I could waltz through a PhD no problem, but I'm not getting paid nearly as much today as I would with a PhD.
And yes, I do get paid more than my former classmates who would not be able to handle a PhD. So that speaks to nonzero ability bias (n=1). On the other hand, the difference in pay between "has an undergrad degree and could handle grad" vs "has an undergrad degree and could not handle grad" is presumably way smaller than the difference between "has an undergrad degree and could handle grad" vs "has a grad degree".
If you want to hypothesize a world where ability bias is actually zero, then yes, you'd have to turn to the weird scenarios in this post. But you don't need any of that to hypothesize a world where ability bias looks like zero - is statistically indistinguishable from zero. For that, you just need a world where ability bias is rounding error on top of the main effect, i.e. signalling, and all the statistical effect from ability bias gets masked by that larger effect.
In particular, if you measure ability bias by looking at earnings after controlling for education level, then you do not need to "fix that more able people more often attend, and conditional on going more often finish" in order to find statistically zero ability bias effect. The effect from "more able people more often attend" etc would be signalling, not ability bias - as the earlier part of the post defines them. Those effects go away when we control for education; that's the whole point of controlling for education. If the large effect is signalling, then of course we're not going to find a large effect when we look for ability bias separate from signalling!
Now, I certainly agree that Berkeley professors arguing for the effectiveness of education should be viewed with an awful lot of suspicion. But maybe rather than just dropping a-priori anvils, look at the data? It's entirely plausible that ability bias effects are statistically indistinguishable from zero, but this post doesn't really provide much evidence toward that question one way or the other.
It isn’t the Scott Aaronson post, but he was was probably making reference to Lockhart’s Lament.
(Remember, these were true/false questions, so 50% means no knowledge at all. Less than that is… worse. Answer key not included, because come on.)
There's one answer that isn't quite clear. Cloning makes most of the genes identical but mitochondrial DNA isn't necessarily completely identical.
Surveys are really hard to design correctly.
Remember, these were true/false questions, so 50% means no knowledge at all.
This isn't apparent from the data. A score of 50% could mean that nobody knows the answer and everyone is guessing randomly. Or it could mean that 50% of survey-takers know the right answer and 50% mistakenly believe the wrong answer. Or something in between. Without more information, we can't distinguish which is which.
I'd also argue that three of the questions were ambiguous or uncertain:
Part of test-taking ability seems to be selectively ignoring ambiguity if you think the people who designed the test weren't testing for that edge case.
[1] http://science.sciencemag.org/content/356/6336/eaal1641
Most people who take high school algebra and geometry forget about half of what they learn within five years
This is not a bug, it's a feature. People forget what they don't use. Most people don't use algebra and geometry, so it's okay for them to forget it. We teach algebra and geometry to everyone, because we don't know who will need tem, because we don't trust teenagers to decide what they want to do for the rest of their lives.
Their science knowledge?
You have to explain to me how science trivia knowledge relates to human capital, in the economic sense. I mean, the effect shouldn't be zero, but it is most likely tiny.
If we got rid of school entirely, would adults score worse?
Good question. Are you going to answer it? It's kind of important to your point. My own answer would be "yes, obviously".
The portion from signaling is <...> probably mostly zero sum for society
Signaling creates obvious positive value. Without signaling, employers would waste a lot more time on hiring incompetent people (or they would just resort to nepotism). Of course, signaling also has costs, and it can be debated what the total sum effect is.
the entire field dedicated to studying this tries to deny that ability bias is substantive enough to be meaningful
I don't understand why you spend half of your post on this. It's completely irrelevant to your point. Should I guess that some logic error is hiding behind your attention to this?
Also, somehow you seem to have transformed "not meaningful" into "doesn't exist", which leads me to believe that you're strawmanning the academics in other ways as well (though I don't care to check).
An Introduction: The Case Against Education
We’ll now focus more on the book and its data, reasoning and arguments, and related questions. There’s a lot to address, starting with the foundations of Bryan’s analysis. Rather than write one novella-long post, I’ll be going through my highlights and talking about related issues, and post sections as I go.
IV
Well, what’d ya know?
Bryan has the data: Not much. You?
Here are some data points about adults:
Most people who take high school algebra and geometry forget about half of what they learn within five years and forget almost everything within twenty-five years. Only people who continue on to calculus retain most of their algebra and geometry.
Yikes. We can argue about the value of geometry and calculus, but a practical proficiency in basic algebra is kind of important to functioning in the world.
Possible conclusions from this, aside from ‘Americans can’t do basic algebra, set your prices and marketing campaigns appropriately and maybe learn Mandarin’ include:
A: We don’t spend enough on algebra! Spend more time and money!
B: Don’t spend more time and money, we already spend a ton. It’s that we are horrible at teaching math. Stop it. Work smart, not hard. Maybe treat math as an adventure and not a series of memorized rules and tables.
(There’s an utterly devestating Scott Aaaronson post about what music education would look like if it was structured like math education, but my Google-fu couldn’t find it, someone link to it in the comments and I’ll edit in the link)
C: Don’t spend more time and money on algebra and geometry. Make calculus mandatory. Study shows that will make people remember algebra!
D: People actually implement things like C (and also A) all the time, so maybe drop mandatory algebra and geometry, and focus on teaching statistical literacy instead.
E: Maybe before we teach anyone algebra we should see if they know how to multiply.
Good idea. How about adults’ basic arithmetic?
Barely half know that saving $.05 per gallon on 140 gallons of oil equals $7.00.
Whoops.
You can either complain that this means our failing schools need even more funding and to imprison our kids even longer, since the right response to something not working despite ludicrous investments of resources is to invest more resources do what isn’t working even harder, or you can admit the entire system is an utter failure.
Their science knowledge?
Table 2.4: Adult Science Knowledge: Some Telling Questions, With % Correct
The center of the Earth is very hot. 81%
The continents on which we live have been moving their locations for millions of years and will continue to move in the future. 78%
Does the Earth go around the Sun, or does the Sun go around the Earth? 73%
All radioactivity is man-made. 68%
Electrons are smaller than atoms. 52%
Lasers work by focusing sound waves. 46%
The universe began with a huge explosion. 33%
The cloning of living things produces genetically identical copies. 80%
It is the father’s gene that decides whether the baby is a boy or a girl. 62%
Ordinary tomatoes do not contain genes, while genetically modified tomatoes do. 47%
Antibiotics kill viruses as well as bacteria. 53%
Human beings, as we know them today, developed from earlier species of animals. 44%
(Remember, these were true/false questions, so 50% means no knowledge at all. Less than that is… worse. Answer key not included, because come on.)
Brutal. In this context, the majority of Americans not believing in evolution isn’t a strange religiously motivated outlier. It isn’t partisan, either. Mostly people just don’t know things.
That’s a bunch of facts, though. Do we teach them how to think?
One researcher tested several hundred Arizona State University students’ ability to “apply statistical and methodological concepts to reasoning about everyday-life events.” How, for example, would subjects assess the claim that students should eat more nutritiously because “the majority of students needing psychological counseling had poor dietary habits”? Would subjects realize psychological problems might cause poor dietary habits, rather than the other way around? Would they feel the need for experimental evidence? No.
In the author’s words: The results were shocking: Of the several hundred students tested, many of whom had taken more than six years of laboratory science in high school and college and advanced mathematics through calculus, almost none demonstrated even a semblance of acceptable methodological reasoning about everyday-life events described in ordinary newspaper and magazine articles. The overwhelming majority of responses received a score of 0. Fewer than 1% obtained the score of 2 that corresponded to a “good scientific response.”
Not so much. The surprising finding is that these findings were surprising.
If we got rid of school entirely, would adults score worse?
And then they voted. But that’s a different Caplan book.
But perhaps the study doth judge too harshly.
Consider this quote, about a study that showed people with poor diets experiencing more health problems (as usual, scientists are hard at work asking the hard questions with high value of information):
Totally ignoring the need for comparison groups and control of third variables, subjects responded to the “diet” example with statements such as “It can’t hurt to eat well.”
How does the author of that sentence think one gains knowledge of the world? Does he think we need a control group to know which switch turns on the light in the bathroom?
The response of “eating healthier foods might help your problem or it might do nothing, but it’s highly unlikely to make things worse, so doing so is a good idea” is exactly what a practical Bayesian analysis looks like, here. Does author think it could hurt to eat well? There’s the obvious problem that we don’t know what ‘eat well’ actually entails, but that doesn’t change the answer.
Comparison groups and control of third variables are important. It is entirely plausible that people who have good diets also take care of themselves in other ways, or both have higher socioecnomic status, or good health enables eating well, or any number of other such things. There’s a reason I tried starting a personalized medical research company. This stuff is hard!
But the response of ‘it can’t hurt’ is the right prior. It is properly taking all that uncertainty into account, whether or not the person could articulate any of it.
What matters is practical knowledge of correlation/causation and related issues. If one goes around thinking correlation proves causation, that’s really bad. But ‘it can’t hurt to eat well’ is not a confident claim that eating well would help. The null hypothesis isn’t being rejected. But if one thinks that one shouldn’t change behavior based on data unless the null hypothesis can be rejected, they’re even more wrong than thinking correlation implies causation.
Are we testing for understanding, or guessing the teacher’s password? It’s no surprise that no one remembers the password decades after the test.
If academics studying learning don’t even know what practical knowledge looks like, how much practical knowledge are students likely getting from the academy? Or as Talib puts it in Skin in the Game:
In academia there is no difference between academia and the real world; in the real world, there is.
We then allow academics to determine our definitions, our beliefs and our policies. And our hidden assumptions.
In particular, the idea that education equals school equals skills.
Thomas Piketty, in Capital in the Twenty-First Century, makes many questionable assumptions. But none of his mongering about inequality was half as maddening to me as when he measured job skills of a population by looking at years spent without a job, in schools. If a population lacked job skills, of course that meant we hadn’t spent enough on education! Because education equals school equals skills.
Yesterday I was at Citi Field for Jackie Robinson Day, when we celebrate Jackie Robinson’s demonstration that we’re all the same by having all the players wear the same number so we can’t tell them apart, and also celebrate Douglas Adams by making that number 42. We got to hear about Jackie’s nine values, a solid list: Courage, determination, teamwork, persistence, integrity, citizenship, justice, commitment, and excellence. And we got quotes about how education (which is noticeably not in Jackie’s nine) was the key to our future… and once again, education equals school.
Academics and politicians don’t reject (or even consider) the signaling model of education. They use a linguistic slight of hand to assume it away. Rendering it unthinkable. Redirecting people’s desire to help educate people into funds to add fuel to a dystopian nightmare.
It all makes me wonder even more what to take away from sentences like this one:
By the time students are in middle school, however, one summer vacation wipes out over three months of reading proficiency.
Multiple choice, choose all that apply:
A: The reading skills lost are real. We should eliminate summer vacation lest our nation fall behind. We could get twice as much reading proficiency, since instead of gaining 9 months then losing 3, we’d gain 12!
B: The reading skills lost are real. We don’t quite need to eliminate summer vacation, but we do need to ruin kids’ fun more by assigning extra reading work so they won’t lose their progress. More homework must mean more education, even if the data doesn’t show it.
C: We should eliminate summer vacation, because during summer vacation children rediscover creativity and forget what the teachers’ password was, setting back test scores by three months and forcing us to reteach them all that each fall.
D: Same observations as Option C, except we should definitely not eliminate summer vacation.
E: Our obsession with tests has forced students to engage in cramming-style behaviors that don’t generate long term knowledge. Of course, any time spent this way is at best zero sum signaling.
F: School has turned children off of reading to the extent that they don’t even read enough to maintain their skill levels without state coercion. Something was wrong.
G: Kids can’t retain skills or knowledge beyond what is supported by their age, developmental level and home environment. Trying to push them beyond that results at most in temporary gains. If kids lose reading skill over the summer, that means we shouldn’t have been trying to teach them that yet.
H: It’s all fraud. At the end of the year, teachers cheat to help students score better. At the start of the new year, teachers if anything want their incoming students to score worse.
As you suspect, my prior is on a combination of D, E, F, G and H.
Bryan doesn’t focus on these types of questions, or modes of thinking, choosing other targets. Books can only be so big, there’s only so many hours in the day, and he’s trying to give the benefit of the doubt.
V
Lets reorient to Bryan’s approach. Bryan starts with the well-known fact that there exists an education premium: Those with more years of schooling earn more money. Note that even Bryan calls it the education premium rather than the schooling premium. Even he’s been hoodwinked. School is not education!
Thus, where Bryan says ‘education’ or ‘educated’ but means ‘school’ or ‘schooled’ I will write ‘school’ or ‘schooled.’
Whatever you call it, the premium is real. Some combination of things is creating that premium. By Bryan’s accounting, there are three suspects:
Human Capital is the traditional justification for most or all of the effect. School makes you smart. It teaches you the skills and values you need to succeed in life. This makes the schooled more productive than the unschooled, so they get paid more.
Signaling is Bryan’s justification for the bulk of the effect. School is a test of your intelligence, conformity and conscientiousness. The longer and more difficult schooling you complete, the more you’ve demonstrated all three skills. Employers want all three skills, so they give better pay and opportunities to the schooled, well beyond the human capital created by schooling.
Ability Bias is also present to an unknown degree. Those who sign up for more school are already smarter, more conformist, more conscientious and generally more able to succeed in today’s economy than those that don’t. The more able students also complete what schooling they start more often than the less able. Thus, they would have been paid a premium even if they hadn’t had more schooling.
The portion of the schooling premium that comes from human capital is real and positive sum – the student creates more value and gets rewarded. The portion from signaling is real and positive for the student but probably mostly zero sum for society (more on this later, as I find this less obvious than Bryan does) – the student doesn’t create more value except insofar that they get better opportunities, but gets better opportunities that probably would have gone to others, and gets paid more. The part that comes from ability bias is a statistical mirage – nothing happened at all.
There is lots of uncertainty about how much of the effect comes from each source. What is beyond obvious is that signaling and ability bias are present and important. We see lots of explicit signaling and rewarding of signaling, so that’s important. Those who go to college have obviously vastly superior economic prospects than those who don’t go to college, even before anyone attends: They are smarter, more knowledgeable, more skilled, have higher socioeconomic status, are healthier, are more conformist, are more conscientious, are less likely to have committed major crimes. You name a thing that correlates with success, with notably rare exceptions they have more of it. The same goes for those who graduate versus those who drop out, or who graduate high school versus drop out, and so on.
It would be utterly completely insane to say that ability bias wasn’t a thing.
So of course, Bryan points out that the entire field dedicated to studying this tries to deny that ability bias is substantive enough to be meaningful:
A famous review of the evidence by eminent economist David Card concludes ability bias is small, nonexistent, or even negative. I call this verdict the Card Consensus. Many, perhaps most, elite labor economists not only embrace it but rely on it for practical guidance. We see the Card Consensus in top scholarly venues like the Journal of Economic Literature. The return to an additional year of education obtained for reasons like compulsory schooling or school-building projects is more likely to be greater, than lower, than the conventionally estimated return to schooling.
What. In. The. World?
Academic labor economists, led by a professor at the University of California, Berkeley (because, of course) think labor success comes from academics, show robust benefits from more and compulsory schooling, deny other justifications for inequality of outcomes. Recommend government intervention and mandated behaviors to help for less fortunate, and lots more money for their and other fields. Also higher status.
On the one hand, film at eleven. Bias is expected here.
On the other hand, what in the world?
What would an Earth without ability bias look like?
This is a world in which (for example), if we select a group of random students who would normally go on to complete high school and force them to drop out of high school, they would earn no more than the average high school dropout. If we took random students who would have attended college, and prevent them from attending college, they will earn no more than the average person who never attends college.
No, really. Stop here and actually think for five minutes: What would this Earth without ability bias look like? Try to construct a toy model where ability bias is zero, and see what happens.
I’ve managed to come up with three possible worlds without ability bias. You could have a world of identical people, likely involving cloning vats. Or perhaps you could go full on Harrison Bergeron. Or as a wildcard, all the jobs are fake, robots do all the work, so no one has any ability. You could have a world where everyone is either forced to or forbidden to attend school, at random, based on nothing that measures their human capital or correlates with their future economic production. And of course, mysteriously (ominous music) no one ever fails. Then there’s a third option involving mass kidnapping, rape and enslavement. Perhaps like the Handmaid’s Tale, except for the competent instead of the fertile.
Feel free to turn them into young adult trilogies or spec scripts for The Orville.
The thinking is this: If you want to break the correlation between school attendance and ability, you have four basic choices.
You can give everyone identical abilities. That seems to be what labor economists endorse: Schooling equals education equals skill. Except that’s obviously insane.
You’d think you could also do this simply by paying people purely on the basis of their educations (and perhaps seniority), like a government or a union shop where ability isn’t relevant. The problem is you’d still need to fix that more able people more often attend, and conditional on going more often finish.
You can make people with different abilities equally likely to attend and finish school. That requires taking away their choice in whether to attend – not just with attendance requirements, but actually taking away all choice. Then you have to make sure ability doesn’t impact ability to finish.
Your third choice is to accept that ability bias is a thing, but create an equal and opposite bias in the opposite direction that cancels it out.
Wrong conclusions are wrong. What we have here is an entire economic field setting one of its most important variables to an impossible number that doesn’t stand up to the slightest logic or scrutiny. It describes a world vastly unlike our own, vastly unlike even a highly ideological, toy academic version of that world. It makes no sense. And they’re doing it in a way that happens to support the status of their ‘field’ and their ideological agendas. If you call that a consensus, then there being a consensus in your field is no longer much evidence in support of that consensus. Even the raw data gathered should be considered highly suspect, and you should put strong weight on your own priors and models.
Those same people are going around saying that while most every other demand curve slopes downward, somehow higher minimum wages don’t lead to decreased employment. Which to be fair I took seriously, and found plausible on the realistic margin in that one case. But this pattern does not inspire confidence.
What this says about other academic fields with similar motivations, structures and conclusions is left as an exercise to you. No need for me to step on that land mine.
I have a lot more quotes and notes to get to, and hope to do so, but I continue to be busy so it might be a while before I get to that.