Epistemic status: I traced one famous anecdote to its printed source and to both coauthors' own accounts of where it came from. Confident it was never a study. No claim either way about whether quantity beats quality in general.
AI use: research (tracing the story to the book, to the coauthors' emails and to earlier LessWrong threads), fact-check of every claim against the sources, footnotes, grammar.
I was reading a book and stumbled onto a claim that there was an experiment. A group of students learning pottery was divided into two groups. One group was told that they would be evaluated based on volume, i.e. the number of pieces they produced. The other group was told that they would be evaluated based on the quality of their best piece. Then both groups were compared based on the quality of the best piece, and the "quantity" group showed better results.
I was extremely intrigued by this experiment and was curious to understand how big the groups were and how they evaluated the quality of a pottery piece (there must be some way to quantify that for the sake of the comparison?).
To my surprise, this wasn't a scientific study, but an anecdote from the book Art & Fear: Observations on the Perils (and Rewards) of Artmaking by David Bayles and Ted Orland (page 29 of the Image Continuum edition).[1]So all the details available about this result are:
Students were divided into 2 groups based on their physical position in the studio (left vs right), which is not properly random.
The quantity group's evaluation criterion was the total weight of their pots: 50 pounds -> A, 40 pounds -> B.
The quality group needed to produce only one pot, but it had to be perfect to get an A (the book does not specify what perfect means).
Results: "Well, came grading time and a curious fact emerged: the works of highest quality were all produced by the group being graded for quantity. It seems that while the "quantity" group was busily churning out piles of work—and learning from their mistakes—the "quality" group had sat theorizing about perfection, and in the end had little more to show for their efforts than grandiose theories and a pile of dead clay." Again, no mention of a definition of the quality metric, nor of the effect magnitude. What does "all works of highest quality" even mean? If you sort all the produced works by quality, where does "highest quality" stop? You have to draw the threshold somewhere.
This is already disappointing, since this is basically an anecdote about a study, and not a very well set up one. However, it gets worse. In 2016 both coauthors separately admitted that they got the story from someone else, and that the original story had nothing to do with ceramics: it was about photography. David Bayles told a Hacker News reader by email that it was "a university level photography class taught by 'a very experienced and sophisticated teacher'" and that apart from the medium it was "the literal truth".[2]Ted Orland told James Clear, in an email dated October 18, 2016, that the book "faithfully captures the scene as Jerry told it to me—except I replaced photography with ceramics as the medium being explored".[3]Jerry is the photographer Jerry Uelsmann, who taught at the University of Florida and was Orland's collaborator for decades.[4]
Why did they change the subject? Both authors are photographers. Orland says they were "consciously trying to broaden the range of media being referenced in the text", and admits "it would've been easier to retain photography".[3]Bayles put it more bluntly to his correspondent: they already had too many photography examples.[2]
The photo detail changes the nature of the experiment. It is fairly easy and cheap to take a photo compared to making a pottery piece. So one way to explain why the quantity group produced better results is to just say that each shot had a (not perfectly correlated) probability of being the best, thus more shots = more lottery tickets.
This anecdote has been mentioned on LessWrong multiple times:
David Bayles and Ted Orland, Art & Fear: Observations on the Perils (and Rewards) of Artmaking (Santa Barbara: Capra Press, 1993; reissued Santa Cruz: Image Continuum Press, 2001), p. 29, the opening paragraph of the section "Perfection". The book gives no source for the story and has no notes or bibliography. A scan of the Image Continuum edition: https://artistryinaction.wordpress.com/wp-content/uploads/2016/03/david_bayles_ted_orland_art_and_fear.pdf (PDF page 35).
Hacker News user kens emailed David Bayles in July 2016 and reported the answer on 2016-07-17: https://news.ycombinator.com/item?id=12109191 ("it wasn't a ceramics class as written, but a university level photography class! They changed it to pottery in the book because they had too many photography examples already."). More details in a 2017 comment: https://news.ycombinator.com/item?id=14499500
Ted Orland, email to James Clear, October 18, 2016, reproduced in full in footnote 1 of Clear's "Why Trying to Be Perfect Won't Help You Achieve Your Goals (And What Will)", https://jamesclear.com/repetitions, and in the chapter 11 endnotes of Atomic Habits. The email became public only with the book in October 2018; the Wayback Machine shows Clear's page still telling the ceramics version, with no footnote, in May 2018.
Orland is credited on Uelsmann's Wikipedia page as the photographer of a 1969 portrait of Ansel Adams and Imogen Cunningham and as the author of Uelsmann/Yosemite (1996) and of the text of Dances with Negatives (2011): https://en.wikipedia.org/wiki/Jerry_Uelsmann. Uelsmann taught at the University of Florida from 1960 to 1998 and died in 2022, so he never confirmed the story himself.
Epistemic status: I traced one famous anecdote to its printed source and to both coauthors' own accounts of where it came from. Confident it was never a study. No claim either way about whether quantity beats quality in general.
AI use: research (tracing the story to the book, to the coauthors' emails and to earlier LessWrong threads), fact-check of every claim against the sources, footnotes, grammar.
I was reading a book and stumbled onto a claim that there was an experiment. A group of students learning pottery was divided into two groups. One group was told that they would be evaluated based on volume, i.e. the number of pieces they produced. The other group was told that they would be evaluated based on the quality of their best piece. Then both groups were compared based on the quality of the best piece, and the "quantity" group showed better results.
I was extremely intrigued by this experiment and was curious to understand how big the groups were and how they evaluated the quality of a pottery piece (there must be some way to quantify that for the sake of the comparison?).
To my surprise, this wasn't a scientific study, but an anecdote from the book Art & Fear: Observations on the Perils (and Rewards) of Artmaking by David Bayles and Ted Orland (page 29 of the Image Continuum edition).[1]So all the details available about this result are:
This is already disappointing, since this is basically an anecdote about a study, and not a very well set up one. However, it gets worse. In 2016 both coauthors separately admitted that they got the story from someone else, and that the original story had nothing to do with ceramics: it was about photography. David Bayles told a Hacker News reader by email that it was "a university level photography class taught by 'a very experienced and sophisticated teacher'" and that apart from the medium it was "the literal truth".[2]Ted Orland told James Clear, in an email dated October 18, 2016, that the book "faithfully captures the scene as Jerry told it to me—except I replaced photography with ceramics as the medium being explored".[3]Jerry is the photographer Jerry Uelsmann, who taught at the University of Florida and was Orland's collaborator for decades.[4]
Why did they change the subject? Both authors are photographers. Orland says they were "consciously trying to broaden the range of media being referenced in the text", and admits "it would've been easier to retain photography".[3]Bayles put it more bluntly to his correspondent: they already had too many photography examples.[2]
The photo detail changes the nature of the experiment. It is fairly easy and cheap to take a photo compared to making a pottery piece. So one way to explain why the quantity group produced better results is to just say that each shot had a (not perfectly correlated) probability of being the best, thus more shots = more lottery tickets.
This anecdote has been mentioned on LessWrong multiple times:
So will I get better at writing by writing a bunch of posts? Sadly, this story does not answer that, but we can just wait and see.
David Bayles and Ted Orland, Art & Fear: Observations on the Perils (and Rewards) of Artmaking (Santa Barbara: Capra Press, 1993; reissued Santa Cruz: Image Continuum Press, 2001), p. 29, the opening paragraph of the section "Perfection". The book gives no source for the story and has no notes or bibliography. A scan of the Image Continuum edition: https://artistryinaction.wordpress.com/wp-content/uploads/2016/03/david_bayles_ted_orland_art_and_fear.pdf (PDF page 35).
Hacker News user kens emailed David Bayles in July 2016 and reported the answer on 2016-07-17: https://news.ycombinator.com/item?id=12109191 ("it wasn't a ceramics class as written, but a university level photography class! They changed it to pottery in the book because they had too many photography examples already."). More details in a 2017 comment: https://news.ycombinator.com/item?id=14499500
Ted Orland, email to James Clear, October 18, 2016, reproduced in full in footnote 1 of Clear's "Why Trying to Be Perfect Won't Help You Achieve Your Goals (And What Will)", https://jamesclear.com/repetitions, and in the chapter 11 endnotes of Atomic Habits. The email became public only with the book in October 2018; the Wayback Machine shows Clear's page still telling the ceramics version, with no footnote, in May 2018.
Orland is credited on Uelsmann's Wikipedia page as the photographer of a 1969 portrait of Ansel Adams and Imogen Cunningham and as the author of Uelsmann/Yosemite (1996) and of the text of Dances with Negatives (2011): https://en.wikipedia.org/wiki/Jerry_Uelsmann. Uelsmann taught at the University of Florida from 1960 to 1998 and died in 2022, so he never confirmed the story himself.