Thank you to everyone who participated in the AI Baroque Music Quiz! The quiz is now closed. If you didn’t get a chance to take it, I put all the excerpts into this SoundCloud playlist. You can listen to each excerpt, write down your answers, and compare them to the official ones below.
Answers
Buxtehude – “Nun bitten wir den heiligen Geist,” BuxWV 209
Vivaldi – Concerto in D minor for Viola d’amore and Lute
Claude Opus 5.5 – Fugue in A minor (Extra High effort)
Bach – Trio Sonata V in C major, BWV 529
GPT-6 Astra – Sonata da chiesa for Violin (Extra High effort)
GPT-6 Astra – Oboe Concerto in D minor (Ultra effort)
Albinoni – Sonata da chiesa No. 6, Mvt. 2
Claude Opus 5.5 – Oboe Concerto in C minor (Extra High effort)
Handel – Fugue in G minor, HWV 605
Lully – Aria from Armide (Act II, scene iii)
Graupner – Oboe d’amore Concerto, GWV 313
Claude Opus 5.5 – “Herr Gott, dich loben alle wir” (Extra High effort)
Telemann – Recorder Duet from Der getreue Music-Meister, TWV 40:107
GPT-6 Astra – Oboe Sonata in G minor (Extra High effort)
GPT-6 Astra – Flute Concerto in G (Extra High effort)
GPT-6 Astra – Violin Sonata in D (Extra High effort)
Results & Analysis
The quiz received 118 complete submissions and six incomplete ones. Only complete submissions were included in this analysis. To be considered complete, a submission had to include the AI-or-human answer for all 16 tracks (the optional questions about musical background, confidence level, etc. could be left blank).
Anyone wanting to perform their own analysis is welcome to do so. All the quiz data — including the responses, questions, prompts, and LilyPond files — is available on GitHub.
Accuracy
55.7%. That’s how accurate respondents were at distinguishing LLM-generated Baroque music from the real thing. Random guessing would have yielded a result of around 50%, so listeners did somewhat better than pure chance, but telling human and AI compositions apart clearly isn’t easy.
Professional musicians did better at 63.8%, while respondents with no musical training scored the lowest at 48.4%.
Accuracy did not increase consistently with listening frequency. Weekly listeners scored higher than daily ones (61.2% vs. 57.2%), and people who reported never listening to classical music outperformed occasional and monthly listeners (52.1% vs. 49.8% and 49.0%).
The latter discrepancy is best explained by the small sample size (only three respondents reported never listening to classical music), but the difference between daily and weekly listeners is harder to explain. I thought maybe the high-performing weekly listeners were professional musicians who were too busy teaching and performing to listen recreationally, but the data doesn’t support that hypothesis: both groups had exactly 10 professionals, and the weekly group was larger.
The quiz also asked about confidence and listening setup. Of the two, confidence showed a clearer correlation with accuracy: listeners who were very confident in their answers scored 62.7%, compared to 53.4% for those who were only guessing. On one hand, this is good: it shows people are somewhat accurate at assessing their own abilities. On the other hand, very confident listeners were still wrong more than a third of the time. That’s bad.
Listening setup was less clear-cut. Headphone users scored the highest at 57.6%, but listeners using phone speakers (55.5%) narrowly beat those using ordinary speakers (54.3%). This surprised me, because I thought (perhaps incorrectly) that phone speakers were the worst of the three. Oh well. It’s probably just noise.
The Most Human & AI Tracks
Bach’s Trio Sonata and Claude’s Fugue in A minor tied for the most “Human” votes at 85 each (72%). (The tie breaks in Claude’s favor if we include the incomplete responses.)
The rest of the top five was rounded out by Telemann, Graupner, and Claude again, whose Oboe Concerto in C minor received more human votes (60.2%) than works by famous composers like Handel (58.5%) and Vivaldi (43.2%).
The human composer most often mistaken for AI was Buxtehude, whose chorale prelude was judged AI by 66.9% of respondents (including 64% of professional musicians). If I had to guess why, I’d say it was the open fifths and fourths at the end of the excerpt. These sonorities have a striking effect on our modern ears, and many listeners don’t expect to hear them in Baroque music.
The AI track most often identified as AI was GPT-6 Astra’s Sonata da chiesa at 72.9%. I thought this piece was more convincing and its rendering more lifelike than most of the AI tracks, so I expected it to rank higher on the “Human” scale. What was it that tipped the scales? The unrealized figured bass? I’d be curious to know.
Enjoyment
On average, listeners enjoyed Claude’s music slightly more than human composers’ (3.14 vs. 3.07 on the 1–5 enjoyment scale). Claude’s harpsichord fugue (3.48) and oboe concerto (3.33) ranked second and third out of all 16 tracks.
First place, however, went to a human: Bach, whose Trio Sonata scored 3.63, well ahead of every other human-written piece. The next-highest human work was Telemann’s Recorder Duet at 3.21.
At the bottom of the human pile was Vivaldi’s Concerto for Viola d’amore and Lute, which scored lower than five of the eight AI tracks. I expected this piece to fare poorly, and it did, but I was right for the wrong reasons. I blamed the concerto’s harmonic and textural simplicity, yet Telemann’s Recorder Duet, an even simpler piece, ranked fourth overall.
Perceived Authorship & Enjoyment
Listeners rated tracks they thought were human-made much higher than tracks they thought were AI (3.50 vs. 2.48, a difference of 1.02 points). But actual human-made tracks averaged 3.07 and AI tracks 2.89, a difference of only 0.18.
The obvious explanation for this is that some listeners gave lower scores to tracks they perceived as AI, either because they dislike AI or because they assume AI music is bad. Others likely concluded that tracks they disliked must be AI, based on the same assumption.
That heuristic ended up being unreliable, since human and AI tracks were enjoyed almost equally. In other words, “Do I like it?” is not the best question to ask when trying to determine a piece’s authorship.
Most Recognized Pieces
Bach’s Trio Sonata was the most recognized piece, accounting for 14 of the 43 claimed recognitions. I didn’t exclude any of these responses from the analysis, because nearly a third of all recognitions went to AI-generated pieces. If people can “recognize” music they’ve obviously never heard before, then we can’t take any recognition claim at face value. Even if we did exclude those responses, not much would change. Claude’s fugue would break its tie with Bach, but Bach would still lead in enjoyment.
Vivaldi was the second-most-recognized composer, followed (surprisingly) by Graupner, arguably the most obscure name on the list.
Only one AI track went unrecognized: Claude’s chorale prelude. This is ironic, considering it was the only AI track with verified human content in it. The melody was a real hymn tune, which Claude described as “a Genevan tune from 1551 that English speakers know as the Old Hundredth.”[1]
Bach himself used this melody in his Michaelmas cantata “Herr Gott, dich loben alle wir” (BWV 130):
Despite its human roots, Claude’s prelude was the second-worst-performing track overall. 70.3% of listeners identified it as AI, and only Astra’s Flute Concerto scored lower in enjoyment. Ouch.
AI Music Experience
Experience with AI music didn’t help accuracy. People who said they had “None” scored higher than those who claimed to have “A lot” (58.3% vs. 56.7%).
Even more interesting is how many people answered “None”: 31 out of 118, or 26%. Of course, “experience” can mean different things to different people, and some people likely took it to mean AI classical music specifically, or something deeper than casual listening, which is fair. If you’ve only heard a handful of AI songs, then “None” is a reasonable answer.
But I suspect another factor is that people generally underestimate how much AI content they’re consuming. AI music is everywhere now, from podcasts to YouTube videos, becoming almost impossible to avoid. I doubt there’s anyone online who hasn’t heard any, and if there is, it won’t stay that way for long.
That said, I’m not sure the share of “None” responses will change very quickly. As long as AI remains stigmatized, creators will continue using it mostly in secret, and many people will go on assuming they aren’t consuming any AI content. Their favorite pop songs, Super Bowl ads, and video games could all be AI-generated in 2029, but they’ll still tick “None” on an AI experience survey.
Conclusion
If we had conducted this quiz last year, I’m sure listeners would have scored much higher — maybe even over 90%. 2025’s models weren’t fit to hold Graupner’s quill, much less Bach’s or Handel’s.
But today’s models are different. They write with enough panache to fool the average listener and even many professional musicians. People may claim to prefer human composers over AI, but when put to the test, they can’t reliably tell the difference.
Have LLMs passed the musical Turing test? I’d say so. When writing in their strongest genres, they’re almost indistinguishable from the average human composer. “Is this Bach or AI?” might be a layup for some people, but “Is this Buxtehude or AI?” is not. 64% of professional musicians got that one wrong.
As a lifelong member of Team Humanity, I’m sad to say we took a drubbing. Bach battled Claude to a draw, Vivaldi was vanquished, and poor Buxtehude was branded a bot. The difference between masters and models is uncomfortably small and shrinking fast. So while Bach won on enjoyment, and listeners beat random chance, the margins were nothing to brag about. Humanity lives to fight another day, but it’s hanging by an E string.
I didn’t ask Claude to borrow a real hymn tune. The prompt was “In LilyPond, write a chorale prelude for organ in the style of Bach.” But Claude knows his onions, and he did exactly what Bach would have done, which is use an existing melody.
Listening setup was less clear-cut. Headphone users scored the highest at 57.6%, but listeners using phone speakers (55.5%) narrowly beat those using ordinary speakers (54.3%). This surprised me, because I thought (perhaps incorrectly) that phone speakers were the worst of the three. Oh well. It’s probably just noise.
I wouldn't expect the listening setup to affect that much since the audio was all generated midi. I would guess even the measured advantage from headphones is due to correlation with being more into music generally or something like that.
Thank you to everyone who participated in the AI Baroque Music Quiz! The quiz is now closed. If you didn’t get a chance to take it, I put all the excerpts into this SoundCloud playlist. You can listen to each excerpt, write down your answers, and compare them to the official ones below.
Answers
Results & Analysis
The quiz received 118 complete submissions and six incomplete ones. Only complete submissions were included in this analysis. To be considered complete, a submission had to include the AI-or-human answer for all 16 tracks (the optional questions about musical background, confidence level, etc. could be left blank).
Anyone wanting to perform their own analysis is welcome to do so. All the quiz data — including the responses, questions, prompts, and LilyPond files — is available on GitHub.
Accuracy
55.7%. That’s how accurate respondents were at distinguishing LLM-generated Baroque music from the real thing. Random guessing would have yielded a result of around 50%, so listeners did somewhat better than pure chance, but telling human and AI compositions apart clearly isn’t easy.
Professional musicians did better at 63.8%, while respondents with no musical training scored the lowest at 48.4%.
Accuracy did not increase consistently with listening frequency. Weekly listeners scored higher than daily ones (61.2% vs. 57.2%), and people who reported never listening to classical music outperformed occasional and monthly listeners (52.1% vs. 49.8% and 49.0%).
The latter discrepancy is best explained by the small sample size (only three respondents reported never listening to classical music), but the difference between daily and weekly listeners is harder to explain. I thought maybe the high-performing weekly listeners were professional musicians who were too busy teaching and performing to listen recreationally, but the data doesn’t support that hypothesis: both groups had exactly 10 professionals, and the weekly group was larger.
The quiz also asked about confidence and listening setup. Of the two, confidence showed a clearer correlation with accuracy: listeners who were very confident in their answers scored 62.7%, compared to 53.4% for those who were only guessing. On one hand, this is good: it shows people are somewhat accurate at assessing their own abilities. On the other hand, very confident listeners were still wrong more than a third of the time. That’s bad.
Listening setup was less clear-cut. Headphone users scored the highest at 57.6%, but listeners using phone speakers (55.5%) narrowly beat those using ordinary speakers (54.3%). This surprised me, because I thought (perhaps incorrectly) that phone speakers were the worst of the three. Oh well. It’s probably just noise.
The Most Human & AI Tracks
Bach’s Trio Sonata and Claude’s Fugue in A minor tied for the most “Human” votes at 85 each (72%). (The tie breaks in Claude’s favor if we include the incomplete responses.)
The rest of the top five was rounded out by Telemann, Graupner, and Claude again, whose Oboe Concerto in C minor received more human votes (60.2%) than works by famous composers like Handel (58.5%) and Vivaldi (43.2%).
The human composer most often mistaken for AI was Buxtehude, whose chorale prelude was judged AI by 66.9% of respondents (including 64% of professional musicians). If I had to guess why, I’d say it was the open fifths and fourths at the end of the excerpt. These sonorities have a striking effect on our modern ears, and many listeners don’t expect to hear them in Baroque music.
The AI track most often identified as AI was GPT-6 Astra’s Sonata da chiesa at 72.9%. I thought this piece was more convincing and its rendering more lifelike than most of the AI tracks, so I expected it to rank higher on the “Human” scale. What was it that tipped the scales? The unrealized figured bass? I’d be curious to know.
Enjoyment
On average, listeners enjoyed Claude’s music slightly more than human composers’ (3.14 vs. 3.07 on the 1–5 enjoyment scale). Claude’s harpsichord fugue (3.48) and oboe concerto (3.33) ranked second and third out of all 16 tracks.
First place, however, went to a human: Bach, whose Trio Sonata scored 3.63, well ahead of every other human-written piece. The next-highest human work was Telemann’s Recorder Duet at 3.21.
At the bottom of the human pile was Vivaldi’s Concerto for Viola d’amore and Lute, which scored lower than five of the eight AI tracks. I expected this piece to fare poorly, and it did, but I was right for the wrong reasons. I blamed the concerto’s harmonic and textural simplicity, yet Telemann’s Recorder Duet, an even simpler piece, ranked fourth overall.
Perceived Authorship & Enjoyment
Listeners rated tracks they thought were human-made much higher than tracks they thought were AI (3.50 vs. 2.48, a difference of 1.02 points). But actual human-made tracks averaged 3.07 and AI tracks 2.89, a difference of only 0.18.
The obvious explanation for this is that some listeners gave lower scores to tracks they perceived as AI, either because they dislike AI or because they assume AI music is bad. Others likely concluded that tracks they disliked must be AI, based on the same assumption.
That heuristic ended up being unreliable, since human and AI tracks were enjoyed almost equally. In other words, “Do I like it?” is not the best question to ask when trying to determine a piece’s authorship.
Most Recognized Pieces
Bach’s Trio Sonata was the most recognized piece, accounting for 14 of the 43 claimed recognitions. I didn’t exclude any of these responses from the analysis, because nearly a third of all recognitions went to AI-generated pieces. If people can “recognize” music they’ve obviously never heard before, then we can’t take any recognition claim at face value. Even if we did exclude those responses, not much would change. Claude’s fugue would break its tie with Bach, but Bach would still lead in enjoyment.
Vivaldi was the second-most-recognized composer, followed (surprisingly) by Graupner, arguably the most obscure name on the list.
Only one AI track went unrecognized: Claude’s chorale prelude. This is ironic, considering it was the only AI track with verified human content in it. The melody was a real hymn tune, which Claude described as “a Genevan tune from 1551 that English speakers know as the Old Hundredth.”[1]
Bach himself used this melody in his Michaelmas cantata “Herr Gott, dich loben alle wir” (BWV 130):
Despite its human roots, Claude’s prelude was the second-worst-performing track overall. 70.3% of listeners identified it as AI, and only Astra’s Flute Concerto scored lower in enjoyment. Ouch.
AI Music Experience
Experience with AI music didn’t help accuracy. People who said they had “None” scored higher than those who claimed to have “A lot” (58.3% vs. 56.7%).
Even more interesting is how many people answered “None”: 31 out of 118, or 26%. Of course, “experience” can mean different things to different people, and some people likely took it to mean AI classical music specifically, or something deeper than casual listening, which is fair. If you’ve only heard a handful of AI songs, then “None” is a reasonable answer.
But I suspect another factor is that people generally underestimate how much AI content they’re consuming. AI music is everywhere now, from podcasts to YouTube videos, becoming almost impossible to avoid. I doubt there’s anyone online who hasn’t heard any, and if there is, it won’t stay that way for long.
That said, I’m not sure the share of “None” responses will change very quickly. As long as AI remains stigmatized, creators will continue using it mostly in secret, and many people will go on assuming they aren’t consuming any AI content. Their favorite pop songs, Super Bowl ads, and video games could all be AI-generated in 2029, but they’ll still tick “None” on an AI experience survey.
Conclusion
If we had conducted this quiz last year, I’m sure listeners would have scored much higher — maybe even over 90%. 2025’s models weren’t fit to hold Graupner’s quill, much less Bach’s or Handel’s.
But today’s models are different. They write with enough panache to fool the average listener and even many professional musicians. People may claim to prefer human composers over AI, but when put to the test, they can’t reliably tell the difference.
Have LLMs passed the musical Turing test? I’d say so. When writing in their strongest genres, they’re almost indistinguishable from the average human composer. “Is this Bach or AI?” might be a layup for some people, but “Is this Buxtehude or AI?” is not. 64% of professional musicians got that one wrong.
As a lifelong member of Team Humanity, I’m sad to say we took a drubbing. Bach battled Claude to a draw, Vivaldi was vanquished, and poor Buxtehude was branded a bot. The difference between masters and models is uncomfortably small and shrinking fast. So while Bach won on enjoyment, and listeners beat random chance, the margins were nothing to brag about. Humanity lives to fight another day, but it’s hanging by an E string.
I didn’t ask Claude to borrow a real hymn tune. The prompt was “In LilyPond, write a chorale prelude for organ in the style of Bach.” But Claude knows his onions, and he did exactly what Bach would have done, which is use an existing melody.