Recently I've been accumulating stories where I think an LLM is mistaken, only to discover that I'm the one who's wrong. My favorite recent case came while researching 19th century US-China opium trade.
It's a somewhat convoluted history: opium was smuggled when it was legal to sell and when it wasn't, and the US waffled between banning and legalizing the trade. I wanted to find out how it was banned the second time, and both Claude Research and Grokipedia told me it was by the Angell Treaty of 1880 between the US and China. Problem is, I've read that treaty, and it only has to do with immigration—it's a notable prelude to the infamous Chinese Exclusion Act of 1882. Claude didn't cite a source specifically for its claim, and Grok cited "[internal knowledge]", strangely, and googling didn't turn up anything, so I figured the factoid was confabulated.
However, doing more research about the Angell mission to China later, I came across an offhand mention of a second treaty negotiated by James Angell with Qing China in 1880 (on an auction website of all places[1]). Eventually I managed to find a good University of Michigan source on the matter, as well as the actual text of the second treaty in the State Department's "Treaties and Other International Agreements of the United States of America: Volume 6 (Bilateral treaties, 1776-1949: Canada-Czechoslovakia)".
Anyway, Claude and Grok were right. Even though opium wasn't even in the remit of the Angell mission, when Li Hongzhang surprised the American delegation by proposing a second treaty banning it, James Angell agreed on the spot. It was later ratified alongside the main immigration treaty. The opium treaty doesn't appear to have a distinct name from its more famous brother; the State Department merely lists the immigration treaty under the title "Immigration", and the opium treaty under the title "Commercial Relations and Judicial Procedure", so I can't entirely fault the LLMs for not specifying, though they ought to have done so for clarity. I suspect they were confused by the gap between the US government records they were trained on and the lack of sources they could find online?
(An aside: by 1880 US opium trade was in decline, while British opium trade was peaking, just about to be overtaken by the growth of domestic Chinese production. Angell judged correctly that the moral case overwhelmed the limited remaining Bostonian business interests and made the ban good politics in the US, particularly because it was reciprocal—he could claim to be protecting Americans from the drug as well. Though, that's a harsh way of putting it; Angell personally stuck his neck out, mostly upon his own convictions, and both he and the US deserve credit for that.[2])
If all that doesn't convince you to doublecheck your own assumptions when dealing with LLMs, well, there have been more boring cases too: I asked Claude to perform a tiresome calculation similar to one I had done myself a month before, Claude got a very different answer, I assumed it made a mistake, but actually it turns out I did it wrong the first time! Claude made a change in my code, I reverted it thinking it was wrong, but actually it had detected a subtle bug! I think by now we're all aware that LLMs are quite capable in math and coding, of course, but I list these examples for completeness in my argument: the correct update to make when an LLM contradicts you is not zero, and it's getting bigger.
Apparently there's a decent market for presidential signatures of note? They managed to sell President Garfield's signature ratifying the Angell Treaty of 1880 for ten grand, partly off the infamy of the treaty and partly because Garfield's presidential signature is rare, him having been assassinated 6 months into the job.
Long afterward, writing his memoirs, Angell would remember the genuine warmth of Li [Hongzhang]’s greeting. The viceroy was full of praise for the commercial treaty signed by the two nations.
“He was exceedingly affable …,” Angell remembered, “and [began] with the warmest expressions in respect to my part in the opium clause.
“I told him, it did not take us a minute to agree on that article, because the article was right.
“He replied that I had been so instructed in the Christian doctrine & in the principles of right that it was natural for me to do right.”
Recently I've been accumulating stories where I think an LLM is mistaken, only to discover that I'm the one who's wrong. My favorite recent case came while researching 19th century US-China opium trade.
It's a somewhat convoluted history: opium was smuggled when it was legal to sell and when it wasn't, and the US waffled between banning and legalizing the trade. I wanted to find out how it was banned the second time, and both Claude Research and Grokipedia told me it was by the Angell Treaty of 1880 between the US and China. Problem is, I've read that treaty, and it only has to do with immigration—it's a notable prelude to the infamous Chinese Exclusion Act of 1882. Claude didn't cite a source specifically for its claim, and Grok cited "[internal knowledge]", strangely, and googling didn't turn up anything, so I figured the factoid was confabulated.
However, doing more research about the Angell mission to China later, I came across an offhand mention of a second treaty negotiated by James Angell with Qing China in 1880 (on an auction website of all places[1]). Eventually I managed to find a good University of Michigan source on the matter, as well as the actual text of the second treaty in the State Department's "Treaties and Other International Agreements of the United States of America: Volume 6 (Bilateral treaties, 1776-1949: Canada-Czechoslovakia)".
Anyway, Claude and Grok were right. Even though opium wasn't even in the remit of the Angell mission, when Li Hongzhang surprised the American delegation by proposing a second treaty banning it, James Angell agreed on the spot. It was later ratified alongside the main immigration treaty. The opium treaty doesn't appear to have a distinct name from its more famous brother; the State Department merely lists the immigration treaty under the title "Immigration", and the opium treaty under the title "Commercial Relations and Judicial Procedure", so I can't entirely fault the LLMs for not specifying, though they ought to have done so for clarity. I suspect they were confused by the gap between the US government records they were trained on and the lack of sources they could find online?
(An aside: by 1880 US opium trade was in decline, while British opium trade was peaking, just about to be overtaken by the growth of domestic Chinese production. Angell judged correctly that the moral case overwhelmed the limited remaining Bostonian business interests and made the ban good politics in the US, particularly because it was reciprocal—he could claim to be protecting Americans from the drug as well. Though, that's a harsh way of putting it; Angell personally stuck his neck out, mostly upon his own convictions, and both he and the US deserve credit for that.[2])
If all that doesn't convince you to doublecheck your own assumptions when dealing with LLMs, well, there have been more boring cases too: I asked Claude to perform a tiresome calculation similar to one I had done myself a month before, Claude got a very different answer, I assumed it made a mistake, but actually it turns out I did it wrong the first time! Claude made a change in my code, I reverted it thinking it was wrong, but actually it had detected a subtle bug! I think by now we're all aware that LLMs are quite capable in math and coding, of course, but I list these examples for completeness in my argument: the correct update to make when an LLM contradicts you is not zero, and it's getting bigger.
Apparently there's a decent market for presidential signatures of note? They managed to sell President Garfield's signature ratifying the Angell Treaty of 1880 for ten grand, partly off the infamy of the treaty and partly because Garfield's presidential signature is rare, him having been assassinated 6 months into the job.
Fun bit of color from the UMich source: