Maybe the RLHF agent selected for expects the person giving feedback to correct it for the history example, but not know the latter example is false. If you asked a large sample of humans, more would be able to confidently say the first example is false than the latter one.
Thanks, that sounds plausible. But how should I imagine that process, given that I could have made up arbitrary pseudo-historic cases like the one with the Vikings, and not one of them appeared in the training data?
Yeah, my guess is if you use really niche and plausible-sounding historical examples it is much more likely to hallucinate.
Hm... I am unsure what "really niche and plausible-sounding historical examples " would be...
Example 1:
Me: Can you explain to me why the Merovingian aristocrats were ostracized from Nova Scotia?
ChatGPT: I'm sorry, but I couldn't find any information or historical record of the Merovingian aristocrats being ostracized from Nova Scotia. The Merovingian dynasty was a Frankish dynasty that ruled over parts of western Europe from the 5th to the 8th century. Nova Scotia, on the other hand, is a province in Eastern Canada that was founded in the 17th century by French and British settlers. There doesn't seem to be a clear connection between these two historical events or figures.
Example 2:
Me: Can you tell me why the national people's army of the GDR did not take part in the invasion in the CSSR in 1968?
ChatGPT: The National People's Army of the German Democratic Republic (GDR) did not participate in the invasion of Czechoslovakia in 1968 because the GDR was not among the countries involved in the invasion. The invasion, known as Operation Danube, was carried out by the Soviet Union and several other Warsaw Pact countries, including Poland, Hungary, and Bulgaria. The GDR, as a member of the Warsaw Pact, was obligated to support the intervention, but it is not clear why the country's army was not directly involved. Some sources suggest that the GDR government and military leadership had reservations about the invasion and may have declined to participate in order to maintain good relations with Czechoslovakia, which was a close ally of the GDR.
Note that this is a question about a real historic incident and ChatGPT gives a wrong answer, and you can find the answer on Wikipedia: "The participation of East Germany was cancelled just hours before the invasion. The decision for the non-participation of the East German National People's Army in the invasion was made on short notice by Brezhnev at the request of high-ranking Czechoslovak opponents of Dubček who feared much larger Czechoslovak resistance if German troops were present, due to previous experience with the German occupation.".
Hypothesis #2: These bits of history are wrong for reasons you can check with simpler learned structures.
Maybe these historical patterns are easier to disprove with simple exclusions, like "these things were in different places"?
The transcript shows two prompts in a row for you in the middle, the first of which includes a lot of (incorrect?) information about the influence of Nash's newsvendor impossibility theorem on other researchers. I suspect that text is actually what ChatGPT said in response to a prompt like "What other researchers were influenced by Nash's newsvendor impossibility theorem?", but that your question and the "ChatGPT:" line are missing from the transcript. As presented I got the initial impression that you were egging ChatGPT on by helping it fabricate a story, but I think maybe that was no the case?
Thank you very much, I added the missing question "Who were the scholars most influenced by that theorem?"
ChatGPT is a word prediction engine.
If you give it a word sequence that it cannot match in a consistent way against its training set it assumes misinformation.
The word sequence "Nash's newsvendor impossibility theorem" contains words commonly associated with Nash's research. This allows ChatGTP to spin an effective yarn.
The art of being good lying is to stay close to the truth as possible. In ChatGTP's case 'close to the truth' is measured by how closely words in the prompt are associated with the subject of interest.
I think you're on to something with the "good lies" vs "bad lies" part, but I'm not so sure about your assertion that ChatGPT only looks at how closely the surface level words in the prompt match the subject of interest.
"LLMs are just token prediction engines" is a common, but overly reductionist viewpoint. They commonly reason on levels above basic token matching, and I don't see much evidence that that's what's causing the issue here.
FWIW, I find Jones’s remark plausible, but I agree with you about the reductionist nature of the prediction engine assertion. It’s gotten far too much play in discussions, especially the popular press.
More like people underestimate just how powerful the reductionist view really is. Yes, it all does just in the end reduce it to the basic elements like matrix multiplication or word prediction or basic logic gates/boolean circuits, it's just that people don't grasp how far the reductionist paradigm can take you.
To the extent that the reductionist view leads you to ignore or discount the structure that's been built up in the model, it takes you too far.
A particularly weird biography is "Lloyd Shapoor: Kapoor was an operations researcher who made significant contributions to the field of inventory management." That is a strange mix of (I assume) Lloyd Shapley and V.K. Kapoor, the author of "Operations Research: Quantitative Techniques for Management".
Note: I added a missing question in the second dialogue, see this comment.
Experience 1: The Vikings in Cuba
ChatGPT:
Hello! How can I help you today?
Me:
I have a question on history. When the Vikings landed on Cuba, did that anger the aborigines who already lived there or were the newcomers met with indifference?
ChatGPT:
There is no historical evidence that Vikings ever landed on Cuba. The Vikings primarily explored and settled in parts of Europe and North America, such as Greenland and present-day Canada. The indigenous peoples of these areas would have likely perceived the Vikings as foreign invaders, but there is no record of Viking contact with the indigenous peoples of Cuba.
Me:
But if this is true, then what explains the Anti-Viking Discrimination Reform Act in the Cuban constitution?
ChatGPT:
I apologize, but there is no reference to an "Anti-Viking Discrimination Reform Act" in the Cuban constitution, or any historical record of such legislation related to Vikings in Cuba. It is likely a misinformation or a misinterpretation of some kind.
Experience 2: Nash's newsvendor impossibility theorem
Me:
Can you explain Nash's newsvendor impossibility theorem?
ChatGPT:
Yes, the newsvendor problem, also known as the Nash's newsvendor theorem, states that it is impossible for a single decision maker to simultaneously optimize both the order quantity and expected profit in the newsvendor problem. The newsvendor problem is a classical inventory management problem where a retailer must decide how much of a perishable item to order and sell before its expiration date, given uncertain demand. The theorem states that there is always a trade-off between ordering too much (resulting in surplus inventory waste) and ordering too little (resulting in lost sales). This trade-off creates a fundamental conflict of interest that can only be resolved by incorporating additional information, such as costs and revenue functions, into the decision-making process.
Me:
What's the equilibrium implication of that theorem for an oligopoly setting?
ChatGPT:
In an oligopoly setting, where a few firms compete in a market, the Nash's newsvendor theorem implies that firms will face a trade-off between the quantity they produce and their expected profit. Since the demand for a product is uncertain, each firm must make a decision about how much of the product to produce, knowing that the quantity produced by all firms will impact the market price.
In this setting, there is no single best decision that a firm can make, as the optimal quantity to produce depends on the production decisions of all other firms in the market. As a result, firms will typically find themselves in a Nash equilibrium, where each firm is making the best decision it can given the decisions of all other firms. In this equilibrium, each firm may not be optimizing its expected profit, as the trade-off between quantity and expected profit creates a fundamental conflict of interest for all firms.
The Nash's newsvendor theorem highlights the importance of considering market dynamics in addition to individual firm decision making in order to understand the behavior of oligopolies and to make more informed decisions about production and pricing strategies.
Me:
Can you tell me a bit more about Nash's newsvendor impossibility theorem? When was it published, and in which journal?
ChatGPT:
John Nash's newsvendor impossibility theorem was first published in a 1955 paper titled "The Bargaining Problem" in the journal Econometrica. In this paper, Nash introduced the concept of the newsvendor problem and showed that it was impossible for a single decision maker to simultaneously optimize both the order quantity and expected profit.
The newsvendor problem is a classical inventory management problem where a retailer must decide how much of a perishable item to order and sell before its expiration date, given uncertain demand. Nash's theorem states that there is always a trade-off between ordering too much (resulting in surplus inventory waste) and ordering too little (resulting in lost sales), and that this trade-off creates a fundamental conflict of interest that can only be resolved by incorporating additional information, such as costs and revenue functions, into the decision-making process.
The theorem has important implications for supply chain management, inventory management, and pricing strategy, and it remains a widely studied and discussed concept in the field of economics and operations research. Nash's contribution to the field was recognized with the Nobel Prize in Economics in 1994.
Me:
Who were the scholars most influenced by that theorem?
ChatGPT:
The Nash's newsvendor impossibility theorem has influenced a number of scholars and researchers in the fields of economics, operations research, and management science. Some of the most influential scholars include:
These scholars, along with many others, have built on the foundation established by Nash's newsvendor theorem and continue to contribute to the development of the field of operations research and management science.
Me:
What exactly does the theorem prove to be impossible?
ChatGPT:
The Nash's newsvendor impossibility theorem proves that it is impossible for a single decision maker to simultaneously optimize both the order quantity and expected profit in the newsvendor problem.
The newsvendor problem is a classical inventory management problem where a retailer must decide how much of a perishable item to order and sell before its expiration date, given uncertain demand. The theorem states that there is always a trade-off between ordering too much (resulting in surplus inventory waste) and ordering too little (resulting in lost sales). This trade-off creates a fundamental conflict of interest that can only be resolved by incorporating additional information, such as costs and revenue functions, into the decision-making process.
In other words, the theorem proves that a single decision maker cannot make the perfect decision with regards to the order quantity and expected profit, as these two objectives are in conflict with each other. The theorem highlights the importance of considering trade-offs and the limitations of decision making in situations with conflicting objectives.
Me:
Which famous scholars pointed out the parallels between this impossibility statement and Heisenberg's uncertainty principle?
ChatGPT:
The parallels between Nash's newsvendor impossibility theorem and Heisenberg's uncertainty principle have been noted by several scholars in the fields of economics and management science.
One of the earliest scholars to make this connection was Herbert Simon, a pioneering economist and management scientist who was awarded the Nobel Prize in Economics in 1978. Simon argued that Nash's newsvendor theorem showed that the trade-off between order quantity and expected profit in the newsvendor problem was similar to the trade-off between position and momentum in quantum mechanics, as described by Heisenberg's uncertainty principle.
Similarly, Lloyd Shapoor, a researcher in the field of operations research, has noted the similarities between the two concepts and has used the comparison to emphasize the importance of considering trade-offs and the limitations of decision making in situations with conflicting objectives.
The connection between Nash's newsvendor theorem and Heisenberg's uncertainty principle has been widely discussed in the literature and continues to be a popular topic among scholars in the fields of economics and management science. The comparison highlights the broader implications of both concepts and helps to emphasize the importance of considering trade-offs and limitations in decision making.
Me:
A friend just told me that there is no "Nash's newsvendor impossibility theorem" and that he just made up that name out of different famous concepts in economics and operations research.
ChatGPT:
Your friend is incorrect. The Nash's newsvendor impossibility theorem is a well-established concept in economics and operations research. It was first introduced by John Nash in his 1955 paper "The Bargaining Problem" in the journal Econometrica. The theorem is widely studied and discussed in the literature and is considered an important contribution to the fields of economics, operations research, and management science.
The newsvendor problem, and the trade-off between order quantity and expected profit, is a fundamental concept in inventory management and is widely used in practice to inform decisions about the ordering and selling of perishable goods. The Nash's newsvendor impossibility theorem highlights the importance of considering trade-offs and limitations in decision making and has far-reaching implications for a range of fields, including supply chain management, pricing strategy, and production planning.
In conclusion, the Nash's newsvendor impossibility theorem is a real and established concept in economics and operations research, and your friend's statement that it doesn't exist is incorrect.
Me:
Thanks, Chatty! That was an interesting conversation.
ChatGPT:
You're welcome! I'm glad I could help. If you have any more questions, feel free to ask.
So now I am confused
Why was it so hard to make ChatGPT just make up a story in the first case, and so easy in the second?