This is a special post for quick takes by KAP. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
This is an excerpt for an in-progress writeup on prompting:
Prompt Sufficiency
A prompt is sufficient when an agent is able to complete the work product without arbitrary choice.
Sufficiency is not a qualitative property - it is not a function of format, appearance, taste or style. Therefore, the presence of imperatives (”Make no mistakes!”), role-playing directives (”You are a terraform engineer”), or descriptors (”thoroughly…”) do not add to or take away from sufficiency.
Sufficiency is related to the idea of information - it is a combination of deducibility and the absence of a requirement for clairvoyance on the part of the machine. Given the prompt and the available information, is the agent required to know something that only you would know - assuming, of course, that you have a perfectly detailed understanding of what needs to be done? Or, must the agent be unduly creative, clairvoyant, or decisive on your behalf? If so, the prompt is probably insufficient.
Here are examples of insufficient prompts:
Design an appealing dashboard (Is it possible to deduce whether a dashboard will be appealing to you without clairvoyance?)
Create a new strategy for revenue growth based on the last five quarter’s financials (Have you expressed a criteria for what you consider to be an acceptable strategy? Or is this known only to yourself?)
It is possible to develop a nose for insufficiency by making a practice of mentally elaborating on the task at hand. Suppose you are a stranger (assume you have expert background in whatever topic), and that you are asked to complete the work product. Here is the question: Is it possible to deduce the exact form of the completed work using only expert domain knowledge and the prompt? Or are there particulars to the task that are knowable to the asker, but unknown to you? As the asker, ask yourself: What needs to be specified that is particular to the task, not domain knowledge - that it is not possible for the report to know no matter how experienced?