Essay by Stefaan Verhulst: “…The need for a science of questions is all the more acute, and growing more acute, because of the rapid advance of generative AI (as emphasized in previous writing: ChatGPT reminds us why good questions matter). AI has brought to the fore the practice of prompting, both as a marketable skill and as a way of eliciting and organizing information by asking generative models questions about the world. But prompting is, in important respects, almost the opposite of genuine questioning. As we argue below, prompting typically begins from a preconceived sense of what a good answer looks like, whereas questioning begins in genuine not-knowing. As machines make answers ever cheaper and more abundant, the capacity that grows scarcer and more valuable is the one prompting does not train (and perhaps actively degrades): the ability to formulate the questions worth asking in the first place. That is the distinction this essay draws out, and it is why the case for a science of questions has become urgent in the age of AI, rather than merely an interesting theoretical exercise.
What a Prompt Is
A prompt is defined by its relationship to a system (or person) that will complete it. It is primarily instrumental: the prompter has some model, explicit or intuited, of how the system responds to inputs, and shapes the input to produce a desired class of outputs. Good prompting is a form of engineering, and its virtues are the virtues of engineering: precision, reproducibility, efficiency, minimal ambiguity, and a clear line between intent and result.
Most prompting, most of the time, also begins from a position of relative epistemic confidence. The prompter may not be able to produce the output themselves, but they can usually recognize a good result when they see it — that’s what makes iterative refinement (prompt, assess, revise) possible. In this sense, a typical prompt is closer to a query than to a question: a request for something whose contours, even if not exact form, the asker already knows.
This is not intended as criticism, and these observations are not categorical or absolute. Prompting can at times be exploratory, and some of the most interesting prompting deliberately courts surprise. But in general, prompting describes a particular and historically recent practice, one characterized by a relationship between an operator and an instrument. The mistake is not in the practice itself. It is in letting that relationship stand in for the far larger territory of human questioning, most of which has nothing to do with directing any system at all…(More)”.
