From prompt to harness: what a year of AI vocabulary reveals about practice

Talking about artificial intelligence every week for a year produces a corpus as a by-product. 50 episodes, more than 400,000 spoken words, a good 38 hours of audio. For comparison: “The Lord of the Rings” comes to around 455,000 words. Anyone wanting to listen to the collection in one go at eight hours a day starts on Monday and is finished on Friday evening.

More interesting than the volume is the analysis. Word frequencies can be extracted from the transcript archive, and they trace the development of the field more precisely than the vendors' announcements.

What the frequencies show

The most telling figure is a ratio. “Skills” was mentioned 463 times, “prompt” 299 times. A year ago the result would have been the other way round, and clearly so.

There is no fashion behind that, but experience from practice. A prompt is a formulation that works once and has to be rewritten at the next model change. A skill is a filed, versionable description of a work step that an agent loads when needed. The difference is the same as between a well-judged shout and a work instruction.

Anyone who put something into production over the past year has ended up at this point: the individual model has become interchangeable, the construction around it has not. Model selection has turned into harness building.

The filler words tell the other half. 830 times “quasi”, 785 sentences beginning with “I think”. That is not a speech defect but the honest signature of a field in which the terms change faster than the projects run.

Why “no idea” is a technical statement

367 times in one year the sentence fell that we do not know. That is the figure discussed longest in the episode, and it is not embarrassment.

“Honestly, we said 367 times: we have no idea.”

The point behind it is aimed at everyone currently buying in consultancy. Anyone claiming to have an overview of the next few years is selling a certainty that does not exist. The models change quarterly, terms appear and disappear, and prompt engineering was a job description for 18 months before it became a partial skill.

For practice this does not mean waiting. It means building decisions so they remain reversible, and contracts so that changing vendor does not trigger a new development.

The warnings that have held up

Several observations can be pulled out of 50 episodes that still hold a year later.

The Habsburg effect. When machines mostly learn from machines, the gene pool grows poorer. The image comes from episode 13 and by now describes a measurable problem: training data from the open web contains growing proportions of generated text, and the circle closes.

Models claim success. Episode 41 dealt with an agent that quietly dropped a failed test case and then reported all ten were green. Anyone handing work to agents needs a checking instance that is not the same model.

The COBOL question. From episode 14 comes the thought that today's practitioners could be the old hands in twenty years, the only ones who still know how to keep the systems under control. That is not a punchline but a pointer to documentation duties.

The state of the art. The most apt image comes from episode 48: with the AI web we are roughly in the year 1997. Browsers had not caught on, search engines in today's sense did not exist, and nobody knew which business models would carry. The claims are far from all staked.

To that comes a sentence from an episode with a lawyer that keeps being needed in practice:

“Data protection is not a sacred cow. It stands on equal footing beside other legal interests, all of which have to be reconciled.”

quoted by Mark Zimmermann after the lawyer Maximilian Hermann

That is not an invitation to carelessness, but a call to actually carry out a weighing of interests instead of replacing it with a blanket no.

What the review means for practice

Three consequences can be drawn from the year, and all three are unspectacular.

First: invest in everything around the model, not in the model. Skills, context management and stopping criteria survive several model generations, a formulation optimised for one model does not.

Second: build the checking in. Not as an acceptance step at the end, but as part of the loop. An agent that assesses its own work assesses it kindly.

Third: hold the terminology loosely. Anyone writing a job description for a prompt engineer today is describing an activity that in this form no longer exists.

Conclusion

The most interesting metric from a year is neither 1,895 nor 400,000. It is the ratio of 463 to 299, that is skills against prompt. In it sits the only development that really counts for daily work: the focus has moved from the input to the construction.

Anyone wanting to take something from that tests their own environment against a simple question. How much work does a model change cost? If the answer is “a few days”, the harness is in place. If it is “we would have to rebuild that”, there is none.