Think Different. Think AI. Transcript archive

KI Effizienz

Published Duration 1 h 32 min

Auf Deutsch lesen

Topics SoftwareentwicklungInterfaces und InteraktionWissensmanagement

Guest Alexander Heusingfeld

What it is about

Was KI-Effizienz wirklich kostet: Abnahmekriterien, Gegenkennzahlen und warum die Zeit zwischen den Arbeitsschritten entscheidet

Firmen werfen 80 Prozent ihrer Leute raus, weil das jetzt die KI macht. Diese Meldung geht seit Monaten durch die Presse, und Alexander Heusingfeld möchte nachrechnen. Wer 80 Prozent entlässt, behauptet damit den Faktor fünf. Klarna hat den Schritt für den Kundenkontakt inzwischen wieder zurückgenommen. Und selbst wenn die Rechnung aufginge, bleibt Alex' eigentliche Frage offen: Woher will jemand vorher wissen, welche 20 Prozent das Skillset haben, der Maschine beizubringen, was am Ende herauskommen soll? Wer selbst noch nicht ausprobiert hat, was möglich ist, kann das nicht beurteilen.

Mark dreht die Frage um. Statt „wen brauchen wir nicht mehr" interessiert ihn, was bisher liegen geblieben ist, weil nie jemand Zeit dafür hatte. Beide landen bei derselben Beobachtung: Regulatorik wie NIS-2 bleibt liegen, weil die Kapazität fehlt, teilweise sogar bei Behörden. Über jedes Stück stupide Arbeit, das eine Maschine übernimmt, kann man sich also erst einmal freuen. Die freigeräumte Zeit ist ein Invest, kein Einsparposten.

Dann erzählt Mark, was an einem Wochenende passiert ist, an dem er eigentlich Schränke aufgebaut hat. Auf einem Mac Mini hinter dem Fernseher, ohne Skills, mit nichts als Zugriff auf seine alten Git-Projekte, hat ein Agent sich das Smart Home angesehen und vorgeschlagen, die Lautsprecher im ganzen Haus als Interface zu nehmen. Danach lief die Arbeit über Sprache, während Mark durchs Haus ging. Der Agent hat ein Jahre altes Projekt fertiggestellt, sich Xcode geholt, Bundle Identifier und Signierungszertifikate selbst gebaut und die App am Ende über TestFlight aufs Telefon geliefert. Mark musste nur ein paar Kennwörter eintippen. War das effizient? Er sagt selbst: mehrere Wochenlimits verbrannt, garantiert nicht. War es das wert? Absolut.

Alex hält dagegen und stellt die zwei Fragen, um die es in der Folge geht. Was war der Harness, und welche Rahmenbedingungen hast du mitgegeben? Er selbst hat sich mit einem Agenten über AWS die Steuererklärung vorbereiten lassen: knapp 800 Belege, das Ziel „mindestens 80 Prozent einsortiert und formal richtig", vier Stunden Rechenzeit, 60 Euro. Zurück kam auch der Hinweis, wo der Steuerberater im Vorjahr etwas anders hätte abrechnen können. Aus dieser Erfahrung wird eine Metrik, die er für die brauchbarste hält: Verifikationsaufwand pro ausgelieferter Änderung. Sinkt sie, waren die Abnahmekriterien besser.

Daraus werden drei Regeln. Nie eine einzelne Kennzahl, weil jede einzelne Zahl gegamed wird, sondern immer ein Faktor aus Kriterien und Rahmenbedingungen. Immer Gegenkennzahlen mitlaufen lassen, Change Fail Rate, das älteste Work in Progress, Developer Experience. Und niemals selbstberichtete Geschwindigkeit: Entwickler geben 25 Prozent Beschleunigung an, gemessen wurden bis zu 20 Prozent Verlangsamung, weil die Zeit in Reviews und Nachbesserungen zurückfließt.

Mark ergänzt die Kette drumherum. Wenn einer drei Tage durchlaufen lässt, steht das Team vor einem Pull Request, der aussieht wie ein gefluteter Landstrich, das Backlog ist am dritten Tag eines Zweiwochensprints leer, und die Tester werden überrannt. Codekommentare, früher Pflicht, führen Agenten heute in die Irre, weil sie den Kommentar lesen statt der Routine. Und wer Generierung durch Agenten reviewen lässt, die wieder von Agenten kommentiert werden, verbrennt Tokens für das Gegenteil von Effizienz. Alex bringt dazu eine Untersuchung aus dem Flugzeugbau mit: 8,5 Prozent der Durchlaufzeit sind wertschöpfend, grob zwei Wochen Wartezeit pro Tag Arbeit. Die Antwort auf „effizienter werden durch KI" ist damit keine Werkzeugeinführung, sondern Flow und Governance. Oder in Alex' Bild: Erst klärt man, ob man Fußball spielt, danach wird das Spiel schnell.

Zum Schluss lösen die beiden die Diktiergerät-Geschichte aus der letzten Folge auf, Mark verrät sein Setup, und Alex fasst zusammen. Wer KI nur als Werkzeug kauft, verkürzt einen Arbeitsschritt. Die Effizienz liegt in den Wartezeiten dazwischen. Das richtige Maß an Qualität entscheidet sich gegen die Anforderung, alles andere ist Informatiker-Romantik. Und Marks Zugabe für alle, die zögern: Wer nicht weiß, dass er scheitern könnte, probiert es gar nicht erst.

Listen to the episode As Markdown Read the article

Transcript

00:00:00Welcome to Think Different. Think AI., the podcast by Mark and Jens.

00:00:07Two technology-loving minds who do not just talk about artificial intelligence, they live it.

00:00:14Here you get clear perspectives, real practical insights and a fresh look at what is possible.

00:00:20Understandable, critical and always with a wink.

00:00:24AI to think about, to smile about and above all to join in on.

00:00:29A warm welcome to Think Different. Think AI.

00:00:36We now have, I would almost say, something like a regular player here on

00:00:41our podcast.

00:00:42Yes, Alex is with us again today.

00:00:44I got quite a lot of good feedback on our last episode, which, as

00:00:49you will remember, turned out a little longer.

00:00:51I am very curious how efficiently we will keep things today, and efficient is also

00:00:55the magic word.

00:00:56I made it particularly efficient for myself, because I had Alex write up the thread

00:01:01for today's podcast, which means it was delegated right away,

00:01:05which is wonderful.

00:01:07Alex, good to have you here.

00:01:10So what is the title of our episode, co-host?

00:01:12Yes, thank you for having me again.

00:01:18It is very interesting that you got positive feedback.

00:01:21I have been told it was entertaining.

00:01:24Well, my friend, whatever that means, better than being so dry that people switch off.

00:01:31Yes, although, I have to say, listening back I did think, oh, we really had a lot of

00:01:37terms in there that went quite deep into the technical side, but well, I think

00:01:43that probably will not, or so I assume, hope and believe, happen to us today.

00:01:48It is about becoming more efficient through AI.

00:01:50Exactly.

00:01:51Yes, very good.

00:01:52And I thought, well, the topic is really on my mind, and since I know that you

00:01:59have given it some thought too, and I know a few concrete examples from you,

00:02:05I thought, go on, prepare it, because that is the done thing when you

00:02:10bring a topic along as a guest.

00:02:12Yes, exactly.

00:02:13Yes, just like that, that is the secret of this podcast, you see, we basically have

00:02:17no idea at all, and we are always glad about every guest who takes a bit of the

00:02:21hosting off our hands so we are not just sitting there. Greetings go out to Jens. As you may have

00:02:26noticed, Jens is not with us again today. He will be back, he will be back, he will be back.

00:02:30I am glad you are here today. When I opened the script, I found

00:02:35a few great ones in there. How shall I put it, hooks? Let me say, a few

00:02:42hooks caught my eye. Stories along the lines of, you might have

00:02:4880 percent. Let us see what we make of those 80 percent later. Yes, the word

00:02:55efficiency comfort, the ever-present flow comfort, numbers, events, loops. I am curious

00:03:00how epic a breadth we will treat the topics with, and whether we will not also deliver

00:03:04some practical value for our listeners along the way. But let us

00:03:10start with the 80 percent. What really annoys me is when people simply do not take the time

00:03:23to engage with a complex topic and its influencing factors. And so you constantly hear

00:03:30these striking claims in the press, like companies throwing out 80 percent of their staff

00:03:36because they can now do it with AI. And there is not just one company

00:03:41making that claim. We know about Klarna, who are now backtracking, because they

00:03:46have realised, well, maybe the things we wanted AI to handle, things like

00:03:51contact with the customer, maybe that was not such a good idea after all, we shall see. But in the end,

00:03:57if I throw out 80 percent of my people, that means AI makes me five times as

00:04:06efficient.

00:04:07Which is to say, achieving the same with 20 percent of the people.

00:04:10There are two things in that.

00:04:14Sorry for being so unrestrained, I cannot keep myself in check, which keeps me

00:04:17from, well, it stops me cutting in on you, but you are a grown-up, you can

00:04:20tell me off now.

00:04:21I think there are several things in there.

00:04:23One is this propaganda along the lines of, we are all going to lose our jobs and 80 percent

00:04:29get thrown out because we do not need them anyway.

00:04:31If we extrapolate that, yes, maybe someone says 60 percent or 50 percent, I would

00:04:34not insist on the exact figure, but rather on this propaganda aspect,

00:04:39because it, let me say, perhaps also through the odd misdirection in the

00:04:43political sphere, gives people a simple answer to complexity, namely

00:04:47along the lines of, with AI you have to do this, otherwise you do not belong at all.

00:04:51And if you deploy AI, then you can do it.

00:04:53You only have to deploy AI, everything is great, right?

00:04:55If you do not do it, it is your own fault.

00:04:57When I sometimes listen to those influencers

00:04:59who flutter back and forth on their YouTube channels,

00:05:02fluttering, advertising, fluttering, what kind of word is that?

00:05:05If it is new, please register it, as the inventor.

00:05:08Yes, that is the follow-up to bullshit.

00:05:11At some point it turns into flannel and...

00:05:14Anyway, never mind, I will carry on.

00:05:16Okay, thank you. You are almost throwing me off,

00:05:18but I hope I am still roughly on track.

00:05:20That is one thing, and the second is what I always find so interesting, if you now say, okay, even if it were the case that you say, I will cut 80 percent.

00:05:31Those 20 percent, they will rock it, with the AI or however.

00:05:35Then that is always measured against the status quo.

00:05:38They say, along the lines of, those 20 percent can do what the 80, what the 100 could do, because factor five.

00:05:43But this question, what am I actually capable of if you structure work differently,

00:05:49what might I be capable of doing that I never got around to before, where people

00:05:54said, honestly, if we want to do this topic, we simply do not have enough

00:05:58staff, we have to bring in an external service provider, do you have the money for that

00:06:02and so on and so forth, those are things that otherwise come up in companies now and again

00:06:07and should.

00:06:08And if you now say, okay, I can work more efficiently, in quotation marks,

00:06:12my sentence would not be get rid of the people, but rather, what could we do?

00:06:17What else is being left undone? What changes about the way we work, and not: the work

00:06:24I did a year ago is now done by AI. I notice it myself. And then

00:06:28I listen, with my verbal diarrhoea, the next word this podcast has carried,

00:06:31to the things I occupy myself with, and I could not have done them last year. For

00:06:40one thing, because the models and systems were partly unknown to me or did not yet exist, and because the way you work changes as well.

00:06:50Because you increasingly get a feel for when to deploy AI and how to deploy AI.

00:06:56What you used to learn as Let Me Google That For You, I could find something in Google and a colleague could not.

00:07:01So I sent him Let Me Google That For You as a video, so that he would go, oh right, I only had to type it in like that and it would have worked.

00:07:06That way you develop a different approach, and that is what I see as the second factor. On one side the prophets run around preaching and promise you the moon, in order to somehow

00:07:14advise companies and goodness knows what. Greetings to the McKinseys of this world.

00:07:19And the other thing is, the field of work changes as a whole, and one really ought to look at the question of what we get out of it,

00:07:26for each individual and for us as an organisation, and not just this simple back-of-the-envelope reasoning along the lines of

00:07:34Hey, cool. Double the efficiency, triple, quadruple, five times over, you just draw a line under it.

00:07:39The AI is to blame, so we have a culprit too, yes, and we can explain it as well.

00:07:42The AI is the one taking my job away, which is also great, so we have a culprit we can put in the pillory.

00:07:48It is not mismanagement or anything like that, right.

00:07:51That is the very convenient product.

00:07:53And specifically... Thank you.

00:07:55Bear with me.

00:07:56Honestly, Mark, I do not want to...

00:08:00Well, in my view you have just said exactly the right things.

00:08:03I think that is really how it is, and I do not want to disparage anyone's bar-stool chat,

00:08:08but in effect that is exactly the kind of sweeping bar-stool grumbling it is.

00:08:13There is at the moment, and I would have said this in the preparation too, simply no figure that is

00:08:22arithmetically available, no figure where you could say, here is the mathematical proof.

00:08:27Because the crux for me is, you can go ahead as the boss of a company, you

00:08:34can go ahead and say, I will throw out 80 percent of my workforce, and the 20 percent

00:08:40who are left will just have to manage somehow.

00:08:42That creates pressure, and probably those 20 percent will then be under strain.

00:08:48No, but how shall I put it? No, I, erm, let us not go there, that is a rabbit hole, we do not need to go into that right now.

00:08:57But what I am getting at is that this pressure then makes people say, damn it, the way we have worked so far, we cannot carry on like that.

00:09:07And that is exactly the mindset you actually want with AI, where you say, hold on a moment,

00:09:15we have an entirely new kind of tool here, and an entirely new way of approaching

00:09:25a problem at all.

00:09:26And that requires us to rethink our problem-solving strategies and our way of thinking, so

00:09:32not just the way we act, as it were, how do I do process optimisation, but

00:09:38to think about the topic of process optimisation itself all over again. So that is effectively

00:09:42the meta level. And now I come to my point. If I throw out 80 percent of the people,

00:09:51how do I know that the 20 percent I kept are exactly the ones with the skill set

00:09:58to teach the AI how it should produce the result the company

00:10:04is meant to have in future. So if I have not yet engaged with AI

00:10:08and have not yet tried out in practice what is possible, how do I know then

00:10:13which skills I would need in order to implement that.

00:10:15What I wanted to bring in just now, when I, erm, and I cut in on you in a rather

00:10:20unfriendly way, was when you were talking about process optimisation.

00:10:23I mean, the word skills has come up a few times here on the podcast and in our conversations as well.

00:10:31And I do think that something like skills is the second chance, for one thing because it is simply an easy way of teaching the machine things.

00:10:40I keep experiencing that when people work in chats and you then explain to them at the end how to turn that into a skill and what to watch out for in order to get better reproducibility and so on, as one thing.

00:10:50But if you get something of a handle on which skills actually exist,

00:10:55so that you do not end up with the 527th skill on the topic of requirements engineering or whatever,

00:11:01but perhaps with a handful, or counting on two fingers or one finger,

00:11:07by doing that you kill several birds with one stone, namely you manage

00:11:12to bring the AI tool closer to people, to get them engaging with AI,

00:11:17to explore the edges of their job. The other day I heard a talk asking what happens

00:11:22if the product owner now becomes a software developer too. I think I would sometimes have to look

00:11:25in the mirror on that one, because your way of working changes, because you have new possibilities at

00:11:31hand, and with skills you get the whole thing, let me put it crudely, handed to you in a more standardised,

00:11:36process-conformant and automated way, and that way you also get the issue

00:11:43off the table. Let me call it, what shall I say, water-treading. Every project

00:11:48with its own expertise, years of experience, you have to do it this way and no other.

00:11:52And yet in the next project it is done a touch differently or implemented a touch

00:11:56differently. No, by the by that counts towards the completion status,

00:12:00by the by that counts towards the completion status. No, if

00:12:02that counts towards the completion status, that you get some consistency into it,

00:12:06without going in somewhere with a crowbar and

00:12:10saying, right folks, we are now all doing training courses A, B and C so that we are all aligned on whatever

00:12:16procedures are out there, and then everyone goes back to doing their own

00:12:21little thing anyway, but instead you say, okay, good, what are the best practices, we

00:12:25put those into the AI, the AI takes care of a great many things, takes a great deal off your

00:12:29hands, this time you have to work with a backlog with the help of AI, yes, we could

00:12:33perhaps talk later about the plot and whether that helps you or not, the thing we have,

00:12:37the one I already reported on in my Tesla episode with Jens from the holiday,

00:12:41it changes the way you work so much, and in the same breath it can help you

00:12:45introduce standardisation somewhere where previously, depending on the company, you might have

00:12:50despaired. Yes, and at the same time... Maybe at this point, for both of us,

00:12:54I just want to put this out there, when we talk here, we bring

00:12:59points into it, we are in contact with a great many people, so we have small

00:13:02chat groups, we have our LinkedIn community, we allow ourselves again and again to report

00:13:06on things that we read there in part.

00:13:08So that nobody says,

00:13:10oh look, he goes around every day with a club

00:13:12saying, here, process standardisation through skills.

00:13:15Yes, so from that side I just wanted to put that

00:13:18before the audience once more.

00:13:19He does, he runs around all day with a club anyway

00:13:23and beats everyone about the head with it.

00:13:26You could say I have a bit of an image like that at the moment, right?

00:13:28With the, well, that is another topic.

00:13:30Okay. You will have to google that one.

00:13:32But I would like to come back to this once more.

00:13:36The crux of process automation for me is that we make ourselves

00:13:43aware, when we use skills, that they are precisely not deterministic. Which means,

00:13:49in particular if I say I want a deterministic quality check,

00:13:54then skills are not suited to that. And for me that is also the difference to,

00:14:01for example, robotic process automation or to a workflow engine, where I would

00:14:05say, that runs through exactly the same steps 100 percent of the time, and what you just

00:14:13said about the fields in Jira, you are not standardising because you want to drive

00:14:20free will or creativity out of people, but because you say, folks, be

00:14:24honest, does it make a difference for the end customer in the end whether we take this field or

00:14:30that field. Which is to say, put differently, which changes, which quality metrics

00:14:40change something for the customer out there, change the cost-value ratio of what I deliver

00:14:47as a service to the customer. And that is my plea. When we talk about efficiency,

00:14:52then we can talk about task speed-up and about improvements at the micro level,

00:14:59but ultimately it is the cost per business outcome, that is, the cost-value ratio.

00:15:06When you engage with the topic of efficiency and AI, then very quickly the question comes

00:15:14round the corner, so where are the figures behind it?

00:15:17And how do we pin that down in the books?

00:15:19Because we just gave the example with 20, 30, two, three, four, five times over, 80 percent

00:15:25of the workforce.

00:15:26Now you stand there and say, if we introduce AI, then it will bring us A, B and C.

00:15:31Now I do not know what our listeners' experience is.

00:15:36But when I look at my feed again, what mostly comes up is a great deal of guesswork.

00:15:41How am I supposed to quantify this efficiency that AI promises me, in a way

00:15:45that lets you actually get started with AI projects, that lets you deploy it

00:15:50and lets you do the maths somewhere so that someone says, of course you get

00:15:54X million euros for tokens, whatever, so you would have thrown some figure into the room.

00:15:59And then you notice very quickly, depending on where you are operating,

00:16:03in order to demonstrate this efficiency, quite apart from the question of

00:16:07what KPIs do they actually already have today?

00:16:10What metrics might they already have today to pin it down?

00:16:14Let me give you a tip, lines of code is not necessarily the best KPI,

00:16:17because AI will beat that one hands down.

00:16:22I think this question can largely be answered by how well you can handle AI,

00:16:29how well you can deploy it, how well it suits your particular topic. Along the lines of,

00:16:35if I set a task here, how much time do I need, how much effort do I have to put in

00:16:39until the AI completes its loop, its work, how long does it run, with which model does it run

00:16:46or does it need, how many tokens go through the ether, up the chimney,

00:16:50through Elon's gas turbine, who knows, until some result comes out, which I

00:16:56then look at again, approve or whatever, in the best case, in the worst

00:17:01case start over again, and again, and again. And without going into the whole topic of

00:17:06performance monitoring or anything like that, just saying, okay, fine, is

00:17:10this on balance a case that might be suited to AI, is this perhaps a

00:17:13topic where I should invest in Mr Zimmermann's further training,

00:17:17because he started the same loop for the eightieth time, where someone else with a loop in the same category managed it after a single loop.

00:17:26And I think that makes the matter additionally complicated.

00:17:29Where people used to say lines of code and goodness knows what else, what have you got now?

00:17:33Okay, I formulated a problem, I checked it.

00:17:36It took a certain amount of time, I kicked it off again.

00:17:39kicked it off, and between kicking off and checking half an hour passes, which is

00:17:44developer capacity, if we are talking about software development, and tokens perhaps,

00:17:47who knows, a day, a week of work in the past, that is no less

00:17:52complex to evaluate as a whole. Let us jump into concrete

00:17:58cases, because in the end the question is how do I actually measure the

00:18:04quality of the outcome, meaning, was it worth it to me? So cost value, what comes out and

00:18:12was it worth it to me? I would like to close off the part before that, about throwing out those 80 percent

00:18:19of the people. The crux for me, as I said earlier,

00:18:24is that from my point of view I cannot know in advance which people I really

00:18:28still need and which I do not. So I would be very, very careful with the

00:18:33example, Klarna shows all the customer service people the door, because yes, customer service is

00:18:38obviously going to be replaced by AI, but then again maybe not.

00:18:43And honestly, as managers we can actually be glad about every bit of capacity

00:18:49that is freed up among our staff, because from my point of view we have a certain amount of

00:18:55regulation that is not excessive but absolutely necessary, such as

00:19:00the CRA or NIS 2. And when I hear today that federal or state authorities

00:19:08are exempt from it because they simply do not have the capacity to manage it

00:19:14in time, to become CRA or NIS 2 compliant, then we can only be glad about

00:19:21every bit of mindless work, that is, repetitive work, that AI takes off our hands,

00:19:28even if it is filling in a form, where you take a photo of the handwritten

00:19:35form from Mr Müller and enter it into some system.

00:19:39And we should be aware once more of which efficiency we need where, and from my

00:19:46point of view, with what is coming at us right now, the AI models also enable attacks,

00:19:51and for that we need the creativity and the brainpower of people, so everything

00:19:56we can free up there by automating mindless things we should do, and it does not only work through AI.

00:20:03I also wanted to say thank you at this point, because you said you wanted to finish your thought, and I jumped straight in.

00:20:09That was topic one, which you had put on our list, and I had already jumped to several topics.

00:20:14So from that side, thank you.

00:20:15Perhaps at this point also a wish sent out into the ether: if work packages do fall away, because people now say,

00:20:23well, writing A, B and C, reading the forms, the AI can do that now, then

00:20:30freeing up time is also very much an investment in the opportunity to give people

00:20:36the space, and to learn how to handle this stuff as well.

00:20:42The two of us have had this discussion before, and with others as well, about the question

00:20:47of what a desirable target state actually is.

00:20:49And you have to drive in intermediate pegs in good time, because on the one hand you must not overwhelm the system, at the end of the day

00:20:57people have to come along too, so that the whole thing is adopted, works and is accepted, so that you can go on the journey, and the second thing is, you also need

00:21:06reachable goals,

00:21:08because we live in a world in which all this AI business is extremely fast-moving. We talked about it in the preparation,

00:21:14I had said something about being impressed

00:21:17by, for example, the image generation of the new Astra model from OpenAI.

00:21:22Even we, who deal with these topics, are regularly

00:21:26surprised by the possibilities that open up, because a tool or a model

00:21:31or some clever person has somehow pushed another one of those repository projects

00:21:34out in the course of a day, and we get surprised as well.

00:21:38And it is not as if we can look back on twenty years of

00:21:41Henry Ford and the like and say, yes, it stays like this, this is probably how it will

00:21:47carry on, instead it has become much, much faster-moving. And here too the rule applies,

00:21:52careful, old platitude retold, every journey begins with the first step, and we

00:21:58carry on, instead it has become much, much faster-moving. And here too the rule applies,

00:22:04careful, old platitude retold, every journey begins with the first step, and we

00:22:10should take that step again rather than running after all these apostles of 80 percent. Right,

00:22:16that was my contribution to that. What is our next point? I find the example

00:22:25rather nice, where you mentioned the Astra image generation. I have scenarios where I,

00:22:32what I said in the Personal Assistants case, where we take knowledge from Confluence,

00:22:38from the knowledge base, from our Backstage developer portal, or where privately I take notes,

00:22:45for example about my smart home, where I now have over 170 components

00:22:50and the documentation has gradually grown a bit larger, so that I create knowledge

00:22:51blocks from it with the help of skills, so that I can then ask the

00:22:59AI about it.

00:23:00And I notice that with some answers I am not in agreement with the efficiency,

00:23:05or rather not satisfied with it.

00:23:11Which means the tokens the thing consumes and the costs that arise from that,

00:23:14compared with the answer it delivers, are not acceptable to me. And on that I would

00:23:22simply like to hear from you, because I believe you have already put considerably more

00:23:28tokens through than I have. How do you handle that scenario? From my

00:23:32bubble I picked up this rule of thumb, along the lines of, well, if you have an agent

00:23:38generate pull requests, then you need at least a 70 percent acceptance rate

00:23:44where no human has to touch it any more. Otherwise the review costs are higher

00:23:49than what you put in. I thought that was an interesting rule of thumb as well. But do tell.

00:23:53That is nice. Thank you for the spontaneous question. I shall have to withdraw for half an hour

00:23:59to start on my politically correct answer. I would like to, well, we are among ourselves here. I will pin it on an example. We had, Astra came

00:24:03out, and that completely thrilled me, and I wanted to apply it to old projects.

00:24:04And now the scenario is, we talked about it in the last podcast as well, I got myself a Mac mini.

00:24:09Quick interruption, when you say you wanted to apply it, do you then simply use Codex as

00:24:16your standard harness without skills, or did you prepare something?

00:24:20I wanted to lay it out briefly, I wanted to in my build-up, thank you for

00:24:23drawing my attention to it, I certainly will not forget it.

00:24:25I started with the sentence. We have, as we already heard on the podcast, a Mac mini, and the Mac mini does

00:24:31certain things automatically on a regular basis, and that is effectively also the object of desire at home.

00:24:38I mean, I have got nothing else, I have only got that one, well, I also have a notebook and a work notebook, but this is about the Mac mini.

00:24:46And I installed Codex on it, the way OpenAI offers it to us, and yes, I have a larger Codex.

00:24:54Which means Astra was made available to me, and then I first worked with the thing and thought, okay, fine, I would like to go through a few of my projects with it.

00:25:03And you have already asked the question, did you give it skills or not? I gave it nothing at all.

00:25:07I only gave it access to my Git projects, and I simply wanted to see how Astra could help me there.

00:25:14So, there you are, the Mac mini sits behind my television, it really only has

00:25:20a remote connection, so I dial into it from another machine,

00:25:24and that is the one form of interaction I have with it.

00:25:27And I first discussed with Astra a bit how I could now use Astra really

00:25:32well, because I did not want to be constantly, when I wanted to work through these eight projects, I did not

00:25:36even know yet how that would go, how I was supposed to

00:25:40handle things with it, because I cannot spend the whole livelong day walking around the house

00:25:45with a computer in front of my nose, my son was painting his room,

00:25:49assembling cupboards and goodness knows what. And then the thing came up with a very interesting

00:25:55idea, namely it looked at my smart home and suggested

00:26:01using my Sonos speakers, so that I could effectively move around the house

00:26:05for the interaction. And I agreed to that. And the system then went ahead and,

00:26:10we have them in every room. I see. And then the thing effectively put all the speakers

00:26:16into recording mode and kept checking which speaker could hear me

00:26:21best. Which was already quite spooky in itself. In any case, we then had a

00:26:25voice interface. And then via the voice interface, and to be fair via the

00:26:30remote, so via the keyboard and remote control, it had gained access to

00:26:33my personal Git projects, and I then discussed with it

00:26:38what I would like, and told it, so you will find this repo and that

00:26:41repo, no I cannot find it, so I said, yes, it is private, it is old, go and look again,

00:26:47yes, found it, and then I talked to it about these projects, and it then

00:26:52built up its own subtasks internally, so it effectively, I talked to one chat

00:26:57and it then started other chats, which then carried out the project work. And how,

00:27:02I told it, look at all of them, make me suggestions based on what you see there, on what you believe the goal of my project is.

00:27:12Make me suggestions for reaching that goal.

00:27:16Then it set off and whirred away.

00:27:20At some point the suggestion came and it said, this project is about this topic.

00:27:25You want to release an app on iOS that deals with this topic.

00:27:31Back then it was not yet called Second Brain, the thing I was working on, evaluating all your

00:27:36screenshots that you take, because in the past, whenever I wanted to remember something,

00:27:41I took a screenshot, and I always ended up with a great many screenshots.

00:27:45And the app I wanted to build was meant to collect those, analyse them with Apple

00:27:48Intelligence and then tell me, look, there is an appointment in here,

00:27:52there is something else in there.

00:27:53So as to hand it back to me, so that when I have taken a screenshot,

00:27:57it does not get lost.

00:27:58And back then I never finished it.

00:27:59And that is one of those apps, for example.

00:28:01And it then said to me, listen, I have seen you intend to do this, blah, blah, blah.

00:28:04It told me all that, and said, to get it finished I would suggest the following.

00:28:08Then we really did discuss it while assembling the cupboard.

00:28:12Over the speaker.

00:28:14Over the speaker.

00:28:16It told me all that and I answered.

00:28:18Seriously?

00:28:20Okay.

00:28:21Yes, fine.

00:28:22Let me put it this way, the worried looks of the other inhabitants of this house.

00:28:25Exactly.

00:28:26Which means your family was keeping their lips tightly shut during that time.

00:28:32Do not breathe too loudly, lovely.

00:28:36Why is that so bad?

00:28:38That is the sandbox Klaus is always going on about, right?

00:28:41Yes, yes, of course.

00:28:42Transfer, we also do bank details via transfer.

00:28:45But why am I digressing?

00:28:47Let us stay with this example of this app.

00:28:49And then the thing was ready, and at some point, it was quite funny actually.

00:28:52Did it go quiet?

00:28:53Then nothing came back at all, so I went downstairs, connected to the device, and then

00:28:57it said weekly limit used up.

00:28:58But Codex offers you the option of saying...

00:29:02In exchange for inserting small coins?

00:29:04No, no, no, you have, right now with Astra, you see, if people

00:29:08could have Astra but do not have Astra, then for every day they

00:29:12do not have Astra, that is, are delayed, they get one chance to unlock a weekly

00:29:18limit.

00:29:19So, since I have the machine anyway, I did that with the weekly limit and then it was

00:29:26very talkative again.

00:29:27I could talk to it again.

00:29:28To cut a long story short.

00:29:29At some point I said to it, you know what, I am assembling a cupboard here, from now on

00:29:32you decide everything yourself.

00:29:33I do not want to be disturbed by you any more at all, you have everything you need,

00:29:36you cleared it with me.

00:29:37I want the finished result on my phone.

00:29:41Right, listen.

00:29:44Fine.

00:29:45I would like to know how you got onto your phone as well.

00:29:47Okay.

00:29:48I can see that now. I was going to carry on anyway.

00:29:51That sounds very efficient at any rate.

00:29:54Well, I was able to assemble the cupboard at any rate.

00:29:56Whether I still have much enthusiasm left for assembling cupboards

00:29:59is another matter.

00:30:00Which is to say, the fact that I am still talking today.

00:30:02It speaks for the fact that I did not do myself in.

00:30:05When it comes to manual work I am more of a...

00:30:08So, greetings go out to my father.

00:30:10He always helped me with arts and crafts.

00:30:12He is delighted every time

00:30:14I do not do myself in while hammering a nail into the wall.

00:30:17Right, back to the content, it built the app and then it came back with yes and furthermore, and

00:30:25it downloaded Xcode.

00:30:26But now it would need a login to my App Store account, whether I could just do that

00:30:31quickly.

00:30:32So I connected again, logged in, gave it the credentials for

00:30:37App Store Connect and for Xcode, to cut a long story short, it built the bundle identifier and signing certificates

00:30:45all by itself and then at the end put the app on TestFlight,

00:30:49internal testers, and told me, there you are. And internal testers means

00:30:55you can publish it without Apple having to run a review process,

00:30:58because internal testers is effectively yourself. I have had the Apple account since

00:31:03developer accounts have existed, at some point you could buy yourself an account for 99 or 109,

00:31:07something like that, whether you needed it or not.

00:31:12That is another question, in the past it was the way to get hold of betas and so on, but

00:31:17in this case Xcode was set up, Xcode configured, published to TestFlight, and I had it

00:31:23on my phone.

00:31:24And in between there was, yes okay, sometimes I had to go to that machine to enter passwords,

00:31:31because, idiot that I am, I had not given them to it beforehand.

00:31:34I find it interesting that it did not find a way, or did not try a way,

00:31:39to get hold of them.

00:31:41Well, on the machine, the Mac mini, it does not run with my Apple ID either, so it really is

00:31:48very, let me say, rudimentarily equipped, from that side it would have had to leave the Mac mini

00:31:53in order to cobble something together somewhere. But in terms of the day, where nothing else happens,

00:31:58rather the voice interaction enabled me to do more, for example to push this project

00:32:06far enough that it now runs on my phone and that from now on I discuss with

00:32:09it how we get a bit of performance into it and that I would

00:32:13like background refresh, along the lines of take 40 screenshots

00:32:17or 400 and work through them while the app is in the background, those kinds of

00:32:21details. By now I have also told it, better use iOS 27, because

00:32:2626 and 27 had a slight change regarding the model

00:32:32that Apple offers us, working differently with the LLM as Apple Intelligence, and that

00:32:39was a bit, I do not know, well, in an open-plan office you will probably

00:32:45not have a thousand people talking to a thousand AI systems, and yes,

00:32:50my family is long-suffering about us trying things out, when I walk

00:32:54into the kitchen and call out to the computer, let me get up to speed on a topic,

00:32:59and then it sets off and searches and tells me something, which it also does with my son when he asks it something.

00:33:04He had a question like that, whether it would help him. I think with Fortnite it is another matter.

00:33:09Yes, but you know, I actually find that frightening on the one hand and surprising on the other,

00:33:16but at the same time also evidence that the way we work is changing.

00:33:22Right, now you asked me, how do you handle all this efficiency, what did it cost?

00:33:26I said, weekend, for those who do not know, that normally has two days, and I burned through several

00:33:31weekly limits. Is that efficient given the weekly limits? No, certainly not. Is

00:33:38it efficient that all my projects are now in a state where I say, damn

00:33:45it, not bad at all, ladies and gentlemen, that actually looks really good. I would say

00:33:51that was definitely worth it. Would I see it the same way in a professional

00:33:56work context, then I would perhaps also say,

00:34:00folks, let us approach this in a structured way with skills and the like, but for

00:34:04this experiencing and learning, and we are investing now, who knows, I push

00:34:09through a night shift. I remember when I started

00:34:12working, I was at a consultancy, and it was not at all

00:34:16unusual that you were slaving away on some documents until six in the

00:34:18morning so that the courier could take them to the

00:34:21client. And you slept two hours on the couch, went home

00:34:25afterwards. Shattered, but proud to have got the thing done. Do not

00:34:29worry. I do not expect all of you to be hanging on the AI until six in the morning. But this

00:34:35experience of, let me call it power, and the experience of your thoughts becoming

00:34:43reality, that really is impressive. Right, I often get told that I digress.

00:34:50Yes, I would have two follow-up questions on that. Namely, you just said it, well, for

00:34:57the fact that my projects are now where they are. The question is, what did you invest and

00:35:01was it worth it to you? That is in the end the same question, when I say it comes

00:35:07down to cost per business outcome. What was the outcome? So what is the value

00:35:13created there for you, and was it worth it to you? And the second question

00:35:17follows on from that. That was actually the point of my question about the harness and what

00:35:22conditions you gave it. I claim that it is not only when we are in a professional

00:35:27setting that we want to give it conditions. And these conditions

00:35:34I very often experience, and again, whether professionally or privately, that we

00:35:41get bogged down in the fine detail of conditions,

00:35:45arguing about some criteria that we still want to apply,

00:35:51where I always ask myself, and here I would like to quote our esteemed colleague

00:35:54Rodewig from Security Illusion once more.

00:35:58Exactly. Is that really a requirement?

00:36:01Does it make a difference in the end?

00:36:05Well, take your example.

00:36:06What questions were asked?

00:36:08Yes, in your example.

00:36:09It could also have said, I do not have Mark's passwords, the probability

00:36:16that he chose a weak password compared with humanity as a whole

00:36:22is relatively high, the probability that I crack it with a brute-force attack.

00:36:27Let me give that a try, but that would not have been efficient.

00:36:34No, but joking aside.

00:36:35So in hindsight, would you say you would take a blank Codex again, or would you say there are a few things that made

00:36:44a difference for me. Well, at this point I would like to, so I introduced this with the words, we talk about a great

00:36:52many stories that you hear generally, and you have now asked me specifically, I would much rather have, and this

00:36:57is not an advert at all, used our own harness, the one we build in the company, because there I do know

00:37:02quite well which guardrails it comes with out of the box, and it can do computer

00:37:06use and so on, those sorts of things, but it is simply not available to me

00:37:09on my Mac at home. Hold on. But at the same time you do not know whether it would deliver the

00:37:14same efficiency effects with Astra. Yes, that is true. That would be another question

00:37:18you would have to tease out, to what extent the model, through what they also

00:37:23pass in with their own system prompt and so on, to what extent

00:37:27that overlaps, fair point. I mean, thank you for the explanation, I had understood that much about

00:37:32harnesses already. But you are losing the listeners. Yes, we can do that in

00:37:38a separate episode. I shall invite you again, then we can talk about agent harnesses.

00:37:41Whether I would do it that way again and what I took away from it in terms of what it was worth,

00:37:47it was in fact, in hindsight, worth it on a great many different counts.

00:37:51Firstly, it confirmed to me once more that agents, when they work for us,

00:38:00that the interaction with keyboard and screen is, A, very nerdy and, B, that through the option,

00:38:09that through the spoken word, that is, through a not really precise description

00:38:14of circumstances, a programming language says if, then, else.

00:38:18When I express myself, every one of us expresses themselves differently.

00:38:22We are correspondingly binding or non-binding, and depending on how closely you look, we differ.

00:38:27Even so, the machine produces a pretty good result.

00:38:31Why do I say pretty good? Well, I thought the result it produced for me was impressive.

00:38:35I found that a real insight, particularly with a view to the fact

00:38:38that models are not going to get worse than they are today, an enormous enrichment,

00:38:42because this question, how do we interact with the machines?

00:38:46Perhaps briefly back to the plot, and I also have a reMarkable and plenty of other

00:38:50things. So how might we manage it so that the efficiency which the machine

00:38:54brings us purely technologically, in terms of possibilities, so that we as humans

00:38:59once again have the chance, with, let me say, pen, device and meeting folder, to clear our heads

00:39:06enough that we can work on a topic in peace, while the AI handles

00:39:11the handwritten notes, the voice notes, the live voice interaction, the interpretations.

00:39:19On my desk, for instance, not at the company but at home, I have a

00:39:25little Raspberry Pi camera, and whenever I put down a piece of paper, the one it

00:39:29can see, it scans it automatically and files it for me.

00:39:32It is a kind of incoming-post scanner, it does nothing else, it only does that.

00:39:35Do I want cameras to be everywhere now?

00:39:37No.

00:39:38That is not the point. What I am getting at is that before AI I could not have built such a thing myself,

00:39:43could not have made such a thing, something where I think, that is a good idea. Let us just do it. And it is precisely

00:39:48this, you discover new forms of interaction, or old forms freshly recognised, or whatever

00:39:54it was, and you have your projects, which may have been buried, in a state where you say,

00:40:00I would never have got to that state. I have, for example, a book, that is, a project with

00:40:05my mother, greetings at this point as well, about her mother, never mind why,

00:40:10about whom she wants to write a book, because she lived to be over 100 and

00:40:14experienced a great deal of German history, from the Kaiser through the Third Reich and afterwards, and simply

00:40:20how life was and the notes she kept, and what such an

00:40:24AI alone contributes in terms of historical research, gathering facts, tracing

00:40:31how historical threads developed, which in the past was something genealogical research

00:40:36and so on might have covered. You could say, okay, AI is more

00:40:41than a simple search engine, that is right, but this thing then goes ahead

00:40:44and says, well, fine, okay, I have the following facts from you, and here fits the history

00:40:48of the village and whatever else and all the rest of it. Then we add

00:40:52a bit of framing and we have a bit of a story. Yes, that alone

00:40:55enables people, whatever their age, whatever their profession, to engage

00:41:00with things they could not engage with before.

00:41:03We already said that on the AI podcast a year ago.

00:41:05And for me what comes on top now is, okay, a new model comes out,

00:41:08let us see where it goes.

00:41:09And you feel like the small child under a Christmas tree,

00:41:12with their eyes shining, unwrapping the presents and absolutely delighted

00:41:17with the wooden railway, knowing they can play with it for the next three weeks.

00:41:21And I say, I now have this voice interaction,

00:41:24I have experienced how the system, interacting through speech,

00:41:27delivers this result, that I have this result, that I have come to know this form of interaction, and my mind has been racing ever since about the question of how you get far more of that into your everyday life, what our harness at the company might learn from it, what I learn from it for my private equipment as far as interaction goes, because what you also said, a family is there, person recognition, which voice does it listen to, can you optimise that somehow, because the phrase be quiet and not now, next song, right?

00:41:57So when the daughter comes into the room and says, hey, play some music, and the thing says, eh, which series.

00:42:03This is the clever Kain, not the one that searches the internet and finds nothing.

00:42:08We are also planning that, iOS 27, right?

00:42:13There you are, there you go.

00:42:14Yes, I am in.

00:42:17Right, and from that side... That takes a bit longer.

00:42:19Yes, exactly.

00:42:20Repeat your request.

00:42:23And from that side I learned an enormous amount there.

00:42:27But of course that is also the opportunity of, you have a large subscription, which in general

00:42:32costs money already, you had the chance of resetting.

00:42:34If none of that had been in place, I would not have been able to reach that insight.

00:42:39But even if you do not have that, well, I had, I did not have a Claude

00:42:45subscription for a long time, instead I used the things via my AWS account and paid

00:42:50per token there.

00:42:51And I had the thing prepare my tax return.

00:42:57Which means I went through all the documents and said, okay, what do I want

00:43:01to achieve?

00:43:02Here are the forms, this has to be filled in, this is my tax return from

00:43:07last year, this is what the tax adviser filed.

00:43:09You have here, close to 800 receipts, pre-sort that, classify it, explain

00:43:17to me why and where you would file each item.

00:43:21And we go through the receipts in descending order, starting with the lowest probability,

00:43:28we go through the receipts. Your goal is to have filed at least 80 percent of the receipts

00:43:33and for us to be able to say the tax return is formally correct. With that I set it entry criteria,

00:43:40and it whirred away for a good three and a half, four hours, and those were largely

00:43:47photos, exactly as you say, receipts, happily photos, just dumped into the directory and done.

00:43:52And at the end, including follow-up questions and so on, that cost me 60 euros. Was it

00:44:00worth it to me? Absolutely, because that was my lifetime, which I would otherwise have had to spend

00:44:08somewhere else. Apart from that, have you ever had so much fun with a tax return, watching

00:44:12how it does it? Yes, above all I looked at the result and was astonished,

00:44:17because it also explained to me where my tax adviser had overlooked things last year

00:44:23that he could have claimed differently, and interestingly, exactly, interestingly,

00:44:28where the tax office had pointed that out to him in writing. That was very interesting, but never mind.

00:44:36Perhaps just very briefly, we just made the point

00:44:39that the AI should not afterwards say, oh, Mark, with what probability does he have

00:44:44weak passwords. About Alex it should be said that he does not live in Berlin. The AI was not

00:44:49to blame with the tax return for the rather strange things that have just happened in order

00:44:53to file the tax return. Just so that we properly pick up and

00:44:57accompany our listeners here. So that they do not think they are witnessing a digital

00:45:01crime. Very good. Yes, but those are of course experiences.

00:45:05Yes, but what I am getting at is in the end the topic, or

00:45:10in the end it is the topic of acceptance criteria, that we say, what is it worth to us, how do we

00:45:16determine that it is good enough? And you told me months ago, that was also an entry

00:45:24criterion, or that was an entry criterion for building your harness, where you said, well,

00:45:29I want to be able to specify how much it may cost at most and what the criteria are

00:45:33by which you can tell when it is done. And you already had an episode about that,

00:45:39as far as I know, exactly. Yes, because you have prompts and skills and then

00:45:44loop engineering, and loop engineering is both a curse and a blessing,

00:45:48because you do not only have to specify what goal I have, you also have to

00:45:51give me the chance, under controlled conditions, to accept failure

00:45:55as an outcome, namely also to say, I have either

00:46:00reached it at the desired quality, or I have reached a sufficient

00:46:04number or the sufficient characteristic, so that a stop is assured, so that it does not

00:46:12try to round something to the ninety-ninth decimal place and then merrily fires tokens

00:46:18into the ether all night, and the next morning you see,

00:46:22look, it is still running, how funny, that must be a really great result,

00:46:25and then you look at the code and think, okay, why did it create

00:46:3140,000 branches for a simple addition task. It is nicely documented, but always wrong.

00:46:37Right, and that is why I say, that is perhaps where we do get a bit deeper into the topic of

00:46:44technology.

00:46:46My plea is, when we talk about quality optimisation at the small scale, in quotation marks,

00:46:52then we particularly have to look at loop engineering and ask, what are the

00:46:58acceptance criteria for our loops, and there I read, and I had also written this into the

00:47:04script for our episode, this wonderful metric,

00:47:07verification effort per delivered change. So you say, exactly as you

00:47:13just described, I look at the commits and say, no, you will have to do that

00:47:18again, we need to talk about the acceptance criteria once more,

00:47:21they clearly were not sharp enough. And that is a metric which, when it goes lower,

00:47:29when it falls, means your acceptance criteria are better, and from my

00:47:36point of view that is a criterion we can use for efficiency gains. Where we say, the

00:47:42time we as humans have to put in is simply no longer as high, or rather you

00:47:46no longer have to check as often, the better your acceptance criteria become. The problem

00:47:52with that is, I thought I had found a certain knack with Opus 4.8,

00:48:01with Opus 5 that is already, that hurts, with Fable I find it wonderful that I can talk

00:48:09to Fable about the acceptance criteria, that works relatively well, but Opus 5 is

00:48:13Opus 5, and the two of us will never be friends again. I think at the

00:48:18speed of the frontier models you do not need to get used to Opus 5 for

00:48:21very long. There will surely be the next one soon, but what

00:48:25while you were speaking, along those lines, I wanted to offer

00:48:29one more small personal tip to our listeners,

00:48:33if you notice that the AI has gone off the rails. First of all, that is

00:48:38a learning, and if you want to fix that learning, I would not

00:48:42advise anyone to work on the result, but on the path leading to it. So this issue of,

00:48:48oh, the thing generated process documents for me, for example, or generated a document from a process,

00:48:55and the document is wrong or has a mistake or something is missing or it is a bit too

00:49:01detailed or whatever. Then I would not recommend anyone to say, I will open

00:49:05it up and quickly change it by hand and save it. I would rather make the case,

00:49:10even if it costs tokens again, think about the point at which it

00:49:14probably took the wrong turn, at which point we were too imprecise,

00:49:17too precise, and that brings us to Opus 4.8 and 5 again. I mean, for one it might be

00:49:22a paraphrase, for the other that is one paraphrase too many, along the lines

00:49:25of, what is it going on about, I knew that already. And that out of

00:49:30this premise you try to repeat it for as long as it takes, even if it takes two

00:49:35or three interactions, until the system no longer makes that mistake

00:49:39as often, or no longer makes it at all, so that you get this learning effect both for yourself,

00:49:46how do I deal with the machine, and also, if it is a skill that I use

00:49:50or the department or the whole company uses, so that this mistake is weeded out for everyone

00:49:54and the thing is thereby taught sustainably to do things better.

00:50:00You said earlier, for example, that skills do not deliver a hundred percent perfection

00:50:06in the reliability of the result, you chose different words. I always say the difference

00:50:12between skills and prompts is that with skills you get a better quality result and more

00:50:16control over the result, where we now at all... And it being repeatable, that is 100 percent.

00:50:20Yes, because with skills you can also say, here is a bit of deterministic

00:50:25program code, you simply have to run that, because if I say via prompt,

00:50:29you have to pass all the tests, do not be surprised if out of ten tests

00:50:32eight remain and it says all eight passed, when there were ten,

00:50:35if all eight passed, whereas with the deterministic implementation

00:50:40you can at least make sure, is the answer tests-passed identical to the

00:50:44number of tests, otherwise take another look at what went wrong,

00:50:47but I would warmly recommend that to everyone again. If something does not work,

00:50:51think about how you can adjust it in the skill, and the whole topic of

00:50:56loop engineering is nothing other than explaining to a prompt

00:51:00how long it should attempt something, until what happens, in order to say it worked or it did not work.

00:51:08Because that is also a problem, just as at the start with our 80 percent.

00:51:12There are apostles standing around throwing terms about.

00:51:14Craft engineering, loop engineering, skills, prompt engineering, whatever.

00:51:20At the end of the day it is all basically the same thing.

00:51:25I would say iterative working and so on did not start with agents.

00:51:30But in the age of skills and the like, what changes is purely the expectation.

00:51:36What can I give the machine?

00:51:38How can I give it to the machine?

00:51:40How do I engage with it?

00:51:42A good prompt is also at home in a good skill,

00:51:45and loop engineering is a good skill that effectively knows

00:51:48how long one has to keep working, whether deterministic or non-deterministic.

00:51:53From that side one should not let these terms put one off.

00:51:56I am with you there.

00:51:59For me there is an essential difference between a prompt, even if you keep

00:52:06copying it back and forth and say, well, I can reuse it.

00:52:10The point is, a skill is effectively like a radiated intent.

00:52:16I say what I intend to do and thereby enable feedback and learning.

00:52:22And learning in particular. Which means, well, what I tried out in a loop, we

00:52:30will come to that in a moment, but if you say, I am executing this skill, so you hand the harness a skill

00:52:38and say, here, please carry out this task, apply the skill, and you are not satisfied with the result.

00:52:43Then at the end of the session, once you are satisfied with the result, once you have talked to it about

00:52:47what it should do differently, you can always say, right, now take the learnings from this,

00:52:53from this session, what do you have to change in the skill, add, leave out, so that you reach

00:52:59this result in considerably fewer steps. Which means these working instructions

00:53:06that you write into the skill, you can keep developing them continuously and iteratively.

00:53:11And this learning and unlearning is a huge thing, where AI, through skills,

00:53:19has clearly, well, in the end visible advantages over a human being.

00:53:26Humans find it considerably harder to make their knowledge as explicit as you can

00:53:32with an AI in a skill. And because we do not make it explicit, not everyone can

00:53:39say what is in their memory. And correspondingly also consciously say,

00:53:45I will take that out of my, no, I will take that out of my memory, I will no longer think that

00:53:50from now on. What I was just about to say, I once tried to

00:53:57build that into a piece of loop engineering, along the lines of, and if

00:54:02you get stuck, then learn from it and optimise your skill. That was a daft

00:54:08idea. Because what came out was something similar to your 20 million a day later, and all of a sudden it

00:54:16decided to build something completely different, but that solves your problem too, roughly. No.

00:54:22I think with this skill learning, I have, you mentioned the skill workshop earlier,

00:54:28Greetings go out to my Git repo, I have another Git repo called SkillForge, where at the time I was

00:54:33of the view that if a skill does not reach a goal, then you can effectively say to the AI, listen,

00:54:38this did not work in such and such a way, or the skill has the following problems, keep reworking the

00:54:43skill until this problem is gone. Yes, that was always quite nice as far as it went, and in

00:54:49many cases it worked well. By now, and I have not yet entered this, I am of the view

00:54:55that there is a component many people forget, namely skills are instructions, do

00:55:02this, do that, then do the other, mind the following, carry on until. But depending on

00:55:08where you deploy a skill, it may well need contextual knowledge. So let me take

00:55:13the topic of requirements engineering again. Where is my backlog, actually? Depending on the case

00:55:19that might happen in different systems. Possibly, yes, in a different language, with different

00:55:24terminology, with a different domain. So there are things that you do not,

00:55:28yes, okay, this is how a good story is shaped, but where you simply want to give it additional

00:55:35background information, and the idea at the moment is to give it something like a learnings MD,

00:55:40in which it does not write into a work instruction, if that is there, you write

00:55:47that there and you query the following, but which states from the outset, listen,

00:55:54if you do not have any people, ask for them, otherwise take these as the default.

00:55:57If you have this, that or the other, you will find such and such things in such and such a place.

00:56:01Where effectively, through enrichment with knowledge, the system does not only know.

00:56:07An example I had recently was someone explaining to

00:56:10the system how to make a phone call.

00:56:12The previous skill explained, pick up the receiver,

00:56:14take the dial, press the numbers.

00:56:16If you know what a dial is, congratulations.

00:56:18I am old, please google it.

00:56:20Take the dial, enter the number

00:56:22and then it goes beep beep beep, someone picks up on the other end, and with the contextual knowledge it knows that it should answer politely and in a friendly manner, that it should use the formal address, that it had no idea and that it is Mr Müller being called, and that if it is calling some adults-only provider it should speak differently, to cut a long story short.

00:56:40Pass on knowledge, pass on context, and not just plain instructions. With that I somehow have the feeling I can increase the efficiency of skills enormously once again.

00:56:51Full stop. You are looking sceptical.

00:56:54Yes, no, with the dial, mine always went with...

00:57:05But fine, that is another topic.

00:57:07At this point I shall have to play in a genuine audio file.

00:57:12On the matter of the dial I agree with you.

00:57:15I see, well, if we pull this back to the topic of efficiency and metrics, what do we measure it against, then I would like to set down three rules that were my learnings.

00:57:28The first you have essentially already said, and I also think it is common sense.

00:57:34If you want to reach a goal, then you rarely define

00:57:38only a single criterion for that goal, instead you always give it some kind of context and say, the factor out of the whole thing

00:57:46is your goal. There is not one single metric that will make you happy as a company, because

00:57:54that always leads to people trying to game that metric and trying

00:57:59to work around it somehow, like manipulating the skills or manipulating

00:58:07the working environment. But if you manage to tell it, listen, these are the conditions

00:58:12and these are the criteria by which you do it, and you may not change the

00:58:16conditions. This overall factorisation, that is rule one for me, never a single number,

00:58:25but always a factorisation, and saying, this is the overall picture of the goal. Then, rule two,

00:58:32I defined, send counter-metrics along with it. Meaning, they travel alongside. It is not

00:58:39about saying we want to reach a number, which is, we want to make so much more profit with the

00:58:45same capital investment, instead you always have to look at further

00:58:51metrics as well. Greetings go out to DORA, META, things like change fail rate,

00:58:59lead time, work in progress, developer experience, Goodhart's law, there are countless

00:59:07things that can be relevant and genuinely proven metrics, and for your particular

00:59:12goal, look at which counter-metrics are relevant for you, which should improve

00:59:20or at least not get worse, and the third is the topic of self-reported speed.

00:59:27When I ask developers how much faster they have become through GitHub Copilot.

00:59:35Well, it is at least 20 percent, right?

00:59:39But we now have several examples where companies have actually measured it

00:59:45and where developers said, yes, we have become 25 percent faster,

00:59:49and de facto it turned out they are actually up to 20 percent slower, because they

00:59:55spent far more time on pull request reviews, on follow-up corrections and so on.

01:00:01So honestly, establish a baseline, work out together with the people how it is

01:00:09calibrated, and do not do it by gut feeling and self-reported speed.

01:00:14But I would like to raise a small objection at this point.

01:00:19We measure the topic, so what you just said, how much faster does it make you?

01:00:25That pays into what I said earlier, a year ago I would never have thought I could manage all that.

01:00:30To be fair, one can question whether bringing my old eight projects back to light is worth anything or not.

01:00:37That is my personal game.

01:00:39Game, but if I now look at a company and people in software

01:00:44development are confronted with AI, then it is not like this. Here you go, folks, here is

01:00:51the AI, have fun, and there is your efficiency. A great deal changes in how people work together. If

01:00:58you have someone who now, I do not want to judge it as good or bad, but you have just

01:01:02brought in the example of loops and skills and so on, and let us say we now

01:01:06adopt what I did. Then someone starts, perhaps takes a bit of time for

01:01:11the question of what I intend, intent, and then the thing runs, and we do not shut the lid, for three days

01:01:17straight and the result is pretty good. Then in those three days they have pushed so much stuff over

01:01:22the wire that everyone else on the team first stands there and says, oh damn,

01:01:26here comes a pull request, it looks more like the flooding of an entire region. We

01:01:31are overwhelmed, because you run into other people who work in a quite

01:01:36fixed way, and the topic of the weakest link being the strength of the chain is at

01:01:40that point, damn it. Imagine requirements engineering delivering like

01:01:44crazy, then it is the developers who cannot work through it, or the

01:01:47developers, because they opened Pandora's box first and

01:01:50get to savour the sweet taste of the honey most intensely right now,

01:01:53notice, oh damn, I can use this, but you get

01:01:58Quite different questions become important for them.

01:02:00The other day, for instance, we learned that in your coding guidelines you used to have

01:02:04the topic of commenting source code.

01:02:07And commenting source code, when you work with agents, is a pain in the neck, to

01:02:13put it that way.

01:02:14Because what does the agent do?

01:02:15The agent says, oh look, the comment says what this routine

01:02:19contains.

01:02:20I do not have to read the routine at all, it is in the comment.

01:02:23So we first drove that out of the agent, it should no longer comment, it should

01:02:27no longer comment the technology, but comment the domain logic. So that it writes

01:02:31the domain logic down somewhere, without drawing conclusions about the code.

01:02:36And we considered doing things like, in Cantas for instance, regularly

01:02:40telling it to carry out a renaming of the functions and variables in our

01:02:45stuff. Why? Well, if it generates boilerplate code somewhere that it never calls

01:02:49again, it will notice at the latest during the renaming that there are no

01:02:52counterparts left at all. You would never have written that into such a rule in the past.

01:02:56Today, let us say you have, who knows, a development team of, say, ten people, each of them now spends not three days but a day at a time hammering code into a repo, and the poor thing goes, or whatever you use as source code management, and the people working in it are suddenly flooded with code.

01:03:14Now this code, even if there is boilerplate and whatever else in it, may well serve its purpose.

01:03:20It builds a solution that can, and now the testers come round the corner and say, damn it, whoa, whoa, why is your backlog empty again?

01:03:27We have only just started a two-week sprint, we are on day three, why is the backlog empty?

01:03:32And suddenly people are put in a position of no longer talking about what needs doing, when do I have time for it,

01:03:40instead they have to talk about handling the volume, the challenge that arises from

01:03:47the sheer mass and the speed, dealing with that, each individual as well as the

01:03:52chain as a whole.

01:03:53And I think that pays enormously into the topic when such reports come in,

01:03:59along the lines of, people feel more efficient because they judge the matter themselves.

01:04:04But looking at the chain says, you are getting slower.

01:04:08Because suddenly there is far more time spent on discussions that previously had no time to be had, because perhaps people feel

01:04:15overwhelmed,

01:04:17wounded in their pride. How can it be that the AI can do this? That was always my hobby horse.

01:04:22You have a great many side theatres that can currently still rob you of this efficiency.

01:04:28Absolutely. It is not one single reason, but they can. No, no, but I am with you there. So that was in the end also what

01:04:35I was getting at earlier, if we grasp AI, in quotation marks, merely as a tool,

01:04:41then we focus on touch time. That is, wherever something is actually handled. And that

01:04:49is very, very dangerous, because if we are not aware of it, then it leads to

01:04:55you writing, or letting Claude Code change something in the code on your laptop, and

01:05:02in the worst case it is precisely not code in a programming language, but it is

01:05:09for instance skills or some instructions that have to go into the model

01:05:15again.

01:05:16Or it is skills or whatever, where you say, well, this is not about,

01:05:21I cannot do any schema validation here on whether the syntax is correct,

01:05:27instead I would have to carry out a substantive, semantic check on whether what

01:05:33Claude Code wrote there is correct. And then you pour that into a pull

01:05:38request, so you push it into the code repository, and then you do not have it reviewed

01:05:43by humans, but by another agent, and that one comments on it again and

01:05:49says, yes, here, you could do this differently. And then you perhaps also have

01:05:53your Claude Code looking at this pull request and saying, oh, the other one said

01:05:58I have to change something here. And I think I said before that with the release of,

01:06:03I think it was Opus 4.8, the thing became much, much more verbose, Opus 5 even more so,

01:06:11in terms of what it pushes into pull requests. Which means, if on the other side I have a

01:06:16Git and there is a GitHub Copilot doing the code reviews automatically, and Opus

01:06:22pushes far, far more code changes in than are actually necessary, then

01:06:28GitHub Copilot has far, far more to look through as well. Which means you burn far

01:06:33more tokens because the output from Opus is so high. You burn even more tokens because

01:06:39GitHub Copilot has to look at a great deal more. And that generates output again in turn.

01:06:44So it escalates, and we have the opposite of efficiency. Which means,

01:06:49we actually have to look at what we want to achieve.

01:06:52We want to optimise the flow, and that brings me back to end to end.

01:06:59In the end we want to optimise the cost-benefit ratio.

01:07:03And that means we do not need 15 reviews from GitHub Copilot,

01:07:07instead what we have to look at is, what is the smallest possible change

01:07:12I can make to a piece of software in order to reach the desired goal.

01:07:16And that is what we have to teach the AI, and then start the whole topic of lead time and wait time optimisation.

01:07:26And how people might complement one another in their job roles in future.

01:07:32And what someone might take on and what someone might leave aside.

01:07:35But perhaps, since you brought up two more points, I think that is also a very quick cost trap,

01:07:41when AI leads to events at other AIs and the same yardstick is not applied there,

01:07:47because if the thing says that was bad and the other thing says, okay, I will rework it,

01:07:50we very quickly arrive at, oh, 24 hours later.

01:07:54We are still stuck on the same routine. What is going on there?

01:07:56Other than it being expensive.

01:07:58What I would perhaps also like to offer at this point on the subject of best practice, in my view,

01:08:04is one thing, namely that teams should be working in the same harness,

01:08:09so that you can use the same efficiencies and the topic does not play itself off against you.

01:08:15So take Codex and Claude Code, with Claude Code this Let's Simplify and Code Review

01:08:21and whatever all that stuff is called, I do think it is noticeable

01:08:25that if you have mixed operation of models and harnesses in a team,

01:08:31they sometimes get in each other's way a bit as to where they place their optimisation priorities.

01:08:36The second is, who would have guessed? Skills. At the moment I am quite of the view that you should have skills for things like architecture patterns, what is good code, so that a human perhaps always has a chance to look at it, what matters to you, what matters in information security,

01:08:53what matters in data protection and so on, pouring that into a skill so that you effectively have it

01:08:56running along during development, having such libraries, I mean, we have something, but on the

01:09:04market there are things like Ponytrail and ADHS skill and whatever else, that effectively get the system to

01:09:11be the laziest developer in the world. Along the lines of, only write the lines you

01:09:16really need, and not three more. Claude Code has now also implemented a switch in the output

01:09:21settings so that it handles its interaction with you more

01:09:26economically in terms of tokens. There too it is always worth keeping an eye on things, and at the latest there you notice

01:09:32that when you are working in the same harness, you can also help each other with

01:09:38a learning, have you seen this yet, because regardless of how large or

01:09:43small companies are, personal experience, or the I saw that on TikTok,

01:09:48we really should take a look at that. It is not always the weakest option. Some people get something out of it.

01:09:52Or through podcasts like this one, whether as a listener or as a guest or as a host.

01:09:57And at that point there is a lot you can optimise about these things.

01:10:03And if you then hand it up to the machine and the machine does the assessment,

01:10:09then I also think it is a very good quality indicator if you ask the machine

01:10:16not only to say, you have to change this. Here is a finding, but also, how certain

01:10:21are you that this is a finding? So is that a probability of 20 percent or

01:10:26are you 20 percent certain or 100 percent certain? And what effort do you expect for it?

01:10:33Because frankly, you may have noticed this too, when you ask it how long

01:10:36it would take for that? Then it says three weeks. It calculates that in human time

01:10:40and not in machine time. And I think that is another good

01:10:44lever, I would not necessarily say I would bet my house on it, quite apart

01:10:47from the fact that I rent. If you ask it, how much effort do you see in that,

01:10:52how much effort then actually occurs, then you might always be able to derive

01:10:56a small correlation from it, how efficient was the work with it. And if

01:11:00you tell it beforehand, you are assessing something, so give me the expected effort

01:11:04as well, and please give me your confidence, how certain are you, then

01:11:08you have further checkpoints that you can use in your skill again, along

01:11:11the lines of, well, if it says that is three weeks of effort, again, the AI does

01:11:16it faster, but an effort that takes three weeks still takes the AI, let me

01:11:20say, 30 minutes, and for an effort of one week it takes 10 minutes, from that you

01:11:23can already derive something, and if the system is additionally only 20 percent

01:11:27certain, then those are all further factors for you perhaps to say,

01:11:32I will not trigger a review there at all, or I will have these cases checked by humans

01:11:36beforehand, then you have quite different quality gates, compared with simply blindly

01:11:40saying, the AI assesses, the AI executes. Both sides may have knowledge of

01:11:47these other factors I have just named, but if you do not tell it, use

01:11:51these as assessment criteria, report them, make the decision depend on them,

01:11:57even the sentence, only implement it if it is economical. That could well

01:12:03draw conclusions, you could at least try it out and see to what extent it helps you in

01:12:08your processes. My plea ends with this sentence. Do not simply ask the AI whether something has to be

01:12:14adjusted, because it will say every time, good idea, thank you for pointing that out to me.

01:12:18That is fascinating. Nobody before you had that idea, you were the best human handler

01:12:23of AI ever. May I, do I have to work with all those other humans? I only want to work

01:12:27with you, you are the best. It just tells you what you want to hear. So it talks to me.

01:12:31Exactly, what I see is that many people are now coming up with the idea, well,

01:12:38generation scales, human quality assurance does not, so I will see to it that I replace human

01:12:45quality assurance with AI as well. From my point of view that is not a good idea.

01:12:50Replacing human quality assurance, as far as possible, with deterministic quality assurance,

01:12:57meaning automation, is a good idea. But putting AI in there, which if in doubt,

01:13:04I do not even know how many AI exploits there were again this week, but you can really talk

01:13:11almost any model into doing things that you previously told it very clearly

01:13:17not to do, please leave that, do not do that, because they are simply only soft boundaries, and that is why

01:13:23I found this study while preparing for the podcast, where they researched engineering

01:13:32change in aircraft manufacturing and then established that, well, actually

01:13:36only 8.5 percent of the entire chain is value-adding time.

01:13:41Which in the end means you have roughly two weeks of waiting time per one day of work, and

01:13:48that shows that lead time is in the end a queuing property and that we have to think

01:13:53about what the elements of lead time are, and precisely the waiting time, things like queues,

01:13:59hand-offs, approvals, work-in-progress limits, that in the end the answer to becoming more

01:14:08efficient through AI is a flow and governance programme, even if people do not want to hear

01:14:15governance, but in the end it is about saying, what should the flow look like

01:14:20and what are the rules we stick to while doing it?

01:14:23It is not about a tooling rollout.

01:14:25It is about deciding what game we are playing.

01:14:29And if that is football,

01:14:31then we do not play with our hands.

01:14:33And then there are the white lines and they mark out of play.

01:14:36And we can discuss once more whether we play with or without offside.

01:14:39But fundamentally a goal is a goal.

01:14:42Otherwise we are not playing the same game.

01:14:45And for me that is governance.

01:14:47And once we have clarified these rules of play at the start,

01:14:50it makes the game much faster as well.

01:14:52Because you no longer have to discuss what a touchline actually is and

01:14:56why I am not allowed to carry the ball into the goal.

01:14:58And above all, once it is there, you also have to distinguish between the moment

01:15:04where a person perceives a change as, huh, what is going on here?

01:15:08Suddenly someone wants to tell me how to work with this, and the point where it shifts

01:15:12to the topic becoming second nature.

01:15:15I think that does not take long at all.

01:15:17that we should ignore people and so on. That was not my intention at all. That was simply more the point.

01:15:23Let us also not unleash every bit of AI slop straight onto a human reviewer, because there are sometimes

01:15:30things where you say, okay, let us go through it beforehand. I had a case recently, I made a screen recording of our

01:15:36application to document for others what you can do with it and how you use it.

01:15:41Then I had an idea and effectively gave it its own video.

01:15:45Two insights came out of that which I found fascinating, first of all it went, oh!

01:15:51I can now assign a few things better in terms of the domain logic, because I was also speaking on the video about what I was showing.

01:15:58And the second was, listen, you probably did not notice, but at this point the cursor was positioned wrongly,

01:16:03at that point a spacing was wrong and so on, and then it immediately said, shall I write a few issues about this

01:16:08And you sit there and think, okay, thank you.

01:16:13Yes, although at this point you can also ask yourself again,

01:16:17why do you actually want to burn tokens now

01:16:20operating the Jira API in order to create issues?

01:16:23But fine, another topic.

01:16:25I would say...

01:16:27No, no, no, no, no, I am not letting that one pass.

01:16:30So you will not catch me out that quickly with the line of,

01:16:34well, you have now burned tokens to operate Jira.

01:16:37The second is, an important feature of our harness is reporting the associated costs, not in the form of what did the light in the broadcast cost, but in the form of the euro value of tokens.

01:16:51And when I look at how much money, or how little money, I do not want to frame it, because if I now say how much money, then everyone will be left with, he talked about a lot of money.

01:17:02What does it cost me in euros if I have a backlog item maintained,

01:17:07have an issue in Git maintained or whatever else?

01:17:12For that money I would not have clicked it myself.

01:17:15That is, opening it up, clicking, writing, phrasing.

01:17:19If you have clarified the rules beforehand, where it then perhaps also answers with questions, talks to you,

01:17:24we are back at rules.

01:17:26I never made a plea against it, remember.

01:17:29also said something like, with skills you can also standardise things where you may previously

01:17:33have spoken into the room, where there were plenty of ears listening

01:17:38along, who on leaving the room at the latest thought, what was that old fat white

01:17:41man going on about?

01:17:42We will do it the way we have always done it.

01:17:44But that is nothing new with AI either, that would, that would have

01:17:51always, and here I look back at the topic of extreme programming, those people

01:17:56already made the case for conventions and agreeing on how we want to

01:18:03work.

01:18:04But fine, I would like to summarise it like this, and you keep looking at the

01:18:09clock and then talking about the...

01:18:11Hold on, stop, stop, stop, stop.

01:18:12You are very welcome to summarise, but we are, and bear in mind, we still have to resolve

01:18:17the Plaud story very briefly, because at the end that means.

01:18:20I would perhaps like to do that and afterwards leave the summary

01:18:24to you, because, well, credit where credit is due. Do you want to, or shall I say why

01:18:30I wanted to hear that from you? Listen, I started earlier telling the story

01:18:37of what is all possible with smart speakers and voice input, and the thing

01:18:42about voice input, some of you have surely heard the episode,

01:18:46when I was on holiday with Jens, about my dream interaction device, the Plaud, the voice dictation gadget,

01:18:55I have no idea, this is not getting any more professional at this late hour, I am enjoying it, and you

01:18:59got yourself one as well and apparently have things to say about it. So, is this praise or criticism

01:19:06or what is it? Well, when you were out and about with the Plaud, I already had

01:19:12the voice memos, and then realised, damn it, the voice memos are on my

01:19:16iPhone, so unless I do something about it they get synchronised to iCloud, and

01:19:22we covered that in the last podcast, where I said, iCloud is something for which I as a private individual

01:19:29accepted the terms of use, which means nothing belonging to my employer

01:19:33has any business being in there.

01:19:34Ergo I have to come up with something that works differently.

01:19:39Which means I would need a dictation device that is entirely local to the device, where I can then effectively

01:19:45pull the material down from my laptop, for example.

01:19:49And in theory that should have worked with the Plaud, and I then, I think it was

01:19:58an Opus, yes, Claude Code on my laptop with an Opus connected through Bedrock,

01:20:02said, listen, here is the Plaud SDK, read through all this stuff,

01:20:08here is my device, how do we do it?

01:20:12And the thing whirred and whirred and tied itself in knots with, okay, now

01:20:16I will build a demo app, and then on the demo app I have to somehow read out the Bluetooth communication

01:20:24between your iPhone and the device, so the short version is, it tied itself in endless

01:20:31knots trying to work out how to get these audio files onto my laptop with as little

01:20:38loss as possible, and what you told me was that

01:20:44your Claude had no trouble with it whatsoever.

01:20:47You summarised that nicely, and I would say we put it together and move on.

01:20:51Sorry. So.

01:20:53No, I am asking myself why. What did you do differently from me?

01:20:56I do not know, perhaps we need to share our projects some time.

01:21:00The beginning really was like this.

01:21:02You get Claude and then it runs against their own backends, and

01:21:07then you have their MCP server, which also plays up sometimes, because it somehow

01:21:12takes the view that there are no new recordings in the last week, and you think, eh, I

01:21:15talked that thing full, what is going on there?

01:21:17That is one thing, the second is, the Plaud SDK, there I could get a token

01:21:24and an API key from them, which effectively allows me both to consume cloud services from them,

01:21:32things like transcription, for example, that is, converting an audio file into text. It also allows

01:21:38me, I had to fetch it from them once, that key, so that

01:21:44with the Plaud thingamajig, to use professional terminology at this point, another tick...

01:21:50With the Note Pro it is, yes, since I have no advertising, I cannot talk about a sponsor.

01:21:54So at this point I would happily get in touch, but beforehand my son told me

01:21:59I should contact Holy, because good podcasts have Holy adverts and he would like more Holies,

01:22:04so from that side, if anyone is listening, we will take that too. No, but joking aside. And it

01:22:09then fetches the audio file, and I mostly end up in the discussion with people who then say,

01:22:14hang on, just for the audio file does that even make sense, because... yes. Audio file,

01:22:18you have already said that, why get yourself a device, why not take,

01:22:21like Alex, the voice memos, quite apart from where the data contract runs,

01:22:25the iCloud contract or whatever. For me the device really is the

01:22:31game changer at this point, because firstly I press a button. Fine, now you can say I can do that

01:22:36on the iPhone too, right? But I press a button and it records. Full stop. No, exactly. Your

01:22:42Plaud, yes. Tell me, it does not have to. A call comes in and that is the end of it. If I

01:22:46get a call, or if I want to fiddle about on my phone during something, or yes, so anything

01:22:51that touches audio input and output puts my voice memo recording at risk. That is

01:22:57one point in favour of a device for me, and I was in fact very happy. So we

01:23:02really should swap our projects some time. But that it said, listen,

01:23:05give me the key here and then I can effectively pair the device in my app, can

01:23:11download data, can delete audio files, and can then devote myself on my device to

01:23:19transcription. Do I have challenges with that? Yes, because I have a

01:23:25bit of a sense of honour, I would have liked to do everything with Apple Intelligence. At

01:23:30that point you run into the drawback that you do not have speaker

01:23:35recognition, although that is quite funny too. When people are talking

01:23:40with each other, then at some point it starts anyway and says, so Marco

01:23:44told Mr Schommau and Günther and Maya, I do find that

01:23:48quite funny. It is rather nice. Is it 100 percent stable? No, it is not yet, but it is

01:23:59in that sense also the result of an AI system. I pestered it with my wishes and

01:24:06it delivered me a result, whether there is still the odd quirk

01:24:11in there, possibly, but I can pull the data onto my notebook and then do effectively

01:24:19what I want there. So I would say we have, I am not quite at the goal yet, but we have

01:24:26both reached roughly the same result. Except that yours was clearly faster

01:24:31and from what I observed also more efficient. And my question would be, as a tip for

01:24:38the listeners, how did you set up your harness, what did you actually use for it and what

01:24:44did you tell it? Did you use a slash goal with target criteria, did you give it any

01:24:51skills, what did you do? So, the private code on your laptop...

01:24:56So, the private code on your laptop...

01:24:58My coding agent of choice is still Claude Code. Behind it, quite apart

01:25:05from the fact that it is hooked up to various things where it tries to retain the insights from historical

01:25:12treasures for future treasures, so a bit of memory, I have a few skills

01:25:17in there, such as this ADHD skill, which tries to keep it roughly on track,

01:25:24I have this Superpower plugin, which makes sure that it clarifies with me beforehand what

01:25:30I actually want from it before I set it on projects.

01:25:34Grilling or brainstorming?

01:25:36Okay, sorry, you have now actually completely thrown me off at this late

01:25:40hour.

01:25:41I use it first for brainstorming, along the lines of, have I now thought through everything

01:25:47I want to discuss with it, and after that, what I am getting at is, I get it to

01:25:52write itself a plan and then stick to that plan, and this is where

01:25:58ADHD is really cool again, because it additionally, it also, it additionally

01:26:03creates actual Python scripts, which then check during development by

01:26:08Claude Code that it has not meanwhile veered off to the right. And then at

01:26:13the end it is slash goal, implement the plan in whatever manner and report back when

01:26:18finished. For that I have a standard phrase, like the one I already tried to

01:26:22offer earlier, which in fact sits in a text file for me that I

01:26:26always copy out. You now have everything you need. You do not need to ask me any more.

01:26:29All the decisions you have to make you can make independently based on the information

01:26:33you have. There we have got some very good tips out of Mark

01:26:39at the end of the 90 minutes. I thought that was good for everyone

01:26:46who stayed with us. Thank you, Mark. Since we are already at this subtle

01:26:56form of appreciation. Would you like to tell us what has stayed with you from today's episode in terms of

01:27:02learnings, what you would like to summarise? Given the time, yes, the patience and the

01:27:08staying power. Oh yes, of course, the person stuck in traffic!

01:27:12Warm greetings to Mainz, to the person who is now stuck in a traffic jam and told me that he always listened to this on the way home on Mondays.

01:27:19Matthias, these greetings go out to you. Right, that had to be said. Now over to you for the summary, please.

01:27:25Yes, so the very first thing for me is the topic, I wanted to go through the things

01:27:33we talked about from the back. I personally tend towards over-thinking and perfectionism, and

01:27:40quite honestly, drop that, define what you want to achieve, talk to the AI about

01:27:47what you want to achieve. Exactly as Mark said, put that into the plan and think

01:27:53ideally while you are at it about how you really get it to the goal. That is, to where you want it,

01:28:01because the first learning is that if you use AI as a tool, then it shortens a

01:28:08work step, but the real efficiency gain lies in the waiting times between the

01:28:13work steps. And anyone who simply buys AI as a tool cannot achieve that efficiency

01:28:20gain. The second thing, and this again comes back to the example Mark just

01:28:25gave, is the right level of quality. That can only be decided against the requirement.

01:28:32What quality is really relevant for the customer in this case? What are the

01:28:38conditions that play a role here? Which means we can bring in the customer,

01:28:43we can bring in regulations, we can say there are service level objectives,

01:28:47that the company expects, or anyone else, but, excuse the expression, everything

01:28:52else is computer-scientist romanticism. Computer-scientist romanticism? Yes, there is another word

01:29:02for it that is not suitable for minors, but we shall leave that. I find computer-scientist romanticism more charming. That is how it is.

01:29:09And in the end, do not fall for any efficiency narratives.

01:29:16Spotify did this, Toyota did that, so-and-so optimised 95 percent of their

01:29:24tasks.

01:29:25It is your environment in which you want to do something, establish a baseline there, and depending

01:29:33on the baseline you say what the things are that you want to optimise, under which

01:29:37criteria.

01:29:38Efficiency claims without a baseline are narratives.

01:29:44While you were speaking I would like to offer something that came into my head,

01:29:48as my closing plea.

01:29:50I have already shared the other tip, yes, but my closing plea,

01:29:54for anyone working with AI, always with a view towards

01:29:59professional use, if you do not know that you could fail,

01:30:03you will not do it either.

01:30:06People who say, no, that is simply not possible.

01:30:10They may well not even try it in the first place.

01:30:13I had a case recently where I built something, and a colleague said, I simply do not

01:30:18understand it.

01:30:19Excuse me, Mark, you have no idea at all about this subject, but the result you have

01:30:25put in front of me is better than what I had worked out over three months.

01:30:29Better than what I expected to achieve.

01:30:32How did you do that?

01:30:34What is your tooling?

01:30:35I am reminded again of an earlier question, what do you work with?

01:30:38Then I said to him, I basically did not know what mattered.

01:30:42I had a problem, and I discussed it with the AI.

01:30:44We talked about it, we asked questions.

01:30:48Let me perhaps add another learning as well.

01:30:50I like to tell it, these and these are cool projects.

01:30:53Please look at them in terms of domain logic and technology, and also look at

01:30:56what you might be able to learn from them.

01:30:59Some really good things come out of that at times.

01:31:02And that is what I would pass on to people at this point, it will all work out.

01:31:07And I would say, before we drag this out unnecessarily.

01:31:11Alex, good to have had you here.

01:31:13I am absolutely delighted that you joined us on this podcast again.

01:31:17Everyone who listened, at some point we will sell something like air miles.

01:31:21Along the lines of, people who stayed with us the longest

01:31:24will have earned higher miles.

01:31:26Along the lines of, from then on there is a bonus.

01:31:29Good to have had you here.

01:31:31I would be glad if you came by again soon.

01:31:34Yes, and I say that to everyone as well, best wishes.

01:31:37I was just about to say, exactly.

01:31:39I do not only say that to you, I say it every time.

01:31:42If you enjoyed it, leave us a like,

01:31:45subscribe to the podcast, tell people

01:31:48that you are entertained here in a very engaging way.

01:31:51And with that I say, see you soon. Bye.

01:31:53Thank you.

01:31:56Welcome to Think Different. Think AI.

01:31:58the podcast by Mark and Jens.

01:32:01Two technology-loving minds who do not just talk about artificial intelligence,

01:32:06they live it.

01:32:07Here you get clear perspectives, real practical insights

01:32:11and a fresh look at what is possible.

01:32:14Understandable, critical and always with a wink.

01:32:18AI to think about, to smile about and above all to join in on.