LLMs and line cooks

It keeps coming up that LLMs can’t be trusted.

Sometimes they solve a problem (typically a coding problem) cleanly, other times they spin and spin trying to get it right. You can ship something 100x faster, or ship nothing after spending hundreds of dollars on tokens.

This is the main challenge we’ve been tackling since the ground suddenly shifted under software engineering, at the beginning of this year. On one hand, the tools are so powerful when they work that they change the game. On the other hand, when they don’t work, it’s incredibly wasteful and frustrating. And – at first – it can be really hard to predict when they will work.

With time and effort, we’re starting to understand it. And it has convinced me that engineers aren’t going to be replaced by LLMs; instead, using the LLMs requires its own kind of engineering.

LLMs charged with technical tasks appear to have a lot in common with human technical experts in many fields. Specifically, they thrive when they are given exactly the information they need to complete their tasks, and nothing extraneous to that.

In Big Tech companies, there were (I think not so much anymore after the recent purges) whole groups of knowledge engineers who specialized in this: making it so software engineers would get exactly the information they needed to work, without having to sift through tons of peripheral documentation to get to the relevant items. It was so important for people that these companies invested a lot of money in it.

Based on my own experiences, I’ve found it helpful to remember that I’ve already learned this from working with line cooks. They are the core of the restaurant, and technical experts who are usually geniuses of one kind or another. And if I got wrapped up in the details of the larger operation and then started giving them too much context, started calling attention to what was happening in service, or whatnot, I wasn’t helping them, I was distracting them and making it hard to focus on the task at hand. But when they received the exact specifications of what their output should be, and nothing more to worry about, that helped them thrive.

So, in terms of “the way we work now”, it seems that all of our new processes need to be grounded in this effort. To make sure every task comes with exactly the information needed to complete it, narrowly scoped, and nothing more. This was already needed in key places, and as the LLMs extend into more and more operations, it becomes a core effort.