One common problem is when LLMs drift off course. For example, you are in the middle of a complex coding problem, making great progress. Then suddenly unrelated code is added or deleted, or your model starts generating code for a different framework, one that you are not using.

Lately I’ve been thinking this is not really a problem with the model, it’s just poor steering. Models respond with whatever makes sense to them, given the general direction we have given. It makes pretty good guesses most of the time, so we think we are buddies, in sync, it knows what’s going on.

But that’s where the problem is, those guesses are also steering the model. If the model guesses wrong, it might be because we haven’t given it the information it needs, even if, for us, it might be obvious.

For example, if you are using FastHTML, and for a code change, the model guesses “this code looks like FastAPI code, so I’ll just generate that!”, that’s something that could be prevented and/or fixed.

  1. Make your assumptions explicit, and repeat them in every prompt. You can’t blame the model for guessing at things that, to you, might seem obvious. Like you are using FastHTML and not FastAPI. Or you are only working on code in a very limited area. Or you want it done like some other thing you’ve seen. This is similar to all the techniques for RAG, covered in langchain RAG from Scratch videos, but applied to prompts.
  2. Verify and retry. Some of that prompting is repeated in verification. Did we introduce any dependencies (e.g. fastapi)? Are changes limited to what we expect?

The models are great at reasoning, but its not reasonable to expect question-answer magic to guess everything in your mind.

Not that I’m doing any of this. It seems it should be measurable:

Prompt1, prompt2 model verification1, verification2 tally