Don’t Fight the Weights
November 14, 2025
For context and prompt engineers (and even chatbot users) it’s helpful to be able to recognize when you’re fighting the weights.
- Here’s some signs you might be fighting the weights:
- You find yourself threatening or pleading with the model.
- The model makes the same mistake, even as you change the instructions.
- The model acknowledges its mistake when pointed out, then repeats it.
- The model seems to ignore the few-shot examples you provide.
- The model gets 90% of the way there, but no further.
- You find yourself repeating instructions several times.
- You find yourself typing in ALL CAPS.
Source: Don’t Fight the Weights
If you’ve been working with large language models for a while in any sort of non-trivial way, then you’ll likely have run into this situation where you simply cannot get it to produce something that you want it to. A classic example was until relatively recent getting an output in JSON format. Time and again I run into the issue when I ask it to produce, say, HTML that it will add extraneous content as well even when I ask it to only produce HTML.
But this framing of the challenge really helps understand perhaps what’s going on and how to work around it.
Drew Breunig calls it fighting against the weights, and this makes a lot of sense.
