Common pitfalls when building generative AI applications

As we’re still in the early days of building applications with foundation models, it’s normal to make mistakes. This is a quick note with examples of some of the most common pitfalls that I’ve seen, both from public case studies and from my personal experience.Because these pitfalls are common, if you’ve worked on any AI product, you’ve probably seen them before.

…

In short, here are the common AI engineering pitfalls:

1. Use generative AI when you don’t need generative AI
   Gen AI isn’t a one-size-fits-all solution to all problems. Many problems don’t even need AI.
2. Confuse ‘bad product’ with ‘bad AI’
   For many AI product, AI is the easy part, product is the hard part.
3. Start too complex
   While fancy new frameworks and finetuning can be useful for many projects, they shouldn’t be your first course of action.
4. Over-index on early success
   Initial success can be misleading. Going from demo-ready to production-ready can take much longer than getting to the first demo.
5. Forgo human evaluation
   AI judges should be validated and correlated with systematic human evaluation.
6. Crowdsource use cases
   Have a big-picture strategy to maximize return on investment.

Source: Common pitfalls when building generative AI applications