I enjoyed this summary article by Martin Fowler on patterns when building GenAI products. If you're trying to build these products in a grown up way, what do you need to do minimise the shortcomings of LLMs? (a lot). martinfowler.com/articles/gen... This bit on RAG templates made me lol though: - ThreadSky

danpoxton.bsky.social • 2 days ago

I enjoyed this summary article by Martin Fowler on patterns when building GenAI products. If you're trying to build these products in a grown up way, what do you need to do minimise the shortcomings of LLMs? (a lot).

https://martinfowler.com/articles/gen-ai-patterns

This bit on RAG templates made me lol though:

Comments

prietschka.bsky.social•2 days ago

“Hope and pray!”

All of that is intractable and not something that can be controlled.

This is all such folly. The only thing I’ve found workable in LLM land over the last year is the magnificent build out of LLM support MLflow has done the past year or so.

prietschka.bsky.social•2 days ago

Truly great if you need to build and serve, though it’s all getting a bit byzantine as they seem to be determined to keep up with all the other garbage frameworks like Langchain. https://mlflow.org/docs/latest/llms/index.html

danpoxton.bsky.social•2 days ago

This little plea they add to the reg template feels like magical thinking though i.e. LLMs don't have meta-cognition skills, so really, this method of trying to ensure factual accuracy just by simply *asking* the LLM to be accurate feels hinky.

danpoxton.bsky.social•2 days ago

I mean, if I ask an LLM to write an nursery rhyme in the style of a gangster rapper, it'll do a fine job because it will have seen examples of content like that and will regurgitate the vocabulary and style nicely. However, if I ask an LLM to mimic the style of "factual accuracy" I'm in trouble.

danpoxton.bsky.social•2 days ago

Factual accuracy isn't a particular style or type of content, it requires proper meta-cognition i.e thinking about the way we think, understanding cognitive biases, and seeking to minimise contradictions and logical fallacies.

danpoxton.bsky.social•2 days ago

LLMs just don't do that, so the most you are going to adding a plea like this to the RAG template is a response which stylistically apes a factual report, without actually being a guarantee of factual accuracy.

danpoxton.bsky.social•2 days ago

I may be way off on this. I've never built a RAG system. Feels very desperate though: begging the LLM to exercise cognitive abilities it patently doesn't have, but just lobbing it into the query because you hope some magic in training data will somehow shine through.

Comments

Posting Rules

Reply