We've been working on just that at @weightsbiases.bsky.social with Weave!
Weave is a lightweight llm tracing and evaluations toolkit, that focuses on letting you iterate fast and make sure that your production LLM based application is not degrading when you change prompts or models!
Weave is a lightweight llm tracing and evaluations toolkit, that focuses on letting you iterate fast and make sure that your production LLM based application is not degrading when you change prompts or models!
Comments
We autopatch most AI provider libraries so you basically only just tend to import and init and you're good to go!
🔗 https://wandb.me/weave