What would you recommend if you‘d want to run a small Text-to-SQL model on a minimal budget? Let’s say 10k requests/month? - ThreadSky

About ThreadSky

tobilg.com • 3 days ago

What would you recommend if you‘d want to run a small Text-to-SQL model on a minimal budget?

Let’s say 10k requests/month?

Comments

tdoehmen.bsky.social•2 days ago

We also have an eval space on huggingface where you can evaluate different prompts and models (for DuckDB) -->
https://huggingface.co/spaces/duckdb-nsql-hub/DuckDB-SQL-Eval
And a leaderboard -->
https://huggingface.co/spaces/duckdb-nsql-hub/DuckDB-NSQL-Leaderboard

tdoehmen.bsky.social•2 days ago

It's an interesting question. Thinking that pay-as-you-go endpoints with general purpose models + rag/prompt eng. may be the more practicable and economical choice here. You could do 10k requests/month with a few $. I'm also working on a paper on how to do that well for DuckDB Text2SQL.

sahuguet.bsky.social•3 days ago

You mean running something like https://motherduck.com/blog/duckdb-text2sql-llm/ on your own cloud/server?

tobilg.com•3 days ago

Yes, the same model. I experimented with a few options on AWS and Hetzner some months ago, but w/o GPU support it doesn’t really work…

Cheapest working option I found was renting a M1 Mac Mini, but this will set you back for 80€/month

nicoritschel.com•3 days ago

Your best bet is probably a metered API or something like https://vast.ai

sahuguet.bsky.social•3 days ago

Have you looked at https://zml.ai/ or https://textsynth.com/index.html .

Posting Rules

Be respectful to others
No spam or self-promotion
Stay on topic
Follow Bluesky's terms of service

Comments

Posting Rules

Reply