Might be a problem with how I had it set up though. I was using llama3.2 (3b params) and while it was able to call a few tools, it obviously had no idea what it was doing. I might have to try other models later to see if they perform better. - ThreadSky

About ThreadSky

joshuashew.bsky.social • 38 days ago

Might be a problem with how I had it set up though. I was using llama3.2 (3b params) and while it was able to call a few tools, it obviously had no idea what it was doing.

I might have to try other models later to see if they perform better.

Comments

joshuashew.bsky.social•35 days ago

I tried "JohnRoger/cogito-v1-preview-qwen-14B-Q6_K-GGUF" but it ran super slow and eventually gave me an error ):

joshuashew.bsky.social•35 days ago

This 8B version performed better on my computer, but it wasn't able to do anything useful with the tools. Same issue as llama3.2

joshuashew.bsky.social•35 days ago

Maybe I'm doing something wrong, or maybe I'm, as HF puts it, "GPU poor"

brianell.in•37 days ago

I’d be curious to hear what you find when trying other local models. I have heard that Cognito does a decent job at local tool calling https://huggingface.co/lmstudio-community/cogito-v1-preview-qwen-32B-GGUF

joshuashew.bsky.social•35 days ago

I'm super close to being able to run something useful on my computer with the 16GB of RAM I have, but it's not quite there yet.

joshuashew.bsky.social•37 days ago

I’ll have to try it out! Thank you for sharing

Posting Rules

Be respectful to others
No spam or self-promotion
Stay on topic
Follow Bluesky's terms of service

Comments

Posting Rules

Reply