I'm shocked that we went from *rightfully* mocking a ex-Google engineer from believing an LLM was sentient after spending too much time without to having half of the tech industry believe the exact same thing. Don't get high on your own supply, don't drink the entire bottle of Koolaid folks! - ThreadSky

About ThreadSky

calvobianco.com • 32 days ago

I'm shocked that we went from *rightfully* mocking a ex-Google engineer from believing an LLM was sentient after spending too much time without to having half of the tech industry believe the exact same thing.

Don't get high on your own supply, don't drink the entire bottle of Koolaid folks!

Comments

royvanrijn.com•32 days ago

True, it's clearly been trained on more advanced models and mimicking gets you pretty far. The reasoning steps work, but big prompts with enough clear information does the same.

I usually gather information using 4o and based on that I let o1 come up with a good answer... that works even better.

calvobianco.com•32 days ago

Mind you: I'm not doubting the usefulness of LLMs, I see the value and use them daily.

But more folks should actually read the research and architecture behind them.

It's not a magic box, it's very clever tech. :)

royvanrijn.com•32 days ago

I believe it's doing mostly the same as Microsoft did with Orca:
https://www.infoq.com/news/2023/12/microsoft-orca-2-llm/

Where it's using a much more advanced model to create synthetic (labelled) data explaining reasoning, and learning from that. Bootstrapping the knowledge and making it _seem_ very smart.

calvobianco.com•32 days ago

It's a very clever approach. And we can wonder: if the thing is not actually reasoning (well, kinda from a linguistic perspective), but the audience believes it does, does that matter?

royvanrijn.com•32 days ago

It's writing down all the things it knows before one-shotting an answer, which is very clever. It has more context and can catch mistakes early (and allows to make it) instead of just hallucinating some wrong answer.

royvanrijn.com•32 days ago

Imagine you need to answer a complex question and you *need* to say the first words that come to mind... no matter what. That won't probably lead to a coherent and correct answer as well.

Posting Rules

Be respectful to others
No spam or self-promotion
Stay on topic
Follow Bluesky's terms of service

Comments

Posting Rules

Reply