Sometimes our anthropocentric assumptions about how intelligence "should" work (like using language for reasoning) may be holding AI back. Letting AI reason in its own native "language" in latent space could unlock new capabilities, improving reasoning over Chain of Thought. arxiv.org/pdf/2412.06769 - ThreadSky

About ThreadSky

emollick.bsky.social • 140 days ago

Sometimes our anthropocentric assumptions about how intelligence "should" work (like using language for reasoning) may be holding AI back. Letting AI reason in its own native "language" in latent space could unlock new capabilities, improving reasoning over Chain of Thought. https://arxiv.org/pdf/2412.06769

Comments

asifk.bsky.social•137 days ago

Nice to see this, since the true value of abstraction lies in transformations in the latent space that can’t easily be done in the input-output space.

It’s worth noting, though, that their results were worse than CoT for grade school math, but better than CoT for random logic/planning puzzles.

scottleibrand.bsky.social•139 days ago

This also would represent a dangerous loss of transparency into the LLM’s internal deliberations, making it far harder to detect duplicitous scheming in superhuman models. It would be better to sacrifice some efficiency to preserve the ability for humans or weaker models to evaluate safety.

ngutten.bsky.social•139 days ago

Sampling does seem to be important though, for modeling searches in infinite-dimensional spaces (e.g. open sequences). It's a bit harder to do that nicely in continuous high dimensional space than with tokens.

prevmed.bsky.social•139 days ago

Hi Ethan! Off topic: who are your favorite people to follow here?

Are you open to creating a post on that? And potentially ask others to share their favorite people?

Thank you for considering this request!

calebsponheim.com•140 days ago

Any thoughts about how one could implement this as a user in current LLM offerings such as sonnet 3.5? Or is COCONUT beyond traditional prompt engineering?

Posting Rules

Be respectful to others
No spam or self-promotion
Stay on topic
Follow Bluesky's terms of service

Comments

Posting Rules

Reply