Anecdote from yesterday for the LLMs-can't-thinkers: Claude debugging zir own visualization code by looking at the image output. (I didn't try any control experiment.) news.ycombinator.com/item?id=4286... - ThreadSky

About ThreadSky

abecedarius.bsky.social • 32 days ago

Anecdote from yesterday for the LLMs-can't-thinkers: Claude debugging zir own visualization code by looking at the image output. (I didn't try any control experiment.)
https://news.ycombinator.com/item?id=42860819

Comments

gojomo.bsky.social•31 days ago

wow. I'd previously pasted Claude a screenshot of a figure it was generating for me, to help explain how it was slightly off. coaching it to self-eval seems next level.
…

gojomo.bsky.social•31 days ago

…
do you think there's any chance it didn't *really* behold the rendering (especially within a single 'turn'), but the exercise itself unlocked a higher/deeper conception of what it was generating? (ala when LLMs hallucinate reliable external tools?)

abecedarius.bsky.social•30 days ago

This was essentially right, ze did capture the image but then only analyzed rough features using console.log as the output channel, rather than vision on the full image the way it can work on a screenshot.

gojomo.bsky.social•30 days ago

so got the image-bits into an execution-environment variable, but never fed them to self's 'eye'-like model input weights (as a user-pasted screenshot would've been available)?

abecedarius.bsky.social•30 days ago

seems that way. I was just trying some more stuff in this session and needed to debug, so tried investigating this some more.

abecedarius.bsky.social•30 days ago

Must say I'm not too pleased with the role of pointy-haired boss of a junior programmer who's impressively educated but unpracticed at doing things

abecedarius.bsky.social•31 days ago

Some such possibility is why I qualified it with "no control". But it did look at the phone photo in the turn before and verbalize the problem it was showing.

Posting Rules

Be respectful to others
No spam or self-promotion
Stay on topic
Follow Bluesky's terms of service

Comments

Posting Rules

Reply