Every time I try one of these tools for science I increasingly feel like people who are stoked about them are just telling on themselves. “It’s good at data analysis” Well, alternatively… - ThreadSky

jbakcoleman.bsky.social • 2 days ago

Every time I try one of these tools for science I increasingly feel like people who are stoked about them are just telling on themselves.

“It’s good at data analysis”

Well, alternatively…

Reposted from Angela Collier

If you need help coming up with a 'hypotheses' you shouldn't be a scientist. Every scientist I know has 100 project ideas and lack time to get to them. No on is sitting around twiddling thumbs without an interesting question to ask.

We are in the dumbest timeline.

Comments

jrgant.bsky.social•2 days ago

I just reread a few sections of this commentary (one of my old favorites), and it is actually a little eerie amid the science agent stuff.

https://journals.lww.com/epidem/citation/1993/05000/the_hypothesis_generating_machine.12.aspx

jrgant.bsky.social•2 days ago

Btw, I think it's tempting to go the "telling on themselves route", but understanding the lit is in part an ingestion problem. Personally, I think we should expect AI agents to (eventually) best us in the synthesis dept, leaving our human brains more time to focus on causal reasoning.

jrgant.bsky.social•2 days ago

Sounds like you have more experience with these tools than I do. (I abandoned them for coding but am exploring LLM-based structured data extraction.)

So I'm just speculating idly.

jbakcoleman.bsky.social•2 days ago

I find that if I’m asking some question of the data I have more domain knowledge and understanding of how that links to a statistical model than I can jam into a prompt. It will give me something back but unless I hold its hand the whole way the analysis is asking a different question.

jrgant.bsky.social•2 days ago

Agree here too. This is a general problem with software abstractions of even simple statistical tools: they're happy to spit numbers at you.

jbakcoleman.bsky.social•2 days ago

And if you just say “how do I analyze this” it’ll often jam data into something common but either limiting or wholly inappropriate. It’s great for like bash scripts and simple manipulations that can be checked, writing documentation etc..

jrgant.bsky.social•2 days ago

Oh yeah, I don't want to offload mapping the scientific problem to an analysis. I should probably use them more for the rote stuff like docs and bash scripts, like you said.

stephenjwild.bsky.social•2 days ago

Are we still using the term "stochastic parrots"?

wpball.com•2 days ago

New Scientist is an absolute joke publication.

jrgant.bsky.social•2 days ago

To be fair, the article isn't totally effusive. They point to failures, too. Though I did not know NS is owned by Daily Mail...

Comments

Posting Rules

Reply