I am genuinely curious abt the planned uses for these agents
Science is asking questions & allowing for unexpected answers
How can an agent surprise us in an experiment? In a survey?
An agent is a predictive model
Such models are often proved wrong by field experiments
https://arxiv.org/pdf/2411.10109
Comments
LLMs should do a reasonably good job of interpreting questions like an "average" human, so this kind of agent could make it much cheaper/easier to iterate on how questions are written.
I use AI
AI is useful for repetitive tasks, some of which are very complicated
Testing hypotheses about human behavior (on humans) is not a repetitive task
In this case, the very complicated rules that the 1000 agents follow are determined by the responses of 1000 people as interpreted by AI
Useful for simulating and calibrating models
The use case of “experiments and surveys” is what has me calling bullshit
(Note that humans are being studied in that sentence)
I can see no case in which AI-generated agents could replace humans as the *subjects* in scientific research into human behavior
Just call it a complicated simulation
🙃
It is often obscured by the sales language
And it’s appropriately modest
I fear this is not how the creators are thinking
If being against gen AI makes me a Luddite, then I'm proud of that
"The Luddites were members of a 19th-century movement of English textile workers who opposed the use of certain types of automated machinery due to concerns relating to worker pay and output quality."
An LLM is (approx) trained on the universe of text to behave like humans do.
To someone who has read and understood the universe of text; the LLM will never surprise. But none of us have!
what I am arguing is unpredictable is not the genius of the scientist, but rather the behavior of the studied humans
Is it synthesizing the results of past experiments?
Designing new experiments?
Extrapolating from the existing results?
It’s very early days with all this stuff of course
A bit like talking to colleagues about the design to see if they have hypotheses about the results I didn't think about.