I just had a "Duh!" lightbulb moment listening to a very nice talk on "Replication and uncertainty" by Anna Dreber Almenberg of the Stockholm School of Economics. Anna did a study on "the design of studies", where she asked a whole bunch of researchers to devise experiments to test a behavioural... - ThreadSky

philipcball.bsky.social • 5 days ago

I just had a "Duh!" lightbulb moment listening to a very nice talk on "Replication and uncertainty" by Anna Dreber Almenberg of the Stockholm School of Economics. Anna did a study on "the design of studies", where she asked a whole bunch of researchers to devise experiments to test a behavioural...

Comments

markusdeserno.bsky.social•5 days ago

David Deutsch makes a similar point in his book “The beginning of infinity”, when he lambasts instrumentalism as “science that does not look for explanations.” He gives the example of behaviorism, e.g. futile attempts to measure whether there’s a genetic basis for “happiness.”

markusdeserno.bsky.social•5 days ago

Deutsch argues that grandiose sounding claims about unmeasurable observables (“happiness”) only ever look at proxies (“do people smile more?”), but without explanatory connections this is useless. Still, people go on to mistake the proxy for the real thing. (Or deliberately conflate the two.)

philipcball.bsky.social•5 days ago

...hypothesis. The challenge was to examine whether or not "competition alters our moral judgement" (I don't recall the exact wording, but it was along those lines). Needless to say, there was plenty of heterogeneity in both experimental design and outcome. /2

deanbooth.bsky.social•5 days ago

Competition for clicks sure has eroded the media’s moral judgment.

philipcball.bsky.social•5 days ago

No surprise there. But the point is that, if these studies were robust enough, they might well get published under a title like "Competition does/does not erode moral values". Yet none would really be testing that hypothesis - because it's pretty meaningless. It's very unlikely that there is... /3

philipcball.bsky.social•5 days ago

...anything general one could say about that issue. This is, however, exactly the sort of framing that the media loves, as well as certain nonfiction authors who write on human behaviour. It's not about replicability (though that could well be an issue too) but about the fact that... /4

philipcball.bsky.social•5 days ago

...such experiments can at best examine only the very specific (and often highly artificial) scenarios they use, and not some more general behavioural or ethical question. I suspect our (or the media's) critical faculties are particularly compromised when the results play into some popular... /5

philipcball.bsky.social•5 days ago

... preconception, or some grand-sounding question ("Was Marx right?"). This is all obvious, of course, but I'm sure I too have sometimes read some wildly generalized claim along the lines of "When people are confronted with X, they do Y" and not challenged it. /6

philipcball.bsky.social•5 days ago

Makes me think of Niels Bohr's insistence on avoiding general questions in quantum mechanics (is it a wave or particle?) and specifying only the outcomes of particular experiments. *Sometimes* we can generalize in science - but in psychology/behavioural economics, maybe not so much. 7/7

Comments

Posting Rules

Reply