I'm thrilled that our paper on base-rate neglect (BRN) and on-the-job experience (w/ @cfchabris.bsky.social and @michellemeyer.bsky.social) was just accepted at Management Science! TLDR: on-the-job experience with base-rate type problems mitigates, but does not eliminate, base-rate neglect. 1/7 - ThreadSky

felipeaaraujo.bsky.social • 95 days ago

I'm thrilled that our paper on base-rate neglect (BRN) and on-the-job experience (w/ @cfchabris.bsky.social and @michellemeyer.bsky.social) was just accepted at Management Science!

TLDR: on-the-job experience with base-rate type problems mitigates, but does not eliminate, base-rate neglect.

1/7

Comments

ahumbleman.bsky.social•95 days ago

📌

fernandaedi.bsky.social•95 days ago

I’ll be talking about base rate neglect in my undergrad class (300 students!) this Tuesday. It will be so cool to let them know about this new work! Thanks for sharing.

felipeaaraujo.bsky.social•95 days ago

Wow. Can I add this to my CV? :)

rfisman.bsky.social•90 days ago

This is really neat and also great class material (as others have noted). A question having just gotten halfway through - does it concern you that the left pair of bars in Fig 1 average out to more than the right pair of bars? That is, medical workers have higher BRN on average?

felipeaaraujo.bsky.social•90 days ago

Thank you Ray! There's no difference in the average rates of perfect BRN (58% for both), but for less experienced, rates are higher for medical, and for more experienced rates are higher for non-medical. That pattern is a little surprising but we can't really unpack it...

rfisman.bsky.social•88 days ago

Yes, sorry, i misread the exact heights of the bars. But what you describe was indeed the concern. Do you try different cutoffs as well? I don't mean to relitigate after it's already gotten its well-deserved acceptance but I'm very interested in this particular result!!

felipeaaraujo.bsky.social•85 days ago

Hi Ray sorry I missed this. You mean the cutoffs on the base-rate neglect classification? Yes we go form 0 to 5 (epsilon). The pattern is also there for the other measures: change in estimates when the base rate changes and when we look at distance to Bayesian posterior as well, which...

felipeaaraujo.bsky.social•85 days ago

I think is very interesting indeed. Maybe less-experienced medical professionals are more overconfident and rely too much on test sensitivity? Maybe some version of the curse of knowledge? I don't really know.

fmckenna.bsky.social•89 days ago

Cool!

felipeaaraujo.bsky.social•95 days ago

We run a survey experiment with medical professionals in a clinical role. These professionals routinely analyze diagnostic tests (~20 a week on average), which require conditional probability assessments where base rates play a crucial role.

2/7

felipeaaraujo.bsky.social•95 days ago

For example, even if the test sensitivity, P(+ | disease), is super high, the probability of disease after a positive result, P(disease | +), can be very low if the disease is sufficiently rare.

3/7

felipeaaraujo.bsky.social•95 days ago

Using a simple medical scenario, we compare the incidence of BRN among professionals with more or less years of experience.

Our control group is composed of non-medical professionals from the same organization, matched on education (roughly) and age.

4/7

everythingnothing.bsky.social•95 days ago

Care to share the details of the scenario?

felipeaaraujo.bsky.social•95 days ago

Here's the scenario we use (p.6):

everythingnothing.bsky.social•95 days ago

Thank you!

felipeaaraujo.bsky.social•95 days ago

We find that more experienced medical professionals (1) are much less likely to completely ignore base rates, (2) provide estimates closer to the Bayesian posterior, and (3) adjust their responses by more when we ask a second question where only the base rate is changed.

5/7

felipeaaraujo.bsky.social•95 days ago

No such differences are observed in the control group; if anything, experience is associated with slightly higher BRN among the non-medical professionals.

6/7

felipeaaraujo.bsky.social•95 days ago

The paper reports a number of robustness checks and explore mechanisms. One mechanism in particular is potentially relevant: the number of false positives encountered.

Link to WP: https://drive.google.com/file/d/1CvaI2vm6PD1POCRl_obBfLN1J4cCHAEA/view?usp=sharing

7/7

Comments

Posting Rules

Reply