Thrilled to have this #openaccess article, co-authored with seven colleagues including @danpemstein.com and @silindberg.bsky.social, published in IPSR! Shockingly long🧵follows. https://doi.org/10.1177/01925121241293459
Comments
Log in with your Bluesky account to leave a comment
In other recent work, colleagues (including @chknutsen.bsky.social, @acrowinghen.bsky.social, @medzihorsky.bsky.social, and @silindberg.bsky.social) and I have discussed the advantages and disadvantages of using experts vs trained coders to code #poliskydata. https://doi.org/10.1017/S104909652300077X
Then, using @vdeminstitute.bsky.social data, we use a variety of techniques to empirically investigate some implications of these arguments, namely the substitutability of crowds for experts and trained coders.
V-Dem asks experts to code temporal trends in different democratic institutions across countries. Experts generally use a five point Likert scale to code their data, where higher scores are normatively better.
For example, this graphic shows the codings experts provided for trends in freedom from political killings in Argentina, 1916-2015. Each dot represents a coding, different colors different experts, with a smoothing line per expert. (Higher scores = more freedom, less killings).
Although there is some variation in scale perception, experts are clearly coding the same trends. (V-Dem uses a measurement model to corrects for this and other potential issues). https://v-dem.net/about/v-dem-project/methodology/
Comments