It is very easy to design experiments that find "bias" when the null hypothesis is that a language model gives consistently dead-even answers to an arbitrary political compass quiz regardless of phrasing and sampling
Comments
Log in with your Bluesky account to leave a comment
There are multiple "levels" of bias. One depends on the training data and on the wording of the promts given. That's indeed hard to assess.
The other is bias introduced by various moderation tools on what topics the LLM allowed output or human prebaked responses and that's easier to assess.
Comments
The other is bias introduced by various moderation tools on what topics the LLM allowed output or human prebaked responses and that's easier to assess.