The Five Commandments of Probing LLMs/AI Models. Want to know what an LLM/AI model can and cannot do? Then this list is for you. (Thread by https://fedihum.org/@[email protected] & @spinfocl.bsky.social)
Comments
Log in with your Bluesky account to leave a comment
1. “The AI” doesn’t exist. It is almost impossible to make valid statements about “the AI”. As of April 2024, there are 603,288 models on Huggingface. They have been trained on different datasets, with different settings and even different architectures.
Do not generalise from your experience with one or a few models to “the AI”. You can make experiments and statements about specific models, and should always include the model name (and version).
2. A single prompt is not an experiment. Models are sensitive to their input. Showing a relationship from input to output with a single prompt does not prove anything (in most cases). You need to have a strategy for testing multiple prompts in different variants.
3. Anecdotal evidence is not evidence. Many experiments involve classifying the LLM responses in some way (e.g., as sexist or not). You need a proper experimental setup to do this, because you are the worst person to do it.
4. Server-side storage of conversations is not secure. Many commercial models offer sharing functionality. Do not trust these. They can disappear at any time. If you want to make provable/reliable statements about model responses, you need to store them somewhere under your own control.
Comments