I think language devs can help in a few ways. Benchmarking is the easiest for us to do and necessary to guide LLM development. I’ve been meaning to writeup my experience being only PL person in the room for the StarCoder LLM development process. It was very informative.
Comments
I will be interested to read that post about your experiences on the StarCoder team