This comes as a bit of a surprise to Christopher Kyba himself, who somehow has a lot of memories of being underground at the SNO site 🤔 #AISlop - ThreadSky

skyglowberlin.fediscience.org.ap.brid.gy • 2 days ago

This comes as a bit of a surprise to Christopher Kyba himself, who somehow has a lot of memories of being underground at the SNO site 🤔

#AISlop

Comments

skyglowberlin.fediscience.org.ap.brid.gy•2 days ago

My daughter just came up with a great exercise: challenge your students to find the title of your PhD using ONLY LLMs (no Google allowed). If any of them manage, they get gummy bears 😃

I asked five different models, and got five different answers, all five […] https://fediscience.org/@skyglowberlin/114717155114415996

Arta.toot.lv.ap.brid.gy•1 day ago

@skyglowberlin ChatGPT answered absolutely correctly about me and my dissertation. Found on web, of course.

skyglowberlin.fediscience.org.ap.brid.gy•1 day ago

@Arta Interesting. What does it get for you for the prompt I used ("Where did Christopher Kyba get his PhD, and what was the title?")?

Both of these returned to me just now are wrong.

Arta.toot.lv.ap.brid.gy•1 day ago

@skyglowberlin i asked (in Latvian): do you know where Arta Snipe got her PhD and What were her thesis about. At first it said they do not have personal info, but when I answered that this is publicly known stuff, it came back with an accurate answer.

skyglowberlin.fediscience.org.ap.brid.gy•1 day ago

@Arta Sorry, I wasn't clear - I was curious what would happen if you asked for my name - whether maybe the model you are using is doing a better job of finding additional data or something.

Arta.toot.lv.ap.brid.gy•1 day ago

@skyglowberlin a, ok 😆

thomas.tikatech.org.ap.brid.gy•1 day ago

@skyglowberlin I just asked Claude and it got place and title of my really obscure thesis exactly right. What is wrong with LLMs is unfortunately not only that they are wrong.

skyglowberlin.fediscience.org.ap.brid.gy•1 day ago

@thomas There's a lot of things wrong with them 😃

I tried you on GPT and it returned Stuttgart. Out of curiosity, what do you get for me on Claude?

thomas.tikatech.org.ap.brid.gy•1 day ago

@skyglowberlin

Claude (which obviously has Internet access):

"University: Christopher Kyba received his PhD in Experimental Particle Physics from the University of Pennsylvania in 2006

Thesis Title: "Measurement of the Atmospheric Neutrino Induced Muon Flux at the Sudbury Neutrino […]

tinoeberl.mastodon.online.ap.brid.gy•2 days ago

@skyglowberlin

The chatbot is better. But you used the API or the model directly, right?

skyglowberlin.fediscience.org.ap.brid.gy•2 days ago

@tinoeberl U of A is still wrong. I did a bachelor degree there. Try telling it I didn't get a degree in Canada - when I tried that, it said I got my degree in Heidelberg, then when I told it my degree was in the USA it said my PhD was from Berkeley, and when I said no, it was in the eastern US […]

fusion.mastodon.social.ap.brid.gy•1 day ago

@skyglowberlin temperature = 0, avoids hallucination (pixtral-12b)?

skyglowberlin.fediscience.org.ap.brid.gy•1 day ago

@fusion I'm not quite sure what you mean by "avoids hallucination"? I mean, reduced variability from the default model, sure, but unless they directly reproduce training data, all text output from LLMs is made up.

But it's a great example, because GFZ isn't a degree granting institution. That's […]

fusion.mastodon.social.ap.brid.gy•21 hours ago

@skyglowberlin “Hallucination” means the deviation from the trained data for the statistical processing of the answers.
BTW: An LLM is not designed as a “reference book” and is therefore the wrong tool for the job.

Comments

Posting Rules

Reply