Yet again a big "meta-analysis" of the effects of ChatGPT on "learning performance" - largely arguing there are big positive effects - is getting wildly and uncritically promoted by the usual big names and yet again it appears to be a review of seriously bad science... - ThreadSky

benpatrickwill.bsky.social • 4 days ago

Yet again a big "meta-analysis" of the effects of ChatGPT on "learning performance" - largely arguing there are big positive effects - is getting wildly and uncritically promoted by the usual big names and yet again it appears to be a review of seriously bad science...

Comments

lotarous.bsky.social•4 days ago

Main arguments?

janetdonavan.bsky.social•3 days ago

Also important to consider the high level of influence ed tech has on ed research and how many ed researchers are funded by ed tech or working directly for them.

benpatrickwill.bsky.social•4 days ago

A huge problem around AI in education is its really shaky evidentiary culture. Bad science can get published in journals with weak review processes, then swept up in poor meta-analyses, published as if the proof is in, then circulated by big-follower influencers who don't interrogate the sources...

benpatrickwill.bsky.social•4 days ago

Ilkka Tuomi has done the digging on this terrible AI in education "evidence" reproduction again over on LinkedIn if you want receipts https://www.linkedin.com/feed/update/urn:li:activity:7326446988420403201?commentUrn=urn%3Ali%3Acomment%3A%28activity%3A7326446988420403201%2C7326544072230240256%29&replyUrn=urn%3Ali%3Acomment%3A%28activity%3A7326446988420403201%2C7326677003904901123%29&dashCommentUrn=urn%3Ali%3Afsd_comment%3A%287326544072230240256%2Curn%3Ali%3Aactivity%3A7326446988420403201%29&dashReplyUrn=urn%3Ali%3Afsd_comment%3A%287326677003904901123%2Curn%3Ali%3Aactivity%3A7326446988420403201%29

lilianedwards.bsky.social•3 days ago

📌

benpatrickwill.bsky.social•4 days ago

The weak evidentiary culture of a lot of AIED is a massive problem that AIED research community leaders challenged only weeks ago after the last similar example - so this is a known problem inside the field, not just an external critique https://osf.io/preprints/psyarxiv/t6uzy_v2

lotarous.bsky.social•3 days ago

(Recommending section 4 of the linked text, for anyone interested but lacking the time to read the whole text.)

benpatrickwill.bsky.social•3 days ago

A more in-depth piece on the quality problem of AIED meta-analyses from Ilkka Tuomi is now up over at LinkedIn too https://www.linkedin.com/posts/ilkka-tuomi-30086_the-effect-of-chatgpt-on-students-learning-activity-7327264463550595073-gl_J?utm_source=share&utm_medium=member_android&rcm=ACoAAFLDL60Bv5rV_wStM3jmcO03lf3CVkIyl6E

shuchig.bsky.social•3 days ago

I too posted this link to Ilkka Tuomi’s post over on X in response to Ethan Mollick uncritically sharing the meta-analysis.

katjathieme.bsky.social•16 hours ago

You are not the only one.

brodiegal.bsky.social•4 days ago

and (in my experience) swept up by weaker students using … assisted means to write their reports (which also prefer easy to scrape sources that aren’t paywalled?). the increased presence of low grade sources in such works is too much to be coincidence 😕

jennifermromig.bsky.social•4 days ago

“Learning styles” paved the way for this unfortunate credulity

andrebn.bsky.social•2 days ago

📌

harrietthespy.bsky.social•3 days ago

I remember when we had to run college papers through a software program that detected plagiarism before we submitted them to a professor. Artificial intelligence is an oxymoron.

interacciones.org•3 days ago

Of course, the junk article has already been shared on LinkedIn by that AI-for-education guru who wrote in his book that until GenAI “we have never built a generally applicable technology that can boost our intelligence.”

dissembly.bsky.social•3 days ago

i know its a stupid statement on the face of it but: Pencils, ballpoint pens, clay tablets, paper, word processors, books, printing press, the internet, databases, dyes, rock art, storytelling techniques, language etc etc etc

interacciones.org•3 days ago

Yes! https://bsky.app/profile/interacciones.org/post/3lfq7jrz5vc2o

dissembly.bsky.social•3 days ago

💜

bert926.bsky.social•3 days ago

Trump's sopranos name; "Donny 2 fingers sexual assault rapist bone spurs 2 dolls", abbreviation Donny 2 dolls; "politically correct", "delusional golfing 13 stroke handicap yelling four after his golf ball goes into the Golf of America"!

m1ngle.bsky.social•3 days ago

Oof. Yeah that’s not surprising. Thanks for surfacing this.

Posting Rules

Be respectful to others
No spam or self-promotion
Stay on topic
Follow Bluesky's terms of service