They could have implemented digital watermarking, but chose not to. Meanwhile schools are hiring edutech companies to tell them how to use AI in class. We're so far underwater with GenAI's effects on education that it's going to take a lot more than shitty AI tutors to get back to baseline. - ThreadSky

carlbergstrom.com • 97 days ago

They could have implemented digital watermarking, but chose not to.

Meanwhile schools are hiring edutech companies to tell them how to use AI in class. We're so far underwater with GenAI's effects on education that it's going to take a lot more than shitty AI tutors to get back to baseline.

Comments

bradleymp.bsky.social•97 days ago

My favourite Edu GenAI story is the one professor whose solution was to essentially paste every student’s essay into ChatGPT and ask “Did you make this?”

Obviously the results were completely unreliable.

bradleymp.bsky.social•97 days ago

Unfortunately the professor stubbornly refused to listen to students trying to explain that’s not how GenAI works and there’s absolutely no way ChatGPT of all tools can be used in such a fashion. IIRC the class revolted and had to get the faculty involved to restore sanity to the course.

angryallosaurus.bsky.social•97 days ago

The low tech way would be to provide proof of multiple drafts and revision, which AI can't replicate. Especially if the first draft has to be handwritten (except for those who physically can't write for prolonged periods).

tleighk.bsky.social•97 days ago

Finding and evaluating primary sources and synthesizing information to create new insights will be a lost ability, like celestial navigation

rahuldave.bsky.social•97 days ago

As a computer science educator, while the impact on me has been less, I've seen the deleterious effects of blindly assuming everything an ai spouts is (a) correct (b) relevant (c) useful. But I have found the effect is asymmetrical: it's been hurtful to the first year students...

rahuldave.bsky.social•97 days ago

...while often being a force multiplier for seniors or more advanced students..it's enabled them to go further, do more research, better implementation. The common thing I have found in the better students has not changed: better base literacy, and a strong desire to learn...

rahuldave.bsky.social•97 days ago

Strangely, separating the gems has gotten easier!

carlbergstrom.com•97 days ago

This makes a lot of sense based on my own experience and other studies I've read — skilled coders work better and faster with AI assistance, whereas unskilled ones are set back by over-relying on it.

rahuldave.bsky.social•97 days ago

Yes and not just coding! Any critical thinking! In the next iteration of 1st year courses I might ban ai entirely ) and not entirely successfully) based on results I have seen on in-class exams without ai. I want my advanced students to use as much as they like. Hope I can do this well!

thepeoplegeek.bsky.social•97 days ago

Depends on what the floor of performance is, so with coders, someone with no coding experience, ai raises their floor of performance. Also with experts it boosts somewhat. With creative ideas it’s more of a floor than ceiling raiser (Koivisto & Grassini, 2023).

natesjacobs.bsky.social•97 days ago

This is exactly the same with learning though. I think your original point was too extreme. The truth is that it is incredibly powerful if done well with the right mindset and scope, and obviously as we’ve seen pretty bad if not.

ed3d.net•97 days ago

Yeah, pretty much. My advice to my employer is to give known, trusted, competent folks access to a chat model and restrict it from everybody else.

Instead they're giving out a full suite to everybody who asks for it, and the results have been...uhh...suboptimal.

andrewlrhyne.bsky.social•97 days ago

I can tell you that I can write code (Python, SQL) about 5-10 times faster, C is something I would never bother with but now work in. My code and commits are far more readable, a billion times more enjoyable. It’s also the best copy editor I have ever used.

peterwilliams97.bsky.social•97 days ago

My experience has been that skilled coders have attacked problems like students who try to understand a subject. The less skilled ones have worked like students who want to be given answers to pass tests.

slowxtal.bsky.social•97 days ago

a nice related study :
https://aidantr.github.io/files/AI_innovation.pdf

sawsharkman.bsky.social•97 days ago

I disagree. The level of writing is now so good it takes AI checks to have a hope of separating real writing from fake. Stanford AI tools now use real references, for example. And even if we know it's AI, we have no way to demonstrate with certainty

rahuldave.bsky.social•97 days ago

I was perhaps not very clear. The students who are good have learned how to AI quite well, by experimenting. It's been a force multiplier for them.

jtcarter.bsky.social•97 days ago

This is why philosophy, liberal arts, and other humanities will be more appreciated again. Critical thinking and reading literacy in an AI world are imperative. Ppl need technical skills AND philosophical skills.

pelagicnemo.bsky.social•80 days ago

Celestial navigation is not a lost art, though perhaps more rare. Ships carry sextants and chronometres as a back-up to GPS, and those of us who keep the practise take great pride in it.

mkreed.bsky.social•97 days ago

There's a UW AI survey/recommendation team that's been in my edu email.

omermano.bsky.social•97 days ago

The potential upside of AI tutors really is huge though (see Bloom's 2 sigma problem). It seems like we'll be able to made progress on this in the next 10 years, but I haven't seen much in the way of actual useful implementations yet.

carlbergstrom.com•97 days ago

Yes, but (1) this study (obviously) and the more recent ones touting this effect in light of AI involve students tutored by *people*. Given that education involves motivation as well as instruction, it's not clear that AI is a substitute.

carlbergstrom.com•97 days ago

and (2) I'm really tired of hearing from tech bros about how the shit they might invent in the future is going to make up for the shit they've broken in the present.

omermano.bsky.social•97 days ago

That's true. They really should have enabled watermarking. Unfortunately now that Llama 3 is out the cat is kind of out of the bag for homework cheating.

ccguerilla.bsky.social•97 days ago

Watermarking doesn't come close to solving the pre existing problem. This is wishful thinking. I say that as someone who enjoyed writing essays as a student, and sees the value of them in a small number of disciplines

ccguerilla.bsky.social•97 days ago

The essay was deceased as a relatively secure assessment form for more than a decade before chatgpt

agent139.bsky.social•97 days ago

I don't think there's a baseline to get back to, when you consider what's happening across "society".

lindstocking.bsky.social•97 days ago

The irony being the conclusion of The Diamond Age was the personal attention of an interested party (ie parent) was required for the virtual teacher to be effective.

idcn.bsky.social•97 days ago

AI isn’t just an unstoppable force; it’s an opportunity to reimagine education in ways we’ve never seen before. For teachers, this could mean a return to the heart of their profession, teaching. Rather than being bogged down by administrative tasks.

johngordon.bsky.social•97 days ago

I’m doing an undergrad course (free for over 62 at MN state colleges including UMN) and was quite sad I have to do in class written exams instead of a paper.

krugerlive.bsky.social•97 days ago

It’s frustrating too because the legitimately good uses get overshadowed. I lead a middle school math product at an edtech co, so lots of market pressure to put AI hype features in (we push back on fluff). But things like realtime simplified language and “explain this” type features can be helpful.

arbiteriapetus.bsky.social•97 days ago

I'm pretty well convinced that digital watermarking, although potentially useful as an arms race to advance the technology, won't be useful to monitor AI-generated content.
https://www.rand.org/pubs/commentary/2024/01/the-case-for-and-against-ai-watermarking.html

arbiteriapetus.bsky.social•97 days ago

Even if it hypothetically could be one day (which I doubt) it certainly isn't ready, yet, and wasn't ready, then, to stop generative AI from killing the homework essay.

menger-spongebob.bsky.social•97 days ago

wym, digital watermarking?

carlbergstrom.com•97 days ago

Gemini can help.

Comments

Posting Rules

Reply