going to interrupt you here and remind you that you made a strong claim and I’ve asked for strong evidence to support it I can’t engage on these secondary points if you won’t engage on the primary one - ThreadSky

lacrunk.bsky.social • 82 days ago

going to interrupt you here and remind you that you made a strong claim and I’ve asked for strong evidence to support it

I can’t engage on these secondary points if you won’t engage on the primary one

Comments

lazyblueberry.bsky.social•82 days ago

I would also like to point out, even without synthetic data, these LLMs are already trained on "lies". The dataset is the entirety of the Internet, it's full of lies.

It's also full of jokes which a predictive text generator can't discern and is how you get the glue pizza answers

lazyblueberry.bsky.social•82 days ago

https://research.ibm.com/blog/LLM-generated-data

lacrunk.bsky.social•82 days ago

ah, so given these sources, I think you think I'm claiming "synthetic training data isn't being used to train LLMs." That is not the claim I'm making, but I can see how you might have thought that.

lacrunk.bsky.social•82 days ago

I'm claiming "1) there is no evidence (yet) that synthetic data (not foibles of the attention mechanism) is the source of false information given by LLMs, i.e. that they spout misinformation that both a) is high-bias, not high-variance, and b) can be traced directly back to synthetic data."

lacrunk.bsky.social•82 days ago

This is a pretty high bar and requires sources that are either bleeding-edge mechanistic interpretability papers or are straight up proprietary secrets, as AI companies don't release their training data sets.

lacrunk.bsky.social•82 days ago

So any claims that "synthetic data is 100% certainly the direct cause of LLM misinfo" are too confident, and we should not be making them until we have that evidence.

lacrunk.bsky.social•82 days ago

there's more fuzzy (but less interesting) discussion in the other claim, which is that "2) AI companies have incentives to reduce the amount of misinformation their products output, because it a) embarrasses them (bad marketing) and b) degrades the utility of their product (not worth paying for)."

lazyblueberry.bsky.social•82 days ago

I literally don't know what the hell you're on about at this point. You already know these programs are effectively black box algorithms, that nobody has access to. We don't know what they're trained on exactly, or how predictions are weighted.

lacrunk.bsky.social•82 days ago

"cutting-edge research in mechanistic interpretability" -> researchers have devised methods to trace LLM outputs to selections of training data.

Ergo, if you found a false claim in the outputs, and traced it back to training data marked as synthetic, you'd be able to prove this relationship.

lazyblueberry.bsky.social•82 days ago

What we do know is that these programs already give erroneous answers given the datasets they're currently using. If they're generating synthetic data from this dataset which again the entirety of the Internet is what they claim- the obvious conclusion is that synthetic data is erroneous as well

lazyblueberry.bsky.social•82 days ago

https://www.thestack.technology/llama33-release/

lazyblueberry.bsky.social•82 days ago

https://lifehacker.com/tech/ai-is-running-out-of-internet

Comments

Posting Rules

Reply