A dataset of 1 million or 2 million Bluesky posts is completely irrelevant to training large language models. The primary usecase for the datasets that people are losing their shit over isn't ChatGPT, it's social science research and developing systems that improve Bluesky. - ThreadSky

stellaathena.bsky.social • 88 days ago

A dataset of 1 million or 2 million Bluesky posts is completely irrelevant to training large language models.

The primary usecase for the datasets that people are losing their shit over isn't ChatGPT, it's social science research and developing systems that improve Bluesky.

Reposted from Jeremy Howard

Did you know that 99% of email today is spam? Your inbox isn’t 99% spam because AI is used to filter it.

The same 99% will happen here too, but if AI researchers continue to get perma-banned for making available the datasets needed to filter it, it’s going to make this platform unusable.

Comments

saxon.me•88 days ago

I'm still mad about the 3-year-late uproar over The Pile

stellaathena.bsky.social•88 days ago

I'm glad that machine learning research can keep some journalists in business by giving them a list of things they can go back to find a new thing to be outraged about every six months. 23 components can last you a decade if you space things out right!

saxon.me•88 days ago

Breaking news! Revealing the secret world of publicly available and documented information

tobywalsh.bsky.social•87 days ago

Many of us are still mad about the people who contributed to this harm and deny they had anything to do with it ....

saxon.me•87 days ago

The harm of anthropic et al using the pile in training their commercial LMs? Or the release of the pile as a research artifact?

stellaathena.bsky.social•87 days ago

The Pile includes Books3. You know this because there's a very large, very prominent table on the second page of the paper saying so. There's also several paragraphs throughout the paper about it.

stellaathena.bsky.social•87 days ago

That's from to the question "did EleutherAI create the Books3 dataset." It's a reasonable question to ask as we did collect the data for many of the datasets used in the Pile. However we didn't collect that one, it was collected by Shawn and contributed wholesale to the project.

swyx.io•87 days ago

do we think that Openai et al use Books3 or similar? if it were easy to prove wouldnt Big Publishing have gone after them for it?

dorialexander.bsky.social•87 days ago

They listed "books2" among the sources of GPT-3 (last time they ever released information on training data). As far as I remember "books3" attempted to build a similar collection based on persistent speculations.

stellaathena.bsky.social•87 days ago

They do use similar things, as well as Books3 itself. This is not a topic that's under dispute. OpenAI, Google, Meta, NVIDIA, Apple, Bloomberg, and more have all done this.

Publishers don't want them to stop, they just want to get paid for it. They are actively working to make that happen.

simonwillison.net•87 days ago

We know Meta used it for LLaMA 1 because they listed it in their paper

stellaathena.bsky.social•87 days ago

All I was trying to do was clarify that point to someone who was making incorrect claims about it.

tobywalsh.bsky.social•87 days ago

You didn't collect the data. But you did include it in your dataset, knowing that this was harmful. Shame on you.

tobywalsh.bsky.social•87 days ago

besonders-kreativ.bsky.social•88 days ago

I feel like no death threats from one side and no trolling from the other side would have helped a lot. The people angry about it in the first place clearly don't get the technical details, and the trolls on the AI side were not doing any good by trolling and making them even more angry instead.

zingbretsen.bsky.social•88 days ago

No productive discussions can be had when both sides are poking the hornets nest

xeiaso.net•87 days ago

I've put my article analyzing the situation and talking about the nuance of the datasets ultimately being low risk on the back burner because I just don't want to poke the bear on either side.

gracekind.net•87 days ago

Looking forward to it when you do post!

xeiaso.net•87 days ago

Thanks! I'm half considering paywalling it to patrons for a fair bit though, if only to limit splash damage

stellaathena.bsky.social•87 days ago

I hope you publish it. I would love to read it.

flere-imsaho.circumstances.run•88 days ago

litbowl.bsky.social•87 days ago

Doesn't matter. Still no right to it.

markab.bsky.social•87 days ago

This issue is meaningless until it gets tested in court. GDPR doesn't apply to private use - worry about it when you actually get a demand letter from someone with real jurisdiction.

skateintraffic.bsky.social•88 days ago

I would rather not have my data scraped and your example is a poor excuse that does not require "AI"

Quit and leave

stellaathena.bsky.social•88 days ago

Why are you on a website that makes it trivial for anyone to download all of your interactions if data privacy is so important to you?

This is part of the design of Bluesky. If data privacy if your priority this is the worse social media site to be on.

skateintraffic.bsky.social•88 days ago

Why would anyone scrape data without permission, if they thought *people would give them permission*

A large portion of people joined this site to avoid the data scraping on X

stellaathena.bsky.social•88 days ago

How should I go about getting permission from individual users to use their data on Bluesky?

skateintraffic.bsky.social•88 days ago

Silliest question you could have asked lol

stellaathena.bsky.social•88 days ago

Why is it silly? DM'ing everyone on a website is generally considered spam and would probably get me blocked. So how should I go about doing it?

skateintraffic.bsky.social•88 days ago

Also, please keep up this implicit consent argument, your stupid ass is proving that the industry depends on stolen content and the huggingface people who claim it was unethical are just doing publicity damage control.

menofnearbush.bsky.social•87 days ago

It's designed to be open with public APIs. However, you can keep everything private by not clicking the "Post" button.

skateintraffic.bsky.social•87 days ago

You're reinforcing my point, champ

Which artists are you promoting for with your slop, and did they give permission to the company you're using?

davidgerard.co.uk•87 days ago

look, if people don't want to stink of smoke, they should just sit in the non-smoking section of the aeroplane

catwell.info•88 days ago

If you joined this site to avoid "scraping" then you don't understand this site.

Bluesky has a firehose like Twitter used to. You don't need to scrape, you just need to ask it to send you all skeets.

catwell.info•88 days ago

This dataset was nothing new, and nothing notable. For instance there is an almost complete copy of all of Bluesky, published 7 months ago. 235 million posts, plus all followers, interaction graph, etc. You just don't know it exists (https://zenodo.org/records/11082879).

flere-imsaho.circumstances.run•88 days ago

as you're perfectly aware, this is a dataset explicitly for research purposes, anonymised and with clearly mentioned data controller.

skateintraffic.bsky.social•88 days ago

Buddy, every proponent of "AI" that's making this argument is proving that the Huggingface backtrack is just publicity management, it's unethical regardless of the victim's awareness of it.

This industry depends on not asking permission for or compensating people for something that has value to it.

privacymatters.bsky.social•87 days ago

📌

guillaumesk.bsky.social•87 days ago

This logic is toxic. Why do US AI people refuse to see that you may consent for your data to be used for a specific purpose (social net.) but not for an aggregated version of it to be syphoned and dumped on another platform for a completely different purpose? Consent is consent, why the nuances?

guillaumesk.bsky.social•87 days ago

The first dataset contained PII and it's against the RGPD.

The other ones still contain PII are just "for lolz", not for social research science. Take it from the guy who created the sets, he did it to piss people off. Yet you are defending this?

boy-5.bsky.social•88 days ago

But none of that research or model training requires breaking the law, violating ToS, or disrespecting the wishes of others. There’s a reason research projects require willing participants. Respect consent.

nuliayuk.bsky.social•88 days ago

Which Laws and/or ToS terms were broken?

boy-5.bsky.social•88 days ago

The GDPR 🙄 read the thread jfc

nuliayuk.bsky.social•88 days ago

But you consented to your posts being public already when you agreed to the ToS. Once they are in the public domain anyone can do with them what they like. https://bsky.social/about/support/privacy-policy

boy-5.bsky.social•88 days ago

📌 this is incredible

nuliayuk.bsky.social•87 days ago

Thank you.

stellaathena.bsky.social•88 days ago

This doesn't break the law (at least where I live in the US) and doesn't seem to break the ToS (which puts requirements on commercial products and services, but this is neither).

stellaathena.bsky.social•88 days ago

It's surprising to me how vehement the opposition is as this is built into the website we are on. The way that these datasets are built is exactly the same as the way people spin up custom BS servers. Making data widely available is baked into the website's very design.

Is this surprising to you?

boy-5.bsky.social•88 days ago

It’s not surprising at all, I posted about the open availability of data, the use of AI training and the future reaction of the sites users weeks ago. Having the ability to do something doesn’t give you the right to do it.

tobywalsh.bsky.social•87 days ago

It's worse than that. Some people are happy to do harm as long as it doesn't significantly increase the harm ....

boy-5.bsky.social•88 days ago

It’s a violation of EU law and a large portion of the people whose data was taken live in the EU. It’s a violation of the very first section of the ToS and Community Guidelines. Do you not think it signals a problem with the industry when people take these things without even considering if they can

stellaathena.bsky.social•88 days ago

I don't see anything relevant in the first section of the ToS. In the second section of the ToS, I see that you grant Bluesky permission to share your data with anyone through the AT protocol. It's a little ambiguous what rights those people have. Is that what you meant to refer to?

stellaathena.bsky.social•88 days ago

The first section of the community guidelines is "Don’t use Bluesky Social to break the law or cause harm to others," which again doesn't seem relevant.

EU law has a broad exemption for text and data mining activities. Why do you think building a dataset isn't covered by the TDM exception?

stellaathena.bsky.social•88 days ago

Or were you thinking of something other than copyright law? What EU laws does this violate?

boy-5.bsky.social•88 days ago

Both datasets violated the GDPR, even article 89. The data wasn’t anonymized, contained PII and didn’t minimize data. The 2nd dataset was specifically created for data maximization as a fuck you to consent

timkellogg.me•88 days ago

the EU violation was mainly because they weren’t respecting deletes. i think if they set it up as a service and required that clients cache for no longer than a week (and rebuild their own dataset that often), that might suffice EU law (afaik)

stellaathena.bsky.social•88 days ago

The dataset was online for a couple hours, it feels disingenuous to say it wasn't respecting deletion requests. If it wasn't bullied offline so quickly they probably would have deleted data that people asked them to delete... HF is generally pretty good about that.

flere-imsaho.circumstances.run•88 days ago

the dude who made the dataset blocked the privacy lawyer who asked about the gdpr compliance; there's nothing disingenuous in thinking that the company's compliance with gdpr is, at best, lax.

timkellogg.me•88 days ago

it **is** disingenuous! the anger is mostly people who don’t understand GDPR and really just want to get angry

boy-5.bsky.social•88 days ago

GDPR requires consent, transparency, minimization and privacy. Both datasets violated all of those even if they did accept deletion requests afterward. The vast majority of users don’t even know what huggingface is let alone how to fill out a deletion request

cvoelcker.bsky.social•88 days ago

I think that discounts how normalizing webcrawling for any kind of training and research is still a socially contentious topic at the moment. I don't disagree with you, but I don't think this perspective is as clear cut as presented here.

tilmanbayer.bsky.social•88 days ago

As for "still a socially contentious topic": Your bio indicates that you care about LGBTQ rights. Would you apply this kind of logic there too? ("a massive outpouring of toxic hate and irrational arguments means that we should be more receptive to those demands")

cvoelcker.bsky.social•88 days ago

I didn't (mean to) say "accept the demands", I said "acknowledge where the hate came from in the first place". And yes, sadly this is very necessary when dealing with anti-anything, including LGBTQ rhetoric.
I don't engage with a fundamentalist by telling them they are wrong. It's not useful.

tilmanbayer.bsky.social•88 days ago

OK, sure. But acknowledging and understanding doesn't mean accepting. It's important to push back, not mainly to convince the speaker themselves that they are wrong, but to make the quiet audience (of many social media users who read along without posting) aware that there are different perspectives

cvoelcker.bsky.social•88 days ago

I don't disagree with you at all. But you want to reach those which are somewhat antithetical to your initial position and there it helps to acknowledge. I might have mis-phrased my initial comment, but rereading it, I never said "accept hate"

cvoelcker.bsky.social•88 days ago

I mean, at some point you obviously have to. But it's a difficult path there.

tilmanbayer.bsky.social•88 days ago

Web crawling for research has been "normalized" for decades already. https://pluralistic.net/2023/09/17/how-to-think-about-scraping/

stellaathena.bsky.social•88 days ago

It's become substantially less so the past couple years because tech execs have been telling people that AI is going to ruin their life with no real evidence. Check out our recent study on changing desires:

https://arxiv.org/abs/2407.14933

cvoelcker.bsky.social•88 days ago

Sure, I know. But times have changed, and people's feelings will change? I don't think shouting at them will get us anywhere?

stellaathena.bsky.social•88 days ago

I am extremely aware of how socially contentious it is. I do research on that exact topic and am currently working on the first LLM trained on openly licensed data.

I think that many people on this website need to be told the info in the OP.

cvoelcker.bsky.social•88 days ago

I know your research, sorry, I didn't want to mansplain this. I just think saying "OpenAI will not use this" doesn't fully address e.g. how I feel about the whole thing.

cvoelcker.bsky.social•88 days ago

If we want to continue gathering data for extremely useful goals, we also need to work on governance structures that ensure the ethical and transparent use of that data. Otherwise the answer will always be "both for OpenAI and us".

stellaathena.bsky.social•88 days ago

OpenAI doesn't want this dataset. Any company that wants Bluesky data will press a button and go get all posts on Bluesky. The very design of the website makes that trivial to do. This doesn't help OpenAI *at all*.

stellaathena.bsky.social•88 days ago

I am a fan of transparency, ethical data governance, and all that. Huge fan. Actively work on it. But the widespread misinformation about the basic use-cases for this data is the problem I am trying to address in this specific post.

cvoelcker.bsky.social•88 days ago

OK, got that!

Comments

Posting Rules

Reply