I just had a horrifying realization about Elon's grand plans here. They're going to do a join on a several different data sets with different collection latencies, data quality practices, etc. They're going to act like all the data is perfect, the schema all match, there are no special cases... - ThreadSky

jacobharr.is • 2 days ago

I just had a horrifying realization about Elon's grand plans here.

They're going to do a join on a several different data sets with different collection latencies, data quality practices, etc. They're going to act like all the data is perfect, the schema all match, there are no special cases...

Comments

kathleenweldon.bsky.social•1 day ago

100%
The world's worst case of "did you even read the documentation"?

jacobharr.is•2 days ago

And so, they will claim to find all sorts of anomalies and fraud that are really just ghosts: the effects of them trying to mash together different datasets that weren't designed to be joined together. They won't understand the folly of this endeavor and they'll invade everybody's privacy

jacobharr.is•2 days ago

Basically, every time Elon talks about massive fraud he's found, it's "tell me you've never worked with real datasets before without telling me" and none of his gang of miscreant youths seem to have any relevant background either.

Whatever AI they feed it into will also just assume data is perfect

jacobharr.is•1 day ago

True story: I worked on a system that streamed out Medicare claims to another part of CMS. Some 40+ years after it was created, we discovered for possibly the first time a race condition where the unique claim IDs were not globally unique (but just unique to the data center they were generated)

jacobharr.is•1 day ago

We only figured this out because a record kept flipping its values (because two different records had the same ID). This was found because nobody had been aggregating the data that way before (and it wasn't designed for that). DOGE is going to run into this stuff and not try to understand it

jacobharr.is•1 day ago

“Oh, you should just fix it upstream”

Buddy, that’s a stable system that underlies the largest single health-insurance payments processing in the world. It’s 4% of the GDP. You don’t “just” anything with it. You have to fix your own assumptions instead.

jacobharr.is•1 day ago

Oops I realized I wasn’t entirely clear. It’s been 40+ years of the Medicare claims processing systems. Our code for replicating claims elsewhere and looking for claims that had changed was new and it aggregated claims across multiple data centers. That’s how we found the bug

lastthird.bsky.social•1 day ago

Years ago, a developer was helping another service engineer and me with a fix to the production service we supported. He was being a bit cavalier until my SE colleague cautioned him about the stakes: "you need to have the taste of fear in your mouth".

myariemer.bsky.social•2 days ago

They’re also going to do Very Bad Things with the data beyond just the false accusations against political foes. There are datasets you just don’t put together because of ethical and safety considerations, and considering they’re also exposing a lot of systems to the open internet? Yeah.

myariemer.bsky.social•2 days ago

They’re doing / going to do every bad thing you can to and with data.

daralind.bsky.social•2 days ago

It’s gonna be so awesome* when they turn this Frankenstein over to ICE

lastthird.bsky.social•1 day ago

Ooo my gawd there are so many ways to fuck this up...
• assume datasets' granularities match
• assume id columns can be treated like primary keys
• assume different datasets are equally validated/reliable
• all sorts of false assumptions about datasets' cardinality
• etc
• etc
• etc

dexy26.bsky.social•2 days ago

😳

amandafrench.net•2 days ago

Oh man you’re right. We learned so much about different collection latencies etc at the COVID Tracking Project and kept trying and trying to explain it to people

jacobharr.is•1 day ago

Elon wondering why these fields don’t match and one is six months behind the other or one is only populated in certain conditions or one is income pre-tax and another is after and both are just named income…

kibbles0515.bsky.social•2 days ago

Sounds like they’re going to argue “you got food stamps, why do you also need Medicaid?”

meeper444.bsky.social•1 day ago

These guys are very null license plate.

ICYMI (https://www.wired.com/story/null-license-plate-landed-one-hacker-ticket-hell/)

Comments

Posting Rules

Reply