Lovely example of how weird real-world data is. I put all 55 million points in Overture maps into a Nomic Atlas map. Lots of points spuriously appearing in the ocean. But the 1° square around 'null island' (0° lat, 0° long) is suspiciously clear -- someone has cleared out bad data just there. - ThreadSky

bschmidt.bsky.social • 53 days ago

Lovely example of how weird real-world data is. I put all 55 million points in Overture maps into a Nomic Atlas map. Lots of points spuriously appearing in the ocean. But the 1° square around 'null island' (0° lat, 0° long) is suspiciously clear -- someone has cleared out bad data just there.

Comments

tilmanbayer.bsky.social•52 days ago

I once trawled the Yellow Sea (+122...° long) fishing for Wikipedia articles about San Francisco to repatriate them to the Bay Area (-122...° long)
https://en.wikipedia.org/w/index.php?diff=prev&oldid=586190050

bschmidt.bsky.social•52 days ago

Related -- can you spot the country with 1.4 billion people but that Overture (basically Meta + Microsoft combining their map datasets) has surprisingly little data about?

sladeofyaupon.bsky.social•48 days ago

very envious of the great firewall

bschmidt.bsky.social•52 days ago

Even cooler! Near null island, there's a set of points shaped like the US (you can see Florida in the lower right.) But when you zoom out, they're off the coast of Liberia, exactly where they'd be if the latitude and longitude were both divided by 10 (eg so NYC is at 4.1, -7.4 instead of 40.7, -74).

bschmidt.bsky.social•52 days ago

Oh wow and there's a third one you can just see *inside* the null island exclusion zone at 1/100 scale (so where NYC would be at 0.41 long, -0.74 lat). I wouldn't be *sure* it was Florida if there wasn't a gun and ammo shop there.

lemonodor.bsky.social•52 days ago

I bet there's a fourth one too.

up87.bsky.social•30 days ago

Latitude and Longitude switched?

roope.bsky.social•52 days ago

Floridas all the way down!

bschmidt.bsky.social•52 days ago

“My god, it’s full of floridas”

jtheibault.bsky.social•51 days ago

This is pretty amazing. I wonder if it is just the US, or do other countries get their own little pockets of 0.1 and 0.01?

tylerbhampton.bsky.social•52 days ago

Would you mind sharing what version of the dataset you downloaded? I might like to use the data as an in-class exercise for students on data cleaning

bschmidt.bsky.social•52 days ago

2024-10-23.0

Here's the code -- gist at https://gist.github.com/bmschmidt/250ac80bb5a843e2db0ffb8da794e164

elfprince13.mumak.app•52 days ago

lmao

semmiotic.bsky.social•52 days ago

It’s just Micromerica. Nothing to see here, move along.

bschmidt.bsky.social•52 days ago

Yeah how could I forget the whole part of Liberian history where the freedmen settling it built a Dubai-style development in the shape of the United States in honor of their old homes

winj1n.bsky.social•15 days ago

And they have a third one, even smaller, United States in there!

yiestee.bsky.social•52 days ago

china had harsh digital data export control, up to RMB 500k fine if any person uploads info to non-mainland servers without authorization

dragonfyre.bsky.social•52 days ago

Is that where you catch Missingno?

postholer.com•52 days ago

Overture data is reprocessed OpenStreetMap data, which is largely crowdsourced, accuracy isn't guaranteed.

More disturbing, geoparquet has row level CRS, not table level. Which means you can have mixed CRS in the data.

elfprince13.mumak.app•53 days ago

This fills me with joy

rickbrannan.com•52 days ago

Had a teacher in middle school who called that point "the armpit of Africa."

sinai-rusinek.bsky.social•52 days ago

@dh-dev.com

davidlaurence.bsky.social•52 days ago

Dept of stray thoughts: would systematic manipulation of lat and long coordinates in this way depict tectonic plate movement through geologic time?

bschmidt.bsky.social•52 days ago

One of my favorite fun facts about CRS is that one of the reasons you can't use lat/long for civil engineering is continental drift. Australia is moving fast compared to the rest of the world, so if you store in lat/long and drill for a sewer line in ten years, you'll hit the gas line next to it.

lovestealstheday.bsky.social•52 days ago

Fun fact: this is where Felonius Gru's secret lair is located

fbedecarrats.bsky.social•52 days ago

Can this come from an error in coordinate reference system? It looks like at some stage points in a longitude/latitude coordinates (eg. WGS84, ranging from -180 to + 180) were merged with points having a projected system (eg. Web Mercator).

bschmidt.bsky.social•52 days ago

I definitely made a mistake in putting the points here (I tried to hand-write the WGS84 -> EPSG:3857 transform in a sql query and got it wrong here.) And yeah I think that is probably where a bunch come from. E.g. below is off the coast of Africa, but appears to be the outline of the shape of the US

bschmidt.bsky.social•52 days ago

So technically not a CRS error I guess. But I wouldn't be surprised if those exist too

Comments

Posting Rules

Reply