🙈 Let's say you have a LONG list of URLs, and some are NSFW… How do you clean the URL list? 🤔 I experimented a bit and came up with something pretty solid! Check each URL using a content blocking DNS server. Huh? I know. Read on 👇 - ThreadSky

About ThreadSky

benschwarz.bsky.social • 10 days ago

🙈 Let's say you have a LONG list of URLs, and some are NSFW… How do you clean the URL list? 🤔

I experimented a bit and came up with something pretty solid!

Check each URL using a content blocking DNS server.

Huh? I know. Read on 👇

Comments

benschwarz.bsky.social•10 days ago

💡 Turns out, Cloudflare, NextDNS & other DNS providers have content-blocking DNS servers, so by pinging their server with a URL, you can see if it's been blocked:

benschwarz.bsky.social•10 days ago

I recently took advantage of this by writing a Node.js script that loops through a list of website URLs, blocked hosts get added to a new JSON file:

dbushell.com•10 days ago

nice!

I'm thinking you could DIY it by spinning up a DNS server like Pi-hole in a container with blocklists, quick search finds https://github.com/hagezi/dns-blocklists for example

probably a waste of time though! unless CloudFlare hits you with a rate limit

benschwarz.bsky.social•10 days ago

I didn’t seemingly get blocked ever… but I suspect out and out abuse would get snuffed out pretty quickly. I’m a nextdns customer now :)

dbushell.com•10 days ago

yeah it's probably a drop in the ocean compared to some of the spam they deal with

now I'm tempting to starting digging every IP from 0.0.0.0 and see how far I get... 🤔

benschwarz.bsky.social•10 days ago

Yes you could! Pretty cool 😎

benschwarz.bsky.social•10 days ago

Boom, presto. You've got a block list, and hopefully learned a tiny something about DNS. Happy Friday 😎

ncoughlin.com•10 days ago

benschwarz.bsky.social•10 days ago

I sure felt clever 🤗

andydavies.me•9 days ago

That's a really good approach!

Posting Rules

Be respectful to others
No spam or self-promotion
Stay on topic
Follow Bluesky's terms of service

Comments

Posting Rules

Reply