It looks like it's not just Google, the WaPo story has also been wiped from all other search engines. No matter what you search, the WaPo link will not appear-- it has essentially been wiped from the internet. This is extremely concerning. Does anyone know how something like this could be done? - ThreadSky

taylorlorenz.bsky.social • 77 days ago

It looks like it's not just Google, the WaPo story has also been wiped from all other search engines. No matter what you search, the WaPo link will not appear-- it has essentially been wiped from the internet. This is extremely concerning. Does anyone know how something like this could be done?

Reposted from "Geno"

I'm not sure. I'm seeing a similar result at DDG, so it may not be just google. Could potentially be something to do with WaPo.

Comments

warmonitor.net•77 days ago

Welcome to authoritarian life.

timz.me•77 days ago

The article has the “noindex” meta tag in the source code, which tells search engines not to index the page.

ioletsgo.gay•76 days ago

bring it up with WaPo because it has the tag in its header data.

taylorlorenz.bsky.social•77 days ago

Ok this is REALLY crazy... I can't even find it when I search on WaPo's own website. I searched for the name of the journalist who flagged this issue to me, and she's right, it appears to be hidden even on WaPo's website

droseph.bsky.social•77 days ago

it does seem like a WaPo issue. best to get in touch with them when their office is open, maybe someone in IT

taylorlorenz.bsky.social•77 days ago

The plot thickens. It seems that someone went into the WaPo CMS and essentially wiped the story from the web. Again this was a front page story that was part of a big reporting initiative on press freedom. Why would a big piece on these female journalists stories be hidden??

ioletsgo.gay•76 days ago

feel like Arc XP warrants a looking into if it's WaPo's CMS and they're behind this.

taylorlorenz.bsky.social•77 days ago

Thank u to all who educated me on SEO etc!! I’m going to reach out to wapo about this and find out what happened. I can’t imagine it was nefarious bc again this was a front page story and was brought up a lot internally as work we were proud of! So I will figure out and report back 🫡. Ty all!!

siggtofana.bsky.social•77 days ago

Holy shit

jonkimball.bsky.social•77 days ago

Either an accident by WaPo or they’ve intentionally done this for some reason. If they have noindex on all your past articles then well…

taylorlorenz.bsky.social•77 days ago

All my other articles are totally fine! ;)

dubbaewwteeeff.bsky.social•77 days ago

Okay, my immediate next question: how many other WaPo articles are out there that have been intentionally de-indexed from search engines?

Is there a pattern? Specific subjects or authors?

Replies mention at least one engine that's ignoring the robots.txt - maybe that's a place to start looking.

hvlint.bsky.social•77 days ago

women journalists violence washington post

sluggishcheetah.bsky.social•77 days ago

Is it on the way back machine?

bendowsett.bsky.social•77 days ago

As I'm sure you've surmised by now, sadly by far the most likely explanation is that this was done internally at WaPo

jhnrbk.bsky.social•77 days ago

“Seems”?

jrobie.bsky.social•77 days ago

I get it as the second search result in Brave search, but nothing at all in Bing. And as others have noted, Google returns stories about the story (including a WaPo piece about it winning an award) but not the story itself

paix120.bsky.social•77 days ago

It's probably a "noindex" setting, which hides site content from search engines

bodisattva.bsky.social•77 days ago

Also probably “norobots”

sarahboo.bsky.social•77 days ago

I agree. That’s the most likely explanation. Which would imply that someone at the WaPo itself intentionally or unintentionally added that setting.

paix120.bsky.social•77 days ago

Yep

potrzebiesystem.bsky.social•77 days ago

Is this a common thing?

ioletsgo.gay•76 days ago

its a web standard that most good faith web scrapers abide by, so yes.

potrzebiesystem.bsky.social•77 days ago

I’m curious for selfish reasons bc I’ve noticed from time to time pages I’ve referenced “disappear” on me. (why I save stuff to Wayback like it’s my job)

paix120.bsky.social•77 days ago

The publisher can tag pages in the content management system that they don't want automated systems to crawl, which essentially hides it.

https://en.m.wikipedia.org/wiki/Robots.txt

potrzebiesystem.bsky.social•77 days ago

Appreciate it

paix120.bsky.social•77 days ago

You're welcome

manos.lol•77 days ago

Using site:washingtonpost.com and putting quotes around the headline, I can find pages that evidently have links to the story, including a sitemap page listing the stories published that day.

politevulgarian.bsky.social•77 days ago

I had a bing search that found it being linked to it on reddit but no direct links.

gspeng.bsky.social•77 days ago

Did you check robots.txt?

connorghrey.bsky.social•77 days ago

It’s definitely not a search engine issue. It’s a problem on the WaPo side. Most likely it is not properly indexed.

theradseed.bsky.social•77 days ago

I mean, if it effects their online publishing, it will effect their ads and buyers may not like to hear someone can just come along and hide it... obviously what matters shouldn't be the money, should be the news. Sorry this is happening, I'm jaded.

sashaperigo.bsky.social•77 days ago

I agree with what others are saying that if it’s happening on multiple search engines then it’s most likely on WaPo’s side.

paix120.bsky.social•77 days ago

I would guess it's some setting on WaPo's site that blocks search engines from indexing it.

kevdog.bsky.social•77 days ago

It would have to be story-specific, I think. But I’m sure that you are correct.

lunaravenwolf7.bsky.social•77 days ago

https://www.washingtonpost.com/investigations/2023/02/14/women-journalists-global-violence/

barstool.com•77 days ago

I tried searching the title with Duck & go, to no avail but I got this article just because it has a link with the title of your article in it

I believe you were intentionally buried (by Bezos?)

https://www.washingtonpost.com/technology/2023/06/29/supreme-court-harassment-ruling-first-amendment/

madycast.com•77 days ago

Most likely WaPo themselves could've done this. Not sure why but that seems more likely than Google doing it on purpose?

madycast.com•77 days ago

Publisher of websites can blacklist URLs from being searchable on SEO tools.

madycast.com•77 days ago

Have you checked other articles you wrote for WaPo?

Comments

Posting Rules

Reply