When Google marks thousands of your pages as “crawled – currently not indexed”, the obvious reaction is to delete them. I didn’t delete a single one. On a community radio station website I run, with about 32,000 posts, I used noindex tags for 13,336 pages and set another 710 to Private, and I’d make the same call again (though ask me in December whether it worked!).
Noindex, private or delete: what actually differs
All three get a page out of Google, but they behave very differently for everyone else. A noindex tag leaves the page working for visitors and for anyone with a link, and asks Google not to list it. Setting a post to Private makes it a 404 for the public and for Google, and the post stays safely in the database. Deleting removes it for good, and you can only get it back from a backup.
Why I picked noindex for the thin posts
The thin posts were show recaps, each with an embedded player, a track list and a line or two of text. I went through about 200 categories, put 121 of them on a noindex list and left 79 alone, and I also noindexed any post under 100 words. Those pages are no use as a search result, but a listener who had just heard the show still wanted the track list. Noindex kept the page for them and took it out of Google’s way. It also takes about a minute to undo, whereas a delete can’t be undone without restoring a backup. I wrote up the code and the Search Console figures in my post on what “crawled – currently not indexed” means.
Why I used Private for the risky ones
Noindex wasn’t strong enough for the sponsored posts. I found about 2,280 of them (gambling, kratom, payday loans and similar), almost none with rel="sponsored" on the paid links. For the 710 highest-risk ones I wanted Google and the public to get a 404 straight away, which Private does, but I still didn’t want them gone forever, because the people who paid for them might ask about them. The other 118 borderline ones got a noindex tag. The remainder moved to a rule that keeps a sponsored post indexed for 12 months and then noindexes it automatically. The reasoning is in my post on site reputation abuse.
When deleting is the right answer
Deleting makes sense when a page is worthless to everyone, nobody links to it and you have a backup. A duplicate, a test post or an empty page would all qualify. If I couldn’t say all of that about a post, it didn’t get deleted. Two things to check first. A noindex tag only works if Google can crawl the page, so don’t also block it in robots.txt. And if a page has real links pointing at it, redirect it to a similar page rather than deleting it.
What happened next
Honestly, nothing yet. Search Console showed 2.25k clicks in the 28 days to 1 October and 2.23k in the 28 days to 16 October, which is flat. Pages in Google’s index fell from about 14,000 to 8,340, so Google did what I asked. Whether having fewer and better pages in the index lifts the rest, I can’t tell you yet. I’m not touching anything until 1 November and 1 December, because making 3 changes at once would leave me unable to say which one worked.
So that’s 13,336 noindexed, 710 private and 0 deleted, and the only thing I’m sure of is that I can still undo every one of them with a single click!