JGaulard
Administrator
Staff member
Site Supporter
Sr. Site Supporter
Power User
- Joined
- May 5, 2021
- Messages
- 319
- Reaction Score
- 2
- Points
- 18
- #1
I was peeking around the Coverage > Crawling - Currently Not Indexed area of Google Search Console (GSC) last night when I discovered something that disturbed me. My website runs on WordPress and for every single page I had on the site, an RSS feed page was created. So If I had:
There would also be:
WordPress creates feed URLs for the homepage, category pages, pages, and posts. Basically everything. Ask me why someone would need a feed page for a blog post and I'll tell you that I have absolutely no idea. It's a crawl budget waste, if you ask me.
Google usually knows what to do with these URLs, but the fact that they're being crawled bothers me. I don't use RSS for anything on the site, so I thought completely getting rid of them would be the best course of action. I looked up what to do, did it, and now I'm here to share the process and results with you.
There are two steps to this solution. The first is to remove the feed URLs from the source code. To do this, I added the following code to the functions.php file of my child theme:
The above code removes all feed code, except the highest level feed for the homepage. I don't know why it doesn't get rid of that one. It does a good job of removing all others though. Since removing the code from the pages isn't good enough, I went a step further. Google had already crawled all my feed URLs, so simply removing the code wouldn't stop Google from crawling them in the future. And if they were crawled, they'd still be considered their own unique URLs. I wanted to allow crawling of the URLs, but I wanted to fold them into their parent URLs. To do this, I added the following code to the website's .htaccss file:
This code redirects the feed URLs to their parents, so if a blog post created a feed URL, that URL would redirect to the blog post itself. Not bad. The only issue I found was the comment feed redirected to an invalid page because there's no "comment" page. But that was only one URL, as far as I could tell.
One word of warning - when editing the functions.php file, edit the one in your child theme, not the parent theme. This way, when the parent theme gets updated, the change to the file won't be overwritten. Also, make a note to yourself somewhere in your files to remind yourself of the change you made. When you update themes in the future, you'll be thankful for that note.
If you have any questions about this process, please ask. I'd be happy to help. Thanks!
https://example.com/category/wolvesThere would also be:
https://example.com/category/wolves/feedWordPress creates feed URLs for the homepage, category pages, pages, and posts. Basically everything. Ask me why someone would need a feed page for a blog post and I'll tell you that I have absolutely no idea. It's a crawl budget waste, if you ask me.
Google usually knows what to do with these URLs, but the fact that they're being crawled bothers me. I don't use RSS for anything on the site, so I thought completely getting rid of them would be the best course of action. I looked up what to do, did it, and now I'm here to share the process and results with you.
There are two steps to this solution. The first is to remove the feed URLs from the source code. To do this, I added the following code to the functions.php file of my child theme:
Code:
// REMOVE RSS FEEDS
// Redirect to the homepage all users trying to access feeds.
function disable_feeds() {
wp_redirect( home_url() );
die;
}
// Disable global RSS, RDF & Atom feeds.
add_action( 'do_feed', 'disable_feeds', -1 );
add_action( 'do_feed_rdf', 'disable_feeds', -1 );
add_action( 'do_feed_rss', 'disable_feeds', -1 );
add_action( 'do_feed_rss2', 'disable_feeds', -1 );
add_action( 'do_feed_atom', 'disable_feeds', -1 );
// Disable comment feeds.
add_action( 'do_feed_rss2_comments', 'disable_feeds', -1 );
add_action( 'do_feed_atom_comments', 'disable_feeds', -1 );
// Prevent feed links from being inserted in the <head> of the page.
add_action( 'feed_links_show_posts_feed', '__return_false', -1 );
add_action( 'feed_links_show_comments_feed', '__return_false', -1 );
remove_action( 'wp_head', 'feed_links', 2 );
remove_action( 'wp_head', 'feed_links_extra', 3 );
// REMOVE RSS FEEDSThe above code removes all feed code, except the highest level feed for the homepage. I don't know why it doesn't get rid of that one. It does a good job of removing all others though. Since removing the code from the pages isn't good enough, I went a step further. Google had already crawled all my feed URLs, so simply removing the code wouldn't stop Google from crawling them in the future. And if they were crawled, they'd still be considered their own unique URLs. I wanted to allow crawling of the URLs, but I wanted to fold them into their parent URLs. To do this, I added the following code to the website's .htaccss file:
Code:
# BEGIN FEED REDIRECT
<IfModule mod_rewrite.c>
RewriteEngine on
RewriteRule ^(.*/)?feed(/rss|/rss2|/atom|/rdf)?/?$ /$1 [R=301,NC,L]
RewriteCond %{QUERY_STRING} (?|&)feed=
RewriteRule (.*) $1/? [R=301,NC,L]
</IfModule>
# END FEED REDIRECTThis code redirects the feed URLs to their parents, so if a blog post created a feed URL, that URL would redirect to the blog post itself. Not bad. The only issue I found was the comment feed redirected to an invalid page because there's no "comment" page. But that was only one URL, as far as I could tell.
One word of warning - when editing the functions.php file, edit the one in your child theme, not the parent theme. This way, when the parent theme gets updated, the change to the file won't be overwritten. Also, make a note to yourself somewhere in your files to remind yourself of the change you made. When you update themes in the future, you'll be thankful for that note.
If you have any questions about this process, please ask. I'd be happy to help. Thanks!