Search

How to Stop Google from Crawling cPanel Webmail Pages?

  • Thread starter CampFireJack
  • Start date
CampFireJack

CampFireJack

Member
Joined
May 9, 2021
Messages
118
Reaction Score
1
Points
16
  • #1
I have doing the "site:" command on my website recently to see what's indexed by Google. I've been noticing some weird pages in the results, such as cPanel ID and Webmail Login. There are quite a few of them. These are both system files that I obviously don't want Google indexing. Is there a way to stop this? How can I block Googlebot from crawling these cPanel system pages? Can I turn off cPanel Webmail? I don't use it anyway - never have. What can be done about this? Here are some screenshots of the Google results:

cpanel-system-pages-crawled-by-google.gif
 
JodyBuchanan

JodyBuchanan

Member
Joined
May 10, 2021
Messages
138
Reaction Score
0
Points
21
  • #2
cPanel Webmail does get crawled sometimes and as you've deduced, the related pages are a pain in the butt to see in the index and to get rid of. Many people try to block the pages in their robots.txt file to no avail. The problem is, these are system files that are being crawled at domain.com:2096 (webmail ports 2095/2096), so they're not able to be blocked.

Instead of attempting to remove Webmail in its entirety, you can easily block access to the pages in WMD. When crawled by Googlebot, they'll return HTTP error 401 unauthorized header status codes.

To set this up, log into WHM. Then either search for Host Access Control in the top search bar or browse to Security Center > Host Access Control. Click that link.

Then, in the first box, type WEBMAILD. In the second box, type ALL. In the third box, type DENY. Instead of denying everything, you may also block by certain IP addresses. Also, if you've got other commands on this page, be sure the DENY commands are listed at the bottom. The further down the page, the higher the precedence.

Here's a screenshot of the completed command on the Host Access Control page.

block-webmail-whm-host-access-control.gif
 
Top