Tag Archives: Google Bot

CSF Installation Problem: Solve Blocking Google Bot

“CSF” and “Mod security” both are very important for your server. If any user wants to install them then they might see it blocking Google bot for some specific rules.

Here is what you should be doing to make Google Bots crawl through your sites again.

You have to restart Apache after running the below rules to mod_security configuration file :

# GoogleBot by user-agent…
SecRule HTTP_USER_AGENT “Google” nolog,allow
SecRule HTTP_USER_AGENT “Googlebot” nolog,allow
SecRule HTTP_USER_AGENT “GoogleBot” nolog,allow
SecRule HTTP_USER_AGENT “googlebot” nolog,allow
SecRule HTTP_USER_AGENT “Googlebot-Image” nolog,allow
##
SecRule HTTP_USER_AGENT “AdsBot-Google” nolog,allow
SecRule HTTP_USER_AGENT “Googlebot-Image/1.0″ nolog,allow
SecRule HTTP_USER_AGENT “Googlebot/2.1″ nolog,allow
SecRule HTTP_USER_AGENT “Googlebot/Test” nolog,allow
SecRule HTTP_USER_AGENT “Mediapartners-Google/2.1″ nolog,allow
SecRule HTTP_USER_AGENT “Mediapartners-Google*” nolog,allow
SecRule HTTP_USER_AGENT “msnbot” nolog,allow read more