Denybot

From Contao Community Documentation


denybot - Hide page content for search engine bots.

Extension-Overview
Name of the developer weke
Version of the extension 1.0.0 stable
Compatibility with Contao Version 2.9.5 - 2.9.5
Link to Extension Repository http://www.contao.org/erweiterungsliste/view/denybot.html
Depending of ff. Extension Bot Detection

Summary

This Extension deny the access to all or single web pages for detected Web crawler.

Functionality

With the usage of the extension Bot Detection the calling agent should be identified. If the calling agent is a search engine bot, the real content of the web page was hidden and a 404-Errorpage will return instead.

Configuration

In System-Settings you could find in sektion Global configuration the option to exclude searchengines global.

If this option is set, every frontend access from a identified search enginge bot will return a 404-error page.

If this option is not set, only pages with selected robot-tag "noindex,nofollow" or "noindex,follow" in their page configuration will generate the 404-error page for detected serarch engine bots.

Why should I use this extension

A normal web site is designed to be found in the internet.

This extension will serve the opposite function and try to hide the content. Search engines should only found not existing pages.

In that case this extension is designed especially for private usage. If you want to personally announce that you created a web page, e.g. to stay in contact with your family and friends across the word, and don't want to invite everyone in the internet, this extension could protect your pages against search engine bots. This extension could only be used as an additional features like using a login to protect your content. It could not replace any other type of data protection.

Restriction

The detection of a search engine bot depends on the quality of Bot Detection engine. The Author BugBuster explicitly pointed out: "There is no reliable detection guaranteed." The search engines are detected by their user agent string and their IP address. BugBusters still collect a huge list of different and popular search engines and he still extends the quality of detection.

Views
Personal tools

Contao Community Documentation

In other languages
Navigation
Discover
Understand
Enhance
Miscellaneous
Tools