By default most CDNs will cache full HTML pages if accessed. This means that a copy of your site will start to appear and be indexed by search engines (in SEO terminology: "duplicate content"). This is duplicate content and can cause search engines to penalize your site in search results. A duplicate site is also confusing to the end-user and unprofessional.

If you enable CDN module's duplicate content prevention feature, the CDN module will ensure that all requests for (Drupal-served) HTML content to the CDN will redirect your end-users to the Drupal site itself.

Duplicate content prevention also works for cached pages.

## Detailed information for the experts

Requests from the CDN are detected by the `User-Agent` request header. The default list of CDN user agents covers the Akamai and Amazon CloudFront CDNs (at the time of writing), and can be modified through CDN's admin UI. Case-insensitive, substring-based matching is used.

An edge case here is that of "Drupal-generated files", such as image styles. These are files that should be served from the CDN, but first have to be generated by Drupal. Unfortunately, there is not yet an API in Drupal to detect this programmatically. Hence, the only feasible solution was to use a heuristic: if the URL ends in `.htm`, `.html` or `.php` (some people prefer this over "clean URLs"; you could easily configure in the `path` or `pathauto` modules), or when the URL does not have a file extension at all, it is assumed to be HTML content, and the duplicate content prevention feature will be applied. Otherwise, it is assumed that the content is a file that should be served from the CDN.
Currently, no UI is provided to override this, but you can programmatically override it, see `CDN_SEO_FORBIDDEN_EXTENSIONS_DEFAULT`.

The CDN will receive a `HTTP/1.0 301 Moved Permanently` response, with a `Location` header to perform the redirect and a `Link: ; rel="canonical"` header to indicate to search engines what the location of the canonical representation of the current web page is. The CDN user agent will not (should not) follow the redirect, and should instead simply cache this response and serve this response to any end-user trying to access it.

You can verify that it is working as intended by accessing a development version of your site with two different browsers, e.g. Google's Chrome and Mozilla's Firefox. Add e.g. "Firefox" to the list of CDN user agents, and then try to load any page (e.g. the front page) in Firefox: you should get an endless redirect loop and the browser should bail; while in Chrome everything should continue to work just fine.