4 Commits

Author	SHA1	Message	Date
Matt Westcott	ab656424b7	Reinstate spidering of other documentation versions besides stable ... These are now returning the correct canonical URL (thanks to a Cloudflare worker script), and letting search engines spider them again will hopefully boost visibility of that canonical URL.	2021-02-02 15:15:43 +00:00
Matt Westcott	cbf0be31d5	revert to @allcaps's original approach to including robots.txt	2020-12-04 13:30:55 +00:00
Matt Westcott	7f4c914e68	try a different approach for including custom robots.txt	2020-12-02 11:13:23 +00:00
Coen van der Kamp	4702b6f14d	Add custom robots.txt to docs	2020-11-12 16:35:05 +00:00