mirror of
https://github.com/discourse/discourse.git
synced 2025-02-25 18:55:32 -06:00
FEATURE: Disallow tags and categories rss feeds
This stops crawlers from hitting tags and category rss feeds to discover new content, instead they should focus on latest/posts if they need to consume something regular
This commit is contained in:
@@ -30,6 +30,8 @@ Disallow: /*?*api_key*
|
||||
Disallow: /groups
|
||||
Disallow: /groups/
|
||||
Disallow: /t/*/*.rss
|
||||
Disallow: /tags/*.rss
|
||||
Disallow: /c/*.rss
|
||||
|
||||
<% if @disallowed_user_agents %>
|
||||
<% @disallowed_user_agents.each do |user_agent| %>
|
||||
|
||||
Reference in New Issue
Block a user