IntenseWebs/discourse
3
0
Fork 0
mirror of https://github.com/discourse/discourse.git synced 2024-11-22 17:06:31 -06:00
Code Issues Packages Projects Releases Wiki Activity

removed broken link and comments from no_index.erb (#25648)

Browse Source 
We intend to move guidance about how to configure robots.txt to the admin interface and a documentation topic on meta.
This commit is contained in:
Tobias Eigen 2024-02-13 20:09:24 -08:00 committed by GitHub
parent cd99e20fd1
commit 709bed5d1c
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 0 additions and 6 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
app/views/robots_txt/no_index.erb
View File

@ -1,9 +1,3 @@
# See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file
#
# Googlebot must be allowed to index so it can remove items from the index
# we return the X-Robots-Tag with noindex, nofollow which will ensure
# indexing is minimized and nothing shows up in Google search results
User-agent: googlebot
Allow: <%= Discourse.base_path + "/" %>
Disallow: <%= Discourse.base_path + "/uploads/*" %>