IntenseWebs/discourse
3
0
Fork 0
mirror of https://github.com/discourse/discourse.git synced 2024-11-25 18:30:26 -06:00
Code Issues Packages Projects Releases Wiki Activity

FIX: Show true content of robots.txt after restoring to default (#24980)

Browse Source 
Meta topic: https://meta.discourse.org/t/reseting-robots-txt-override-doesnt-seem-to-work-as-expected/287880?u=osama

Discourse provides a default version for `/robots.txt` which can be customized by admins in `/admin/customize/robots`. In that page, there's a button to reset back to the default version that Discourse provides. However, there's currently a bug with the reset button where the content appears to change to some HTML document instead of the default `robots.txt` version when clicking the button. Refreshing the page shows the true/correct content of `robots.txt` which is the default version, so the reset button actually works but there's a display problem.

What causes this display problem is that we use Rails' `render_to_string` method to generate the default content for `robots.txt` from the template, and what we get from that method is the `robots.txt` content wrapped in the application layout. To fix this issue, we need to pass `layout: false` to the `render_to_string` method so that it renders the template without any layouts.
This commit is contained in:
Osama Sayegh 2023-12-20 23:00:37 +03:00 committed by GitHub
parent 547aafba4f
commit 87883a1963
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
2 changed files with 9 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
app/controllers/admin/robots_txt_controller.rb
View File

@ -29,9 +29,9 @@ class Admin::RobotsTxtController < Admin::AdminController
def original_robots_txt
if SiteSetting.allow_index_in_robots_txt?
@robots_info = ::RobotsTxtController.fetch_default_robots_info
render_to_string "robots_txt/index"
render_to_string "robots_txt/index", layout: false
else
render_to_string "robots_txt/no_index"
render_to_string "robots_txt/no_index", layout: false
end
end
end

8
spec/requests/admin/robots_txt_controller_spec.rb
View File

@ -101,11 +101,17 @@ RSpec.describe Admin::RobotsTxtController do
it "resets robots.txt file to the default version" do
SiteSetting.overridden_robots_txt = "overridden_content"
delete "/admin/customize/robots.json"
SiteSetting.allowed_crawler_user_agents = "test-user-agent-154"
delete "/admin/customize/robots.json", xhr: true
expect(response.status).to eq(200)
json = response.parsed_body
expect(json["robots_txt"]).not_to include("overridden_content")
expect(json["robots_txt"]).not_to include("</html>")
expect(json["robots_txt"]).to include("User-agent: test-user-agent-154\n")
expect(json["overridden"]).to eq(false)
expect(SiteSetting.overridden_robots_txt).to eq("")
end
end