FIX: add X-Robots-Tag header for check_xhr-covered GET actions, too (#9868)

FIX: add X-Robots-Tag header for check_xhr-covered GET actions, too (#9868)

  • FIX: add X-Robots-Tag header for check_xhr-covered GET actions, too

see https://meta.discourse.org/t/missing-x-robots-tag/152593/3 for context

  • test: a spec making sure X-Robots-Tag header is present when needed

/groups path responds to anonymous requests and doesn’t skip check_xhr method, so we can use it here.

diff --git a/app/controllers/application_controller.rb b/app/controllers/application_controller.rb
index 2f2b8a9..a0f0a56 100644
--- a/app/controllers/application_controller.rb
+++ b/app/controllers/application_controller.rb
@@ -39,12 +39,12 @@ class ApplicationController < ActionController::Base
   before_action :redirect_to_login_if_required
   before_action :block_if_requires_login
   before_action :preload_json
+  before_action :add_noindex_header, if: -> { is_feed_request? || !SiteSetting.allow_index_in_robots_txt }
   before_action :check_xhr
   after_action  :add_readonly_header
   after_action  :perform_refresh_session
   after_action  :dont_cache_page
   after_action  :conditionally_allow_site_embedding
-  after_action  :add_noindex_header, if: -> { is_feed_request? || !SiteSetting.allow_index_in_robots_txt }
 
   layout :set_layout
 
diff --git a/spec/requests/groups_controller_spec.rb b/spec/requests/groups_controller_spec.rb
index 11eb7eb..eb6b4d5 100644
--- a/spec/requests/groups_controller_spec.rb
+++ b/spec/requests/groups_controller_spec.rb
@@ -176,6 +176,12 @@ describe GroupsController do
       )
     end
 
+    it 'should return correct X-Robots-Tag header when allow_index_in_robots_txt is set to false' do
+      SiteSetting.allow_index_in_robots_txt = false
+      get "/groups"
+      expect(response.headers['X-Robots-Tag']).to eq('noindex, nofollow')
+    end
+
     context 'viewing groups of another user' do
       describe 'when an invalid username is given' do
         it 'should return the right response' do

GitHub sha: 12544c02

This commit appears in #9868 which was approved by eviltrout. It was merged by eviltrout.