FIX: drop deleted posts from search index

FIX: drop deleted posts from search index

This does two things

  1. Our “index grace period” has been wound down to 1 day, there is no point keeping a bloated index for a week, usually when people delete stuff they mean for it to be removed

  2. We were never dropping deleted posts from the index, only posts from deleted topics

These changes speed up search a tiny bit and reduce background work.

diff --git a/app/jobs/scheduled/reindex_search.rb b/app/jobs/scheduled/reindex_search.rb
index 6d3b431..b941a22 100644
--- a/app/jobs/scheduled/reindex_search.rb
+++ b/app/jobs/scheduled/reindex_search.rb
@@ -5,7 +5,7 @@ module Jobs
   class ReindexSearch < Jobs::Scheduled
     every 2.hours
 
-    CLEANUP_GRACE_PERIOD = 1.week.ago
+    CLEANUP_GRACE_PERIOD = 1.day.ago
 
     def execute(args)
       rebuild_problem_topics
@@ -79,8 +79,12 @@ module Jobs
           FROM post_search_data
           LEFT JOIN posts ON post_search_data.post_id = posts.id
           INNER JOIN topics ON posts.topic_id = topics.id
-          WHERE topics.deleted_at IS NOT NULL
-          AND topics.deleted_at <= :deleted_at
+          WHERE (topics.deleted_at IS NOT NULL
+          AND topics.deleted_at <= :deleted_at) OR (
+            posts.deleted_at IS NOT NULL AND
+            posts.deleted_at <= :deleted_at
+          )
+
         )
       SQL
     end
diff --git a/spec/jobs/reindex_search_spec.rb b/spec/jobs/reindex_search_spec.rb
index b7f5d83..c5ce22c 100644
--- a/spec/jobs/reindex_search_spec.rb
+++ b/spec/jobs/reindex_search_spec.rb
@@ -115,21 +115,25 @@ describe Jobs::ReindexSearch do
       post2.save!(validate: false)
       post3 = Fabricate(:post)
       post3.topic.trash!
-      post4 = nil
+      post4, post5, post6 = nil
 
       freeze_time(described_class::CLEANUP_GRACE_PERIOD) do
         post4 = Fabricate(:post)
         post4.topic.trash!
+
+        post5 = Fabricate(:post)
+        post6 = Fabricate(:post, topic_id: post5.topic_id)
+        post6.trash!
       end
 
-      expect { subject.execute({}) }.to change { PostSearchData.count }.by(-2)
+      expect { subject.execute({}) }.to change { PostSearchData.count }.by(-3)
 
       expect(Post.pluck(:id)).to contain_exactly(
-        post.id, post2.id, post3.id, post4.id
+        post.id, post2.id, post3.id, post4.id, post5.id
       )
 
       expect(PostSearchData.pluck(:post_id)).to contain_exactly(
-        post.post_search_data.post_id, post3.post_search_data.post_id
+        post.id, post3.id, post5.id
       )
     end
   end

GitHub sha: 74c4f926

1 Like