FEATURE: Send missing post uploads stat to prometheus

approved
#1

FEATURE: Send missing post uploads stat to prometheus

diff --git a/plugin.rb b/plugin.rb
index 6e06533..92f6ad1 100644
--- a/plugin.rb
+++ b/plugin.rb
@@ -75,6 +75,15 @@ after_initialize do
     metric.job_name = stat.name
     metric.duration = stat.duration_ms * 0.001
     $prometheus_client.send_json metric.to_h
+
+    if stat.name == "Jobs::EnsurePostUploadsExistence"
+      name = Jobs::EnsurePostUploadsExistence::MISSING_UPLOADS
+      metric = DiscoursePrometheus::InternalMetric::Custom.new
+      metric.name = "Missing post uploads"
+      metric.value = PostCustomField.where(name: name).count
+      metric.labels = { name: name }
+      $prometheus_client.send_json metric.to_h
+    end
   end
 
   DiscourseEvent.on(:sidekiq_job_ran) do |worker, msg, queue, duration|

GitHub sha: e839fabe

#2

If we are doing this kind of stuff we should have a filtered index here, making it unique also protects us from data corruption

Follow Up #3
#5

Now missing post uploads check is changed to a rake task instead of a scheduled job. So we’re not going to run this for all the sites regularly. I will change the above metric sending code too. Then we don’t need to calculate the count here.

#6

The index on the custom field is more of an insurance policy, it makes sure that we never have multiple names for a single post id. Since it is filtered it only consumes space for stuff that is part of the index.

So if we have:

create unique index idx on post_custom_fields(post_id) where name = 'Missing post uploads'

We then have an insurance policy that we will never have to deal with dupes there which is awesome. Plus all lookups and counts are way faster.

1 Like
DEV: add unique missing uploads index in post custom fields
#7

Thanks for the explanation :heart:. I will create an index.

2 Likes
#8

It’s added in commit DEV: add unique missing uploads index in post custom fields · discourse/discourse@8d5c900 · GitHub

1 Like
Approved #10