FEATURE: Send missing post uploads stat to prometheus

FEATURE: Send missing post uploads stat to prometheus

diff --git a/plugin.rb b/plugin.rb
index 6e06533..92f6ad1 100644
--- a/plugin.rb
+++ b/plugin.rb
@@ -75,6 +75,15 @@ after_initialize do
     metric.job_name = stat.name
     metric.duration = stat.duration_ms * 0.001
     $prometheus_client.send_json metric.to_h
+
+    if stat.name == "Jobs::EnsurePostUploadsExistence"
+      name = Jobs::EnsurePostUploadsExistence::MISSING_UPLOADS
+      metric = DiscoursePrometheus::InternalMetric::Custom.new
+      metric.name = "Missing post uploads"
+      metric.value = PostCustomField.where(name: name).count
+      metric.labels = { name: name }
+      $prometheus_client.send_json metric.to_h
+    end
   end
 
   DiscourseEvent.on(:sidekiq_job_ran) do |worker, msg, queue, duration|

GitHub sha: e839fabe

If we are doing this kind of stuff we should have a filtered index here, making it unique also protects us from data corruption

Now missing post uploads check is changed to a rake task instead of a scheduled job. So we’re not going to run this for all the sites regularly. I will change the above metric sending code too. Then we don’t need to calculate the count here.

The index on the custom field is more of an insurance policy, it makes sure that we never have multiple names for a single post id. Since it is filtered it only consumes space for stuff that is part of the index.

So if we have:

create unique index idx on post_custom_fields(post_id) where name = 'Missing post uploads'

We then have an insurance policy that we will never have to deal with dupes there which is awesome. Plus all lookups and counts are way faster.

1 Like

Thanks for the explanation :heart:. I will create an index.

2 Likes

It’s added in commit DEV: add unique missing uploads index in post custom fields · discourse/discourse@8d5c900 · GitHub

1 Like