PERF: Introduce absolute limit of digests per 30 minutes (PR #10845)

To avoid blocking the sidekiq queue a limit of 10,000 digests per 30 minutes is introduced.

This acts as a safety measure that makes sure we don’t keep pouring oil on a fire.

On multisites it is recommended to set the number way lower so sites do not dominate the backlog. A reasonable default for multisites may be 100-500.

This can be controlled with the environment var

DISCOURSE_MAX_DIGESTS_ENQUEUED_PER_30_MINS_PER_SITE

GitHub

LGTM :+1: There are failed tests but seems unrelated.