FEATURE: Oneboxer cache response body (PR #12562)

Some oneboxes may fail if when making excessive and/or odd requests against the target domains. This change provides a simple mechanism to cache the results of succesful GET requests as part of the oneboxing process, with the goal of reducing repeated requests and ultimately improving the rate of successful oneboxing.

To enable:

Set SiteSetting.cache_onebox_response_body to true

Add the domains you’re interesting in caching to SiteSetting. cache_onebox_response_body_domains e.g. example.com|example.org|example.net

Optionally set SiteSetting.cache_onebox_user_agent to a user agent string of your choice to use when making requests against domains in the above list.


The title of this pull request changed from “Oneboxer cache response body” to "FEATURE: Oneboxer cache response body

Can you extract this user agent string into a constant?

Wow, we were calling it with wrong arguments?! :astonished:

I think you can pass a block to any?

      SiteSetting.cache_onebox_response_body_domains.split("|").any? { |domain| uri.hostname.ends_with?(domain) }

Do we need .present?? Is the return value ever used?

    Discourse.redis.without_namespace.setex(key, 1.minutes.to_i, response)

The upshot was we were setting a non-zero value for 1 second, rather than setting a non-zero value for 1 day. Meaning we have have been doing a few more GETs than necessary, although with most sites defaulting to https this probably wasn’t a huge issue.