PERF: Memoize allowed user fields more efficiently (PR #8968)

Previously we were caching by user_id, but the there are only two possible outcomes. Therefore we only need to cache two values.

This removes another N+1 query when serializing multiple user cards.


You’ve signed the CLA, davidtaylorhq. Thank you! This pull request is ready for review.