FIX: Remove superfluous topic allowed users on group invite (PR #14656)

It does seems slightly faster, so will go with it:

with topic_allowed_users.topic_id = :topic_id

Nested Loop  (cost=4.38..26.19 rows=10 width=4) (actual time=0.008..0.008 rows=0 loops=1)
   ->  Nested Loop  (cost=0.15..11.31 rows=1 width=8) (actual time=0.007..0.008 rows=0 loops=1)
         ->  Seq Scan on group_users  (cost=0.00..1.12 rows=1 width=8) (actual time=0.007..0.007 rows=0 loops=1)
               Filter: (group_id = 1)
               Rows Removed by Filter: 10
         ->  Index Only Scan using index_topic_allowed_users_on_user_id_and_topic_id on topic_allowed_users  (cost=0.15..8.17 rows=1 width=4) (never executed)
               Index Cond: ((user_id = group_users.user_id) AND (topic_id = 73590))
               Filter: (user_id <> 11898)
               Heap Fetches: 0
   ->  Bitmap Heap Scan on topic_allowed_groups  (cost=4.23..14.79 rows=10 width=4) (never executed)
         Recheck Cond: (group_id = 1)
         ->  Bitmap Index Scan on index_topic_allowed_groups_on_group_id_and_topic_id  (cost=0.00..4.23 rows=10 width=0) (never executed)
               Index Cond: (group_id = 1)
 Planning Time: 0.172 ms
 Execution Time: 0.025 ms
(15 rows)

with topic_allowed_groups.topic_id = :topic_id

Nested Loop  (cost=4.36..23.75 rows=7 width=4) (actual time=0.005..0.006 rows=0 loops=1)
   ->  Nested Loop  (cost=0.15..9.31 rows=1 width=4) (actual time=0.005..0.005 rows=0 loops=1)
         ->  Seq Scan on group_users  (cost=0.00..1.12 rows=1 width=8) (actual time=0.005..0.005 rows=0 loops=1)
               Filter: (group_id = 1)
               Rows Removed by Filter: 10
         ->  Index Only Scan using index_topic_allowed_groups_on_topic_id_and_group_id on topic_allowed_groups  (cost=0.15..8.17 rows=1 width=4) (never executed)
               Index Cond: ((topic_id = 73592) AND (group_id = 1))
               Heap Fetches: 0
   ->  Bitmap Heap Scan on topic_allowed_users  (cost=4.21..14.37 rows=7 width=4) (never executed)
         Recheck Cond: (user_id = group_users.user_id)
         Filter: (user_id <> 11906)
         ->  Bitmap Index Scan on index_topic_allowed_users_on_user_id_and_topic_id  (cost=0.00..4.21 rows=7 width=0) (never executed)
               Index Cond: (user_id = group_users.user_id)
 Planning Time: 0.420 ms
 Execution Time: 0.036 ms
(15 rows)