FIX: Catch UndefinedConversionError for inbound emails (#13000)

FIX: Catch UndefinedConversionError for inbound emails (#13000)

Some emails coming in via the mail receiver can still end up with bad encoding when trying to enqueue the job. This catches the last encoding issue and forces iso-8559-1 and encodes to UTF-8 to circumvent the issue.

diff --git a/app/controllers/admin/email_controller.rb b/app/controllers/admin/email_controller.rb
index 6edc510..249eee8 100644
--- a/app/controllers/admin/email_controller.rb
+++ b/app/controllers/admin/email_controller.rb
@@ -158,7 +158,7 @@ class Admin::EmailController < Admin::AdminController
 
     begin
       Jobs.enqueue(:process_email, mail: email_raw, retry_on_rate_limit: true, source: :handle_mail)
-    rescue JSON::GeneratorError => e
+    rescue JSON::GeneratorError, Encoding::UndefinedConversionError => e
       if retry_count == 0
         email_raw = email_raw.force_encoding('iso-8859-1').encode("UTF-8")
         retry_count += 1
diff --git a/spec/fixtures/emails/encoding_undefined_conversion.eml b/spec/fixtures/emails/encoding_undefined_conversion.eml
new file mode 100644
index 0000000..c5e3f6c
--- /dev/null
+++ b/spec/fixtures/emails/encoding_undefined_conversion.eml
@@ -0,0 +1,12 @@
+Return-Path: <discourse@bar.com>
+From: Foo Bar <discourse@bar.com>
+To: someone@else.com
+CC: team@bar.com, wat@bar.com, reply+d400310beeae61d785c2ac6a2aacb210@bar.com
+Subject: The more, the merrier
+Date: Fri, 15 Jan 2016 00:12:43 +0100
+Message-ID: <30@foo.bar.mail>
+Mime-Version: 1.0
+Content-Type: text/plain; charset=UTF-8
+Content-Transfer-Encoding: quoted-printable
+
+It is more fun with more people. 𝗜 𝗮𝗺 𝗮 𝘀𝗶𝗻𝗴𝗹𝗲 person.
diff --git a/spec/requests/admin/email_controller_spec.rb b/spec/requests/admin/email_controller_spec.rb
index 817c5c5..fa14683 100644
--- a/spec/requests/admin/email_controller_spec.rb
+++ b/spec/requests/admin/email_controller_spec.rb
@@ -229,6 +229,12 @@ describe Admin::EmailController do
       expect(response.status).to eq(200)
       expect(response.body).to eq("email has been received and is queued for processing")
     end
+
+    it "retries enqueueing with forced UTF-8 encoding when encountering Encoding::UndefinedConversionError" do
+      post "/admin/email/handle_mail.json", params: { email_encoded: Base64.strict_encode64(email('encoding_undefined_conversion')) }
+      expect(response.status).to eq(200)
+      expect(response.body).to eq("email has been received and is queued for processing")
+    end
   end
 
   describe '#rejected' do

GitHub sha: c187ede3

This commit appears in #13000 which was approved by lis2. It was merged by martin.