FIX: prefix should precede folder path (follow-up on 10fbb07e)

FIX: prefix should precede folder path (follow-up on 10fbb07e)

diff --git a/lib/tasks/uploads.rake b/lib/tasks/uploads.rake
index c7b72e2..9f850ab 100644
--- a/lib/tasks/uploads.rake
+++ b/lib/tasks/uploads.rake
@@ -275,7 +275,7 @@ def migrate_to_s3
 
   s3_objects = []
   prefix = Rails.configuration.multisite ? "#{db}/original/" : "original/"
-  options = { bucket: bucket, prefix: prefix + folder }
+  options = { bucket: bucket, prefix: folder + prefix }
 
   loop do
     response = s3.list_objects_v2(options)

GitHub sha: ff8f9dc1

The listing issue is fixed for both kinds of bucket paths :arrow_down:

Migrating uploads to S3 for 'default'...
Uploading files to S3...
 - Listing local files => 64 files
 - Listing S3 files. => 64 files
 - Syncing files to S3

@ZogStriP could I have a sanity check on this output? Especially the remap and replace destinations?

without folder path

DRY_RUN=1
DISCOURSE_S3_REGION=“us-east-1”
DISCOURSE_S3_BUCKET=“rishabhmultisite3”
DISCOURSE_S3_CDN_URL=“https://discourse-cdn-aws1.com/pocketgems
rake uploads:migrate_to_s3

****************************** DRY RUN ****************************** Migrating uploads to S3 for ‘default’… Uploading files to S3…

  • Listing local files => 64 files
  • Listing S3 files. => 0 files
  • Syncing files to S3

uploads/default/original/1X/7dc054e635cfb985d8d790584fd14996fed12eae.jpeg => original/1X/7dc054e635cfb985d8d790584fd14996fed12eae.jpeg

uploads/default/original/1X/29e24a112e295db84e91826862da2fccb96d0df3.png => original/1X/29e24a112e295db84e91826862da2fccb96d0df3.png

Updating the URLs in the database… REPLACING ‘/uploads/default/original/(\dX/(?:[a-f0-9]/)[a-f0-9]{40}[a-z0-9.])’ WITH //rishabhmultisite3.s3.dualstack.us-east-1.amazonaws.com/original/\1

REMAPPING http://localhost//rishabhmultisite3.s3.dualstack.us-east-1.amazonaws.com TO https://discourse-cdn-aws1.com/pocketgems Done!

with folder path

DISCOURSE_S3_REGION=“us-east-1”
DISCOURSE_S3_BUCKET=“rishabhmultisite3/cloud”
DISCOURSE_S3_CDN_URL=“https://discourse-cdn-aws1.com/pocketgems
rake uploads:migrate_to_s3

****************************** DRY RUN ****************************** Migrating uploads to S3 for ‘default’… Uploading files to S3…

  • Listing local files => 64 files
  • Listing S3 files. => 0 files
  • Syncing files to S3

uploads/default/original/1X/7dc054e635cfb985d8d790584fd14996fed12eae.jpeg => cloud/original/1X/7dc054e635cfb985d8d790584fd14996fed12eae.jpeg

uploads/default/original/1X/21e6700c21f70f299af6c4e74048c9e9ccf32983.png => cloud/original/1X/21e6700c21f70f299af6c4e74048c9e9ccf32983.png

Updating the URLs in the database… REPLACING 1/uploads/default/original/(\dX/(?:[a-f0-9]/)[a-f0-9]{40}[a-z0-9.]) WITH //rishabhmultisite3.s3.dualstack.us-east-1.amazonaws.com/cloud/original/\1

REMAPPING http://localhost//rishabhmultisite3.s3.dualstack.us-east-1.amazonaws.com/cloud TO https://discourse-cdn-aws1.com/pocketgems Done!

Looks good from a cursory look. Have you tested multisite?

I did so by manually setting prefix to use "#{db}/original/" on my test install.

without folder path using a multisite prefix
uploads/default/original/1X/b92f9ef1eb09102e8c48fb0c1d872c82c9fcbdff.png => default/original/1X/b92f9ef1eb09102e8c48fb0c1d872c82c9fcbdff.png
uploads/default/original/1X/21e6700c21f70f299af6c4e74048c9e9ccf32983.png => default/original/1X/21e6700c21f70f299af6c4e74048c9e9ccf32983.png

Updating the URLs in the database...
REPLACING '/uploads/default/original/(\dX/(?:[a-f0-9]/)*[a-f0-9]{40}[a-z0-9\.]*)' WITH '//rishabhmultisite10.s3.dualstack.us-east-1.amazonaws.com/default/original/\1'
REMAPPING 'http://localhost//rishabhmultisite10.s3.dualstack.us-east-1.amazonaws.com' TO 'https://discourse-cdn-aws1.com/pocketgems'
with folder path using a multisite prefix
uploads/default/original/1X/b92f9ef1eb09102e8c48fb0c1d872c82c9fcbdff.png => cloud-file-uploads/default/original/1X/b92f9ef1eb09102e8c48fb0c1d872c82c9fcbdff.png
uploads/default/original/1X/21e6700c21f70f299af6c4e74048c9e9ccf32983.png => cloud-file-uploads/default/original/1X/21e6700c21f70f299af6c4e74048c9e9ccf32983.png

Updating the URLs in the database...
REPLACING '/uploads/default/original/(\dX/(?:[a-f0-9]/)*[a-f0-9]{40}[a-z0-9\.]*)' WITH '//rishabhmultisite10.s3.dualstack.us-east-1.amazonaws.com/cloud-file-uploads/default/original/\1'
REMAPPING 'http://localhost//rishabhmultisite10.s3.dualstack.us-east-1.amazonaws.com/cloud-file-uploads' TO 'https://discourse-cdn-aws1.com/pocketgems'