DEV: Add --profile and --fail-fast when running plugin specs, tweak parallel test balancing

DEV: Add --profile and --fail-fast when running plugin specs, tweak parallel test balancing

diff --git a/lib/tasks/docker.rake b/lib/tasks/docker.rake
index d4f09ff..95d81aa 100644
--- a/lib/tasks/docker.rake
+++ b/lib/tasks/docker.rake
@@ -139,7 +139,8 @@ task 'docker:test' do
             spec_partials = Dir["spec/**/*_spec.rb"].sort.in_groups(total, false)
             # quick and dirty load balancing
             if (spec_partials.count > 3)
-              spec_partials[0].concat(spec_partials[total - 1].shift(40))
+              spec_partials[0].concat(spec_partials[total - 1].shift(30))
+              spec_partials[1].concat(spec_partials[total - 2].shift(30))
             end
 
             params << spec_partials[subset].join(' ')
@@ -154,7 +155,7 @@ task 'docker:test' do
           if ENV["SINGLE_PLUGIN"]
             @good &&= run_or_fail("bundle exec rake plugin:spec['#{ENV["SINGLE_PLUGIN"]}']")
           else
-            @good &&= run_or_fail("bundle exec rake plugin:spec")
+            @good &&= run_or_fail("RSPEC_FAILFAST=1 bundle exec rake plugin:spec")
           end
         end
         puts "travis_fold:end:ruby_tests" if ENV["TRAVIS"]
diff --git a/lib/tasks/plugin.rake b/lib/tasks/plugin.rake
index c775250..afd36da 100644
--- a/lib/tasks/plugin.rake
+++ b/lib/tasks/plugin.rake
@@ -86,10 +86,11 @@ end
 desc 'run plugin specs'
 task 'plugin:spec', :plugin do |t, args|
   args.with_defaults(plugin: "*")
+  params = ENV['RSPEC_FAILFAST'] ? '--profile --fail-fast' : '--profile'
   ruby = `which ruby`.strip
   files = Dir.glob("./plugins/#{args[:plugin]}/spec/**/*_spec.rb")
   if files.length > 0
-    sh "LOAD_PLUGINS=1 #{ruby} -S rspec #{files.join(' ')}"
+    sh "LOAD_PLUGINS=1 #{ruby} -S rspec #{files.join(' ')} #{params}"
   else
     abort "No specs found."
   end

GitHub sha: a5b804fe

When running a large test suite, I feel that it is better for all the failure specs to be logged so that can fix the affected specs instead of having to run all the specs again locally to see which specs are failing.

1 Like

I think it’s fine. The behavior was only change for plugin specs.

It is a tradeoff, speed vs. full report on all failing tests, that’s why it’s limited to the docker build. And as Regis notes, this just brings plugin specs in line with core specs.