My first attempt to solve this problem was similar to what you suggest. It would build a set of objects before any tests were run. I abandoned this effort because, unfortunately, a significant number of the tests assume they are starting with a blank slate.
As far as deciding between prefabricate vs let (vs let!), this is how I think about it:
If the choice was just between let vs let! and performance wasn’t a concern, I would always choose let!. It’s easier to predict what a test using let! will do since you don’t have to figure out what order the let bodies will execute in or if they will execute at all and this matters when there are side effects.
In this light, let is a cheaper let!. It’s almost observationally the same, except in the presence of side effects or divergence.
However, prefabricate is also just a cheaper let! and is also almost observationally equivalent. Moreover, prefabricate is cheaper or equal to let as long as the thing in question is actually used.
So, my answer to “which should I use?” is “default to prefabricate for active record objects”.