Hello, and thank you for this gem!
Description
At the moment, it looks like migrate_task wraps each task in a transaction. While this is desirable in many cases, it seems like tasks that contain large data backfills would benefit from not being wrapped in a transaction.
Assume we have a users table with a unique constraint on (foobar, column_one, column_two). And a task that looks like this:
namespace :migrations do
task :migrate_users => :environment do
User.where(...).find_each do |user|
begin
user.update_attributes(column_one: 'baz', column_two: 'qux')
rescue ActiveRecord::RecordNotUnique
# noop
end
end
end
end
That task is idempotent, and would (at least in theory) succeed on the first run. However, should the task need to be run again, it would actually fail due to the unique constraint on (foobar, column_one, column_two), despite our rescuing ActiveRecord::RecordNotUnique exceptions. This is because any database execution error will poison the current transaction the task is running is and result in an PG::InFailedSqlTransaction exception being thrown on any subsequent queries.
Expected Behavior
The idempotent task runs successfully multiple times.
Actual Behavior
The idempotent task runs successfully the first time, but fails on subsequent runs with an ActiveRecord::StatementInvalid: PG InFailedSqlTransaction exception after the first database execution error is raised/rescued and a subsequent query issued.
It doesn't seem like there's currently a way to bypass the task being wrapped in a transaction. Would it be beneficial to have a flag that disables the default behavior? Is that something you'd be open to supporting in this gem? I'd be happy to open a PR if so!
Hello, and thank you for this gem!
Description
At the moment, it looks like
migrate_taskwraps each task in a transaction. While this is desirable in many cases, it seems like tasks that contain large data backfills would benefit from not being wrapped in a transaction.Assume we have a
userstable with a unique constraint on(foobar, column_one, column_two). And a task that looks like this:That task is idempotent, and would (at least in theory) succeed on the first run. However, should the task need to be run again, it would actually fail due to the unique constraint on
(foobar, column_one, column_two), despite our rescuingActiveRecord::RecordNotUniqueexceptions. This is because any database execution error will poison the current transaction the task is running is and result in anPG::InFailedSqlTransactionexception being thrown on any subsequent queries.Expected Behavior
The idempotent task runs successfully multiple times.
Actual Behavior
The idempotent task runs successfully the first time, but fails on subsequent runs with an
ActiveRecord::StatementInvalid: PG InFailedSqlTransactionexception after the first database execution error is raised/rescued and a subsequent query issued.It doesn't seem like there's currently a way to bypass the task being wrapped in a transaction. Would it be beneficial to have a flag that disables the default behavior? Is that something you'd be open to supporting in this gem? I'd be happy to open a PR if so!