Question: How to achieve sequential processing in Rails?

Question

How to achieve sequential processing in Rails?

Answers 1
Added at 2016-11-23 13:11
Tags
Question

I have a task which has to be performed in sequential manner. It's mosty a database writing.

The problem is that I have hard time to achieve it in Rails 5.

I'm implementing it in an ActiveJob task. First I've tried mutex:

class EventLoadJob < ApplicationJob
  MUTEX = Mutex.new

  rescue_from(StandardError) do |exception|
    puts exception.to_s
  end

  def perform
    MUTEX.synchronize
      SomeModel.new.save!
    end
  end
end

But this causes deadlock. Let assume: one thread is inside critical section and other wait at critical section entry. So the first thread block on database query. It wait on the query forever. I suppose there iss problem with ActiveRecord thread/connection - it's somehow blocked. But why?

I've tried the same with locking on file:

class EventLoadJob < ApplicationJob
  rescue_from(StandardError) do |exception|
    puts exception.to_s
  end

  def perform
    File.open(Rails.root.join('tmp', 'event_mutex'), File::RDWR|File::CREAT, 0644) do |file|
      file.flock(File::LOCK_EX)
      SomeModel.new.save!
      file.flock(File::LOCK_UN)
    end
  end
end

And I got the same strange behavior. I've tried the same in Controller instead of ActiveJob. The same: deadlock at ActiveRecord query.

Do you guys know a way to perform a task in sequential manner in Rails app? A proper way to create critical section?

Maybe there is ActiveJob queue adapter which is single threaded, without context switching, etc? I don't want to add redis or other stuff to my simple app.

Answers to

How to achieve sequential processing in Rails?

nr: #1 dodano: 2016-11-23 16:11

The only way you're likely to achieve this while still using ActiveJob (or any async job framework) is to break the job into multiple separate jobs, where the first piece of work enqueues the next piece of work as another job. Or you will have to store the state of progress through the job in some other data store, so each job can check if previous jobs have completed.

The mutex isn't working because the jobs could be run in different processes/threads or at completely different times depending on your resources.

Source Show
◀ Wstecz