I am trying to make a single-threaded part multi-threaded. The initial single thread code works fine, but after I put it into a thread, the server hangs without any log information in the console.
Initial version:
data_array.each do |data|
service = SomeService.new(data)
data['answers'] = service.answer_data
end
Multi-threaded version:
threads = []
threads << Thread.new {
data_array.each do |data|
service = SomeService.new(data)
data['answers'] = service.answer_data
end
}
threads.each(&:join)
I tried binding.pry and get some more information:
Class called AggregationReport < ReportSomeService tried to run AggregationReport.new, it's stuck there forever.Please help me understand what is happening here, any thoughts/help would be much appreciated!
Ruby version: Ruby27x Rails: 5.2
I'm making some assumptions about why you're threading this, but I'd expect typical threading code to follow the following basic structure:
threads = data_array.map do |data|
Thread.new { SomeService.new(data).answer_data }
end
answers = threads.map(&:value)
For each piece of work you need done:
If your service needs a database connection (e.g. postgres, redis, etc) you'll need to manage this as well. For activerecord add ActiveRecord::Base.with_connection{ } inside the thread to allocate a db connection. There are similar approaches for Redis/etc. Then ensure you have sufficient connections for the # of threads you want to create (look at the pool: setting in config/database.yml).
Thread.new { ActiveRecord::Base.with_connection{ SomeService.new(data).answer_data } }
For this and other reasons, it isn't always a great idea to spawn N threads, so instead of this approach you may consider using a thread pool of X and ensure you have a db connection pool of at least X+1 connections. All of this can be complex. There are great gems that sit on top of ruby threading and make it easier to do this exact kind of thing. I'd strongly recommend checking out concurrent-ruby (bundled with Rails).