1

I have a Rails 5 application with Resque as the background job backend. A typical job does quite a few things, mainly it collects stuff from HTTP services, processes and aggregates some of the downloaded data and stores it in the local database via ActiveRecord.

The question is, where should this be written?

If it is all in the job code, that code will be long, very procedural, and my code quality tool (Rubocop) will tell me it's too many lines of code and too complex in several different ways. But is that really a problem in this case? Or is it kind of an inherent feature of a job? I feel it probably shouldn't be.

The other option would be to put some of it in the models. Some of my models could know how and from where they can be downloaded, how the raw data should be processed, etc., and then they could just save themselves to AR. This way the job would mostly just orchestrate collection of the data, the actual stuff would be in the AR models. But does it belong there? The Rails way is thin controllers, but do jobs act as controllers in this regard? Doing it this way would make the job much harder to read for example.

1

Since these background jobs have lots of coordination to do that doesn't fit into traditional ActiveRecord model classes, you can break it down into smaller and smaller chunks of related logic until you get manageable chunks of code.

An example of breaking this down would be:

  • Job: The high level thing you are trying to accomplish
  • Tasks: Smaller chunks of work that still require coordination, but do so in a more focused manner
  • Services: Classes that make web service calls, etc
  • Entity classes: Your ActiveRecord classes

An example of breaking this down would be this (naively) implemented order processing background job for an e-commerce application.

The job (OrderProcessorJob) gets the pending orders, submits payment, ships the inventory and completes the order, saving it to the database. It involves web service calls, database calls, and a task (ProcessOrderTask) which has the knowledge of processing a single order.

The OrderProcessorJob class:

class OrderProcessorJob
  def perform
    pending_orders.each do |pending_order|
      order_to_process = Order.find pending_order.order_id

      begin
        if order_to_process
          process_order_task.process order_to_process pending_order
        else
          # Log that we got an order we couldn't process
        end
      rescue Exception => e1
        # Log error
      end
    end
  rescue Exception => e2
    # Log error, mark this job as "failed"
  end

private

  def order_service
    @order_service ||= OrderService.new
  end

  def process_order_task
    @process_order_task ||= ProcessOrderTask.new
  end

  def pending_orders
    order_service.pending_orders
  end

end

The OrderProcessorJob has the following responsibilities:

  1. Get pending orders from a web service

  2. Fetch each Order from the database and pass the pending order, and the order to process to the ProcessOrderTask

  3. High level error handling ensuring the background job doesn't crash, and it can continue processing orders in case of problems with individual orders

The ProcessOrderTask class:

class ProcessOrderTask
  def process(order_to_process, pending_order)
    billing_service.confirm_payment! order_to_process
    inventory_service.ship_order! order_to_process
    order_to_process.complete!
    order_to_process.save!
  rescue BillingException => billing_error
    # Log error, mark order as failed due to billing reasons
  rescue InventoryException => inventory_error
    # Log error, mark order as failed due to inventory issues
  end

private

  def billing_service
    @billing_service ||= BillingService.new
  end

  def inventory_service
    @inventory_service ||= InventoryService.new
  end

end

The responsibilities of the ProcessOrderTask class include:

  1. Calling the billing web service to ensure payment is confirmed
  2. Calling the inventory service to update the inventory to reflect that the order is complete
  3. Mark the order as completed and save it to the database
  4. Error handling in case of billing or inventory problems

The big advantage of this approach is you put most of the logic in other classes, which could be reused outside the scope of a background job. When each class has a more focused responsibility, you make it easier to test, because there is less code to test, and the numbers of dependencies for each class are reduced.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.