Queue how to preserve requests?

Question

I'm working with Laravel queues lately. I've all set up and running fine. What I see is that I perform a http request to my system, the controllers manage the request issuing a Model having request details as parameters. The model is the injected in the Job, then job+data(model) are serialized in my queue. (I'm using DB as driver to have better visibility of what's happening)

Here comes my doubt, I suppose that queue "pattern" has a purpose of a temporary location to store request to be performed and the cleaned up.

But what would be the approach to set it up in order to understand what's happened in the last X days to requests? (Should I store myself my Models at creation time in the corresponding DB table before injecting it in Jobs?)

I already know there's the chance to set up a failed_queue storage. I see there's only a management of failures (with Event)

But how about being sure something has been processed? Or maybe queue are planned to be like I accept requests and deliver (doesn't mind to store success?)

class MailQueueController extends Controller
{
    public function queue(Request $request)
    {
        $mailQueue = new MailQueue();

        $mailQueue->batch_id = 1;
        $mailQueue->email_id = 401;
        $mailQueue->email_address = '[email protected]';
        $mailQueue->json_data_input = '{"hi":"1"}';

        $mailQueue->created_at = Carbon::now()->toDateTimeString();

        /////////////////////////////////
        /// should I?
        /////////////////////////////////
        $mailQueue->save();

        //immediate
        dispatch(new ProcessEmailQueue($mailQueue));
    }
}

Answer 1

I found the problem. Yes I must save the Eloquent model (if using a eloquent model) injected in JOB.

Reading better the documentation says:

"If your queued job accepts an Eloquent model in its constructor, only the identifier for the model will be serialized onto the queue. When the job is actually handled, the queue system will automatically re-retrieve the full model instance from the database."

Then you must save it, otherwise it won't find the reference.

And also this answers to my question: how to trace requests. (Having them saved as Eloquent objects aka Db recordset).

Sorry for bothering. But maybe this will help somebody else.

Answer 2

The 'queuey' way to do this is to have a fan out from your initial message

incoming T-> worker
         |
         L-> reporting tasks

This allows you to attach extra processes into your flow at any point.

But for what your talking about logging from your worker is probably enough. Look at some of the big data style logging solutions like logstash