Dec 21, 2016 | laravel, laravel 5.3, queues

Update to queue workers in Laravel 5.3

This is a series of posts on New Features in Laravel 5.3.

!
Warning: This post is over a year old. I don't always update old posts with new information, so some of this information may be out of date.

Queues are one of those tools in Laravel that everyone knows is there, but very few people understand deeply. It's understandable--Laravel is often the first place folks have run into queues, and to be honest, they're not simple.

Thankfully, very little has changed on a user-facing front with regard to how queues work in Laravel 5.3.

Daemon as default

The biggest change is that the command you would've once used to "listen" for queue jobs:

php artisan queue:listen

... is no longer the default. Instead, running queue:work as a daemon is now the default:

php artisan queue:work

This was possible in the past by running php artisan queue:work --daemon, but now, you don't have to pass --daemon (instead, pass --once if you want to only work on a single job), and Laravel is recommending you use queue:work (daemon style) instead of queue:listen as your default.

What's the difference?

php artisan queue:listen listens to your queue and spins up the entire application every time it operates on a queue job. This is slower, but doesn't require rebooting the worker every time you push new code.

php artisan queue:work keeps the application running in between jobs, which makes it faster and lighter, but you'll need to restart the listener every time you push new code. The best way to do this is to run php artisan queue:restart on every deploy.

Supervisor

It's now recommended that you run a Supervisor process on your Linux hosts to watch your queue listener and restart it if it gets stopped. The docs now have a writeup on how to set up Supervisor correctly.

Essentially, you're going to install it using apt-get, configure it using the /etc/supervisor/conf.d file, and define that the queue worker should be restarted if it fails. You can even define how many queue workers you'd like to run at a given time.

Under the hood

The last final change is one that's largely transparent to us as developers, but the new queue infrastructure has a different model of how the primary worker handles control of each job. It's complicated, but it gives us the benefit of the worker having a lot more control over the behavior of long-running or misbehaving queue jobs. The new system also takes advantage of PHP 7.1's pcntl_async_signals when it's available.

As a reminder, you can control these long-running jobs using --timeout and retry_after; you can define that a queue worker process will kill a child process if it takes longer than a given amount of time using --timeout:

php artisan queue:work --timeout=90

Note that you can use this timeout in combination with retry_after, which is a setting in your queue configuration file. retry_after defines how long the worker should wait before assuming that a job has failed and needs to be re-added to the queue for a second try. As the docs note, make sure that your retry_after is at least a few seconds longer than your timeout so you don't get an overlap spinning up multiple copies of the same job.

Conclude

That's it for now! It's pretty simple and light stuff, but I think it makes the entire setup a little bit cleaner and more predictable. Good stuff.

Comments? I'm @stauffermatt on Twitter

Tags: laravel • laravel 5.3 • queues

This is part of a series of posts on New Features in Laravel 5.3:

Introducing Laravel Echo: An In-Depth Walk-Through

Jun 16, 2016 | laravel, laravel 5.3, echo, websockets
The new $loop variable in Laravel 5.3

Jun 27, 2016 | laravel, laravel 5.3
Customizing additional parameters in FirstOrCreate in Laravel 5.3

Jun 29, 2016 | laravel, laravel 5.3, eloquent
The new cache() global helper in Laravel 5.3

Jul 6, 2016 | laravel, laravel 5.3
Advanced operations with Collection::where in Laravel 5.3

Jul 8, 2016 | laravel, laravel 5.3
New JSON-column where() and update() syntax in Laravel 5.3

Jul 8, 2016 | laravel, laravel 5.3, eloquent
Image dimension validation rules in Laravel 5.3

Jul 25, 2016 | laravel, laravel 5.3
Customizing pagination templates in Laravel 5.3

Jul 26, 2016 | laravel, laravel 5.3
Routing changes in Laravel 5.3

Jul 27, 2016 | laravel, laravel 5.3, routing
5.3 feature announcement notes from Laracon

Jul 27, 2016 | laravel, laravel 5.3, laravel scout, laravel passport, mailable
Introducing Laravel Scout

Jul 29, 2016 | laravel, laravel 5.3, laravel scout
Introducing Laravel Passport

Jul 30, 2016 | laravel, laravel 5.3, laravel passport, oauth
Introducing Mailables in Laravel 5.3

Aug 5, 2016 | laravel, laravel 5.3, mail, laravel mailables
Directory structure changes in Laravel 5.3

Aug 8, 2016 | laravel, laravel 5.3
The new Notification system in Laravel 5.3

Oct 19, 2016 | laravel, laravel 5.3, laravel notifications, notifications
Using Vue in Laravel 5.3, with the Vue bootstrap and sample component

Dec 21, 2016 | laravel, laravel 5.3, vuejs, vueify, authorization
Update to queue workers in Laravel 5.3

Dec 21, 2016 | laravel, laravel 5.3, queues
Defining console commands via closure in Laravel 5.3

Jan 30, 2017 | laravel, laravel 5.3, artisan