Update to queue workers in Laravel 5.3
Posted on December 21, 2016
Warning: This post is over a year old. I don't always update old posts with new information, so some of this information may be out of date.
Queues are one of those tools in Laravel that everyone knows is there, but very few people understand deeply. It's understandable--Laravel is often the first place folks have run into queues, and to be honest, they're not simple.
Thankfully, very little has changed on a user-facing front with regard to how queues work in Laravel 5.3.
Daemon as default
The biggest change is that the command you would've once used to "listen" for queue jobs:
php artisan queue:listen
... is no longer the default. Instead, running
queue:work as a daemon is now the default:
php artisan queue:work
This was possible in the past by running
php artisan queue:work --daemon, but now, you don't have to pass
--daemon (instead, pass
--once if you want to only work on a single job), and Laravel is recommending you use
queue:work (daemon style) instead of
queue:listen as your default.
What's the difference?
php artisan queue:listen listens to your queue and spins up the entire application every time it operates on a queue job. This is slower, but doesn't require rebooting the worker every time you push new code.
php artisan queue:work keeps the application running in between jobs, which makes it faster and lighter, but you'll need to restart the listener every time you push new code. The best way to do this is to run
php artisan queue:restart on every deploy.
It's now recommended that you run a Supervisor process on your Linux hosts to watch your queue listener and restart it if it gets stopped. The docs now have a writeup on how to set up Supervisor correctly.
Essentially, you're going to install it using
apt-get, configure it using the
/etc/supervisor/conf.d file, and define that the queue worker should be restarted if it fails. You can even define how many queue workers you'd like to run at a given time.
Under the hood
The last final change is one that's largely transparent to us as developers, but the new queue infrastructure has a different model of how the primary worker handles control of each job. It's complicated, but it gives us the benefit of the worker having a lot more control over the behavior of long-running or misbehaving queue jobs. The new system also takes advantage of PHP 7.1's pcntl_async_signals when it's available.
As a reminder, you can control these long-running jobs using
retry_after; you can define that a queue worker process will kill a child process if it takes longer than a given amount of time using
php artisan queue:work --timeout=90
Note that you can use this
timeout in combination with
retry_after, which is a setting in your queue configuration file.
retry_after defines how long the worker should wait before assuming that a job has failed and needs to be re-added to the queue for a second try. As the docs note, make sure that your
retry_after is at least a few seconds longer than your
timeout so you don't get an overlap spinning up multiple copies of the same job.
That's it for now! It's pretty simple and light stuff, but I think it makes the entire setup a little bit cleaner and more predictable. Good stuff.
Comments? I'm @stauffermatt on Twitter
Tags: laravel • laravel 5.3 • queues
(This is part of a series of posts on New Features in Laravel 5.3. )
Jun 16, 2016 | laravel, laravel 5.3, echo, websocketsLaravel Echo is a tool that makes it easy for you to bring the power of WebSockets to your Laravel applications. It simplifie...
Jun 27, 2016 | laravel, laravel 5.3Let's take a look at another new feature that's coming in Laravel 5.3. What are Blade directives? Laravel's Blade templating ...
Jun 29, 2016 | laravel, laravel 5.3, eloquentMore Laravel 5.3 goodies! This time, it's an update to the Eloquent firstOrCreate method. If you've never used it before, you...
Jul 6, 2016 | laravel, laravel 5.3Time for another new feature in Laravel 5.3! Shortcut global helpers in Laravel As I was writing my book I noticed a pattern ...
Jul 8, 2016 | laravel, laravel 5.3, eloquentWhile Laravel has had the ability to cast your data to and from JSON since version 5.0, it was previously just a convenience&...
Jul 8, 2016 | laravel, laravel 5.3If you want to filter a Laravel collection to only those records which meet particular criteria, you're most likely going to ...
Jul 25, 2016 | laravel, laravel 5.3In Laravel 5.3, we have a new validation option: image dimensions for image uploads. The validation rule is called dimensions...
Jul 26, 2016 | laravel, laravel 5.3Laravel's pagination library is brilliant, because pagination is a common task that is a surprising amount of work to impleme...
Jul 27, 2016 | laravel, laravel 5.3, laravel scout, laravel passport, mailableTHIS POST IS STILL IN ACTIVE PROGRESS Once the videos are released I'll be able to update this post with more info, since I w...
Jul 27, 2016 | laravel, laravel 5.3, routingThe last few versions of Laravel have showed the way routing works shifting around a bit. This is usually a sign that we're f...
Jul 29, 2016 | laravel, laravel 5.3, laravel scoutSearch tools ElasticSearch and Algolia have gained a lot of popularity in the Laravel community in the last few years as powe...
Jul 30, 2016 | laravel, laravel 5.3, laravel passport, oauthAPI authentication can be tricky. OAuth 2 is the reigning ruler of the various standards that you might consider, but it's co...
Aug 5, 2016 | laravel, laravel 5.3, mail, laravel mailablesFor the longest time, sending mail in Laravel has felt clumsy compared to the relatively light APIs of most other Laravel fea...
Aug 8, 2016 | laravel, laravel 5.3We've already covered the new routes directory and the changes that will make to the directory structure. But there's one oth...
Oct 19, 2016 | laravel, laravel 5.3, laravel notifications, notificationsIn a previous blog post I've covered Laravel's new Mailable feature, which introduces some important changes to how we send m...
Dec 21, 2016 | laravel, laravel 5.3, queuesQueues are one of those tools in Laravel that everyone knows is there, but very few people understand deeply. It's understand...
Dec 21, 2016 | laravel, laravel 5.3, vuejs, vueify, authorizationIn Laravel 5.3, it's easier than ever to write and use Vue components out of the box. This means 5.3 has a somewhat more opin...
Jan 30, 2017 | laravel, laravel 5.3, artisanBefore Laravel 5.3, defining an Artisan console command—something like php artisan sync:dates—required you to create a ne...