The sign-up form felt slow for one reason: it sent an email. The user waited while the server talked to the SMTP host, and whenever SMTP was having a slow day, the page hung along with it. Moving the send onto a queue solved that in a few lines.
What I did not realise was that I had traded one problem for another.
Failure with no sound
When the send runs inside the request, a failure is visible immediately. Once it moves onto a queue, the failure moves somewhere nobody is looking. As long as no worker is running, the work simply piles up in the jobs table, while the page cheerfully keeps reporting success.
I only found out when someone asked why their email had never arrived.
How I decide now
Before moving anything onto a queue, I ask two questions:
- Does the user need the result right now? If so, do not queue it.
- If this work fails quietly, who will find out, and how?
When the answer to the second question is "nobody", I am not ready to put that work on a queue.
For the sign-up email I ended up not using a queue at all. The send runs after the response has gone to the browser, and a failure is written to the log without failing the sign-up itself. The page stays fast, and if SMTP is having problems, there is a trail.
Queues are still worth having
For work that genuinely is long, processing a large file or assembling a report, a queue is still the right choice. There is only one condition: make sure something is running the worker, and make sure failures are visible. A queue without monitoring does not remove the problem, it only moves it somewhere darker.
