@@ -486,23 +486,30 @@ Deploying to Production
486486
487487On production, there are a few important things to think about:
488488
489- **Use Supervisor to keep your worker(s) running **
489+ **Use a Process Manager like Supervisor or systemd to keep your worker(s) running **
490490 You'll want one or more "workers" running at all times. To do that, use a
491- process control system like :ref: `Supervisor <messenger-supervisor >`.
491+ process control system like :ref: `Supervisor <messenger-supervisor >`
492+ or :ref: `systemd <messenger-systemd >`.
492493
493494**Don't Let Workers Run Forever **
494495 Some services (like Doctrine's ``EntityManager ``) will consume more memory
495496 over time. So, instead of allowing your worker to run forever, use a flag
496497 like ``messenger:consume --limit=10 `` to tell your worker to only handle 10
497- messages before exiting (then Supervisor will create a new process). There
498+ messages before exiting (then the process manager will create a new process). There
498499 are also other options like ``--memory-limit=128M `` and ``--time-limit=3600 ``.
499500
501+ **Stopping Workers That Encounter Errors **
502+ If a worker dependency like your database server is down, or timeout is reached,
503+ you can try to add :ref: `reconnect logic <middleware-doctrine >`, or just quit
504+ the worker if it receives too many errors with the ``--failure-limit `` option of
505+ the ``messenger:consume `` command.
506+
500507**Restart Workers on Deploy **
501508 Each time you deploy, you'll need to restart all your worker processes so
502509 that they see the newly deployed code. To do this, run ``messenger:stop-workers ``
503510 on deployment. This will signal to each worker that it should finish the message
504- it's currently handling and should shut down gracefully. Then, Supervisor will create
505- new worker processes. The command uses the :ref: `app <cache-configuration-with-frameworkbundle >`
511+ it's currently handling and should shut down gracefully. Then, the process manager
512+ will create new worker processes. The command uses the :ref: `app <cache-configuration-with-frameworkbundle >`
506513 cache internally - so make sure this is configured to use an adapter you like.
507514
508515**Use the Same Cache Between Deploys **
@@ -665,11 +672,25 @@ times:
665672 startsecs =0
666673 autostart =true
667674 autorestart =true
675+ startretries =10
668676 process_name =%(program_name)s_%(process_num)02d
669677
670678 Change the ``async `` argument to use the name of your transport (or transports)
671679and ``user `` to the Unix user on your server.
672680
681+ .. caution ::
682+
683+ During a deployment, something might be unavailable (e.g. the
684+ database) causing the consumer to fail to start. In this situation,
685+ Supervisor will try ``startretries `` number of times to restart the
686+ command. Make sure to change this setting to avoid getting the command
687+ in a FATAL state, which will never restart again.
688+
689+ Each restart, Supervisor increases the delay by 1 second. For instance, if
690+ the value is ``10 ``, it will wait 1 sec, 2 sec, 3 sec, etc. This gives the
691+ service a total of 55 seconds to become available again. Increase the
692+ ``startretries `` setting to cover the maximum expected downtime.
693+
673694If you use the Redis Transport, note that each worker needs a unique consumer
674695name to avoid the same message being handled by multiple workers. One way to
675696achieve this is to set an environment variable in the Supervisor configuration
@@ -692,7 +713,7 @@ Next, tell Supervisor to read your config and start your workers:
692713 See the `Supervisor docs `_ for more details.
693714
694715Graceful Shutdown
695- ~~~~~~~~~~~~~~~~~
716+ .................
696717
697718If you install the `PCNTL `_ PHP extension in your project, workers will handle
698719the ``SIGTERM `` POSIX signal to finish processing their current message before
@@ -708,6 +729,88 @@ of the desired grace period in seconds) in order to perform a graceful shutdown:
708729 [program:x]
709730 stopwaitsecs =20
710731
732+ .. _messenger-systemd :
733+
734+ Systemd Configuration
735+ ~~~~~~~~~~~~~~~~~~~~~
736+
737+ While Supervisor is a great tool, it has the disadvantage that you need system
738+ access to run it. Systemd has become the standard on most Linux distributions,
739+ and has a good alternative called *user services *.
740+
741+ Systemd user service configuration files typically live in a ``~/.config/systemd/user ``
742+ directory. For example, you can create a new ``messenger-worker.service `` file. Or a
743+ ``messenger-worker@.service `` file if you want more instances running at the same time:
744+
745+ .. code-block :: ini
746+
747+ [Unit]
748+ Description =Symfony messenger-consume %i
749+
750+ [Service]
751+ ExecStart =php /path/to/your/app/bin/console messenger:consume async --time-limit =3600
752+ Restart =always
753+ RestartSec =30
754+
755+ [Install]
756+ WantedBy =default.target
757+
758+ Now, tell systemd to enable and start one worker:
759+
760+ .. code-block :: terminal
761+
762+ $ systemctl --user enable messenger-worker@1.service
763+ $ systemctl --user start messenger-worker@1.service
764+
765+ # to enable and start 20 workers
766+ $ systemctl --user enable messenger-worker@{1..20}.service
767+ $ systemctl --user start messenger-worker@{1..20}.service
768+
769+ If you change your service config file, you need to reload the daemon:
770+
771+ .. code-block :: terminal
772+
773+ $ systemctl --user daemon-reload
774+
775+ To restart all your consumers:
776+
777+ .. code-block :: terminal
778+
779+ $ systemctl --user restart messenger-consume@*.service
780+
781+ The systemd user instance is only started after the first login of the
782+ particular user. Consumer often need to start on system boot instead.
783+ Enable lingering on the user to activate that behavior:
784+
785+ .. code-block :: terminal
786+
787+ $ loginctl enable-linger <your-username>
788+
789+ Logs are managed by journald and can be worked with using the journalctl
790+ command:
791+
792+ .. code-block :: terminal
793+
794+ # follow logs of consumer nr 11
795+ $ journalctl -f --user-unit messenger-consume@11.service
796+
797+ # follow logs of all consumers
798+ $ journalctl -f --user-unit messenger-consume@*
799+
800+ # follow all logs from your user services
801+ $ journalctl -f _UID=$UID
802+
803+ See the `systemd docs `_ for more details.
804+
805+ .. note ::
806+
807+ You either need elevated privileges for the ``journalctl `` command, or add
808+ your user to the systemd-journal group:
809+
810+ .. code-block :: terminal
811+
812+ $ sudo usermod -a -G systemd-journal <your-username>
813+
711814 Stateless Worker
712815~~~~~~~~~~~~~~~~
713816
@@ -2087,6 +2190,8 @@ middleware and *only* include your own:
20872190 If a middleware service is abstract, a different instance of the service will
20882191 be created per bus.
20892192
2193+ .. _middleware-doctrine :
2194+
20902195Middleware for Doctrine
20912196~~~~~~~~~~~~~~~~~~~~~~~
20922197
@@ -2270,6 +2375,7 @@ Learn more
22702375.. _`streams` : https://redis.io/topics/streams-intro
22712376.. _`Supervisor docs` : http://supervisord.org/
22722377.. _`PCNTL` : https://www.php.net/manual/book.pcntl.php
2378+ .. _`systemd docs` : https://www.freedesktop.org/wiki/Software/systemd/
22732379.. _`SymfonyCasts' message serializer tutorial` : https://symfonycasts.com/screencast/messenger/transport-serializer
22742380.. _`Long polling` : https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-short-and-long-polling.html
22752381.. _`Visibility Timeout` : https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-visibility-timeout.html
0 commit comments