In this guide we'll be using Circus to start and persist python services. Circus can ensure that the process restarts if killed, or that a requisite number of worker threads run and even provide hooks and other advanced features.

Installation

Let's install Circus, for most platforms this should be as simple as installing through Pip.

pip3 install circus

Now, I'm going to adopt the Nginx & Uwsgi approach here of using an 'available folder and symlinking things on into 'enabled folder, so for us it'll be scripts-available and scripts-enabled. This enables you to easily turn services on or off, whilst maintaining the config.

mkdir -p /etc/circus/scripts-available /etc/circus/scripts-enabled /var/log/circus

Configure circus

Here's the /etc/circus/circus.ini

[circus]
check_delay = 5
endpoint = tcp://127.0.0.1:5555
pubsub_endpoint = tcp://127.0.0.1:5556
include = scripts-enabled/*.ini
umask = 002

Daemonise circus

Here's the /etc/systemd/system/circus.service

[Unit]
Description=Circus process manager
After=syslog.target network.target nss-lookup.target

[Service]
Type=simple
ExecReload=/usr/local/bin/circusctl reload
ExecStart=/usr/local/bin/circusd /etc/circus/circus.ini
Restart=always
RestartSec=5

[Install]
WantedBy=default.target

Create a service

Here's the /etc/circus/scripts-available/widgets.ini

[watcher:widgets_controller]
working_dir = /srv/widgets/
uid = 1000
gid = 1001
cmd = venv-bottle/bin/python3
args = -u  controller.py $(circus.wid) $(CIRCUS.ENV.VAR)
warmup_delay = 0
numprocesses = 1

# will push in test.log the stream every 300 ms
stdout_stream.class = FileStream
stdout_stream.filename =  /var/log/circus/widgets_controller.log
stderr__stream.class = FileStream
stderr_stream.filename =  /var/log/circus/widgets_controller.err

# optionally rotate the log file when it reaches 1 gb
# and save 5 copied of rotated files
stdout_stream.max_bytes = 1073741824
stdout_stream.backup_count = 5

Gotchas The paths will need to be specified as absolute for the logs, and unless you specify working_dir for the watcher, so will the pythonscript (and/or virtual environment), and of course the application won't be pwd in its own path, hence it's strongly recommended to specify working_dir.

Here's what you should have:

• /
• etc
  • circus
    • circus.ini
    • scripts-enabled
    • scripts-available
      • widgets.ini

Activate your service

Now to enable widgets:

root@linux:~#: ln -s /etc/circus/scripts-available/widgets.ini /etc/circus/scripts-enabled/widgets.ini
root@linux:~# circusctl start widgets_controller
root@linux:~# circusctl status
widgets_controller: active

And your service should be running. Any error information or output will be in /var/log/circus/widgets_controller.*