1. Introduction to supervisor
   Supervisor is a set of general-purpose process management programs developed in Python. It can turn an ordinary command-line process into a background daemon, monitor the process status, and automatically restart when it exits abnormally. It uses fork/exec to start these managed processes as child processes of the supervisor. In this way, it is only necessary to write the path of the executable file of the process to be managed in the supervisor configuration file. It also realizes that when the child process hangs up, the parent process can accurately obtain the information that the child process hangs up, and can choose whether to start and alarm by itself. Supervisor also provides a function to set a non-root user for supervisord or each child process, and this user can manage its corresponding process.

Two, supervisor installation
After configuring the yum source, you can install it directly

yum install epel-release #Install yum source
yum install -y supervisor

Modify the configuration file

$ /etc/supervisord.conf
[unix_http_server]
file=/var/run/supervisor/supervisor.sock   ; (the path to the socket file)
chmod=0700                 ; sockef file mode (default 0700)
[inet_http_server]         ; inet (TCP) server disabled by default
port=*:9001        ; (ip_address:port specifier, *:port for all iface)
username=admin              ; (default is no username (open server))
password=admin             ; (default is no password (open server))
[supervisord]
logfile=/var/log/supervisor/supervisord.log  ; (main log file;default $CWD/supervisord.log)
logfile_maxbytes=50MB       ; (max main logfile bytes b4 rotation;default 50MB)
logfile_backups=10          ; (num of main logfile rotation backups;default 10)
loglevel=info               ; (log level;default info; others: debug,warn,trace)
pidfile=/var/run/supervisord.pid ; (supervisord pidfile;default supervisord.pid)
nodaemon=false              ; (start in foreground if true;default false)
minfds=1024                 ; (min. avail startup file descriptors;default 1024)
minprocs=200                ; (min. avail process descriptors;default 200)
[rpcinterface:supervisor]
supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface
[supervisorctl]
serverurl=unix:///var/run/supervisor/supervisor.sock ; use a unix:// URL  for a unix socket
serverurl=http://127.0.0.1:9001 ; use an http:// url to specify an inet socket
username=admin              ; should be same as http_username if set
password=admin                ; should be same as http_password if set
[include]
files = supervisord.d/*.ini

start supervisor

systemctl enable supervisord && systemctl start supervisord

3. Use of supervisor
   Supervisor configuration file: /etc/supervisord.conf

Subprocess configuration file path: /etc/supervisord.d/*.ini

4. Configuration file description
   /etc/supervisord.conf

[unix_http_server]
file=/var/run/supervisor/supervisor.sock ; UNIX socket file, supervisorctl will use
chmod=0700 ;The mode of the socket file, the default is 0700
chown=root:root ;owner of the socket file, format: uid:gid
;HTTP server, providing web management interface
[inet_http_server]
port=*:9001 ;The IP and port running in the background of the web management, if open to the public network, you need to pay attention to security
username=admin ;Username for logging in to the management background
password=admin ;The password for logging in to the management background
 
[supervisord]
logfile=/var/log/supervisor/supervisord.log ; log file, the default is $CWD/supervisord.log
logfile_maxbytes=50MB ;The log file size will be rotated if it exceeds, the default is 50MB, if it is set to 0, it means unlimited size
logfile_backups=10 ; The default number of log file backups is 10, set to 0 means no backup
loglevel=info ;log level, default info, others: debug,warn,trace
pidfile=/var/run/supervisord.pid ;pid file
nodaemon=false ;Whether to start in the foreground, the default is false, that is, start as a daemon
minfds=1024 ;The minimum value of the file descriptor that can be opened, the default is 1024
minprocs=200 ; the minimum number of processes that can be opened, the default is 200
 
[supervisorctl]
serverurl=unix:///var/run/supervisor/supervisor.sock ; connect supervisord through UNIX socket, the path is consistent with the file in the unix_http_server part
serverurl=http://127.0.0.1:9001 ;Connect supervisord via HTTP
username=admin ; (default is no username (open server))
password=admin ; (default is no password (open server))s
 
;Include other configuration files
[include]
files = relative/directory/*.ini ; One or more configuration files ending with .ini can be specified

Subprocess configuration file description:

Write a configuration file for the child process (program) that needs to be managed, put it in the /etc/supervisor.d/ directory, and use . put together)

Such as arbitrarily defining an option group (/etc/supervisord.d/grafana.ini) related to the project name of the script:

;Item name
[program:grafana]
; script directory
directory=/data/grafana
; script execution command
command=/data/grafana/bin/grafana-server
;Whether to start at the same time when the supervisor starts, the default is True
autostart=true
;Automatically restart when the program exits, optional values: [unexpected,true,false], the default is unexpected, which means that the process will restart after accidental killing, false means never restart, true means automatic restart
autorestart=true
; The number of automatic retries on startup failure, the default is 3
startretries=3
; This option is how many seconds after the child process is started, if the state is running at this time, we consider the startup to be successful. The default value is 1
startsecs=1
; The user identity used to start the program
user = root
;stdout log file, you need to pay attention that it cannot start normally when the specified directory does not exist, so you need to manually create the directory (supervisord will automatically create log files)
stdout_logfile=/data/grafana/blog_stdout.log
;Redirect stderr to stdout, default false
redirect_stderr = true
;stdout log file size, default 50MB
stdout_logfile_maxbytes = 20MB
;stdout log file backup count
stdout_logfile_backups = 10

Five, supervisor command description

supervisorctl status # View the status of all processes
supervisorctl stop es # stop es
supervisorctl start es # start es
supervisorctl restart es # restart es
supervisorctl update # After the configuration file is modified, use this command to load the new configuration
supervisorctl reload # Restart all programs in the configuration, equivalent to supervisorctl restart all