Gunicorn 源码阅读

gunicorn “Green Unicorn”，脱胎于ruby社区的Unicorn，是一个 WSGI HTTP Server。学习gunicornpython教程后，咱们能够把之前的 Bottle 程序Django正式布置起来。老规矩，本文分下面几个部分:

• gunicorn 项目结构简介
• gunicorn 运用
• gunico源码网站rn-application 完毕
• arbiter完毕
• sync-worker完毕
• 小结
• 小技巧

gunicorn 项目结构简介

gunicorn 源码挑选的版别是 20.0.0，首要的文件及包如下:

文件http://192.168.1.1登录	描绘
app包	guincodjango结构rn 的 Application （不是wsgi界说的applicaton）
hdjango结构首要用来做什么ttp包	gunicorn 对 http协议的一些处理
workershttp 404包	gunicorn 的作业类完毕 ，包含同步sync完毕，线程池版别完毕gthread，以及异步版别完毕 geventlet，gevent等
arbiter.py	guicorn 的master完毕

根据gdjango布置unicorn的规划特色：

Gunicorn is based on the pre-fo源码网站rk worker model. Thislinux装备ip地址 means that there is a central master process thatdjango菜鸟教程 manages a set of worker processes. The master ndjango布置ever knows anything abdjango教程out individual cl源码年代ients. All requestshttp://www.baidu.com and responses are handled c源码本钱om源码编辑器pletely by worker processpython基础教程es.

gdjango是什么意思unhttp://192.168.1.1登录icorn运用pre-fork 作业模型，也就是maspython123ter提早forhttp://www.baidu.comk出预定数量的workhttp署理，处理worker调集。一切的re源码quest和response都由worker进程处理。

咱们关键放在：gunicorn的服务完毕，master-worker怎样完毕和协作上。

gunicorn 运用

编写检验源码本钱app，能够看到这是一个符合wsgi标准的application：

# myapp.py
def app(environ, start_response):  # env 和 http 情况及头设定回调
data = b"Hello, Whttp 404orld!n"
start_response("200 OK"httpclient, [
("Content-Type", "text/plain"),
("Content-Ldjango面试题ength", str(len(data)))
])
return iter([data])  #django菜鸟教程 回来数据

运用4个源码work节点，日志等级debug的办法建议服务，加载 myapp:appdjango面试题

# gunicorn -w 4 --lopython为什么叫爬虫g-level debug  myapdjango是什么意思p:app
[2021-02-23 17:58:57 +0800] [502源码年代58] [DEBUG] Current configurpython123ation:  # 预备装备
...
[2021-02-23 18:01:12 +0800] [5046http署理2] [INFO] Starting gunpython123icpython为什么叫爬虫orn 20.0.0  # 建议gunicorn
[2021-02-23 18:01:1HTTP2 +0800] [50462] [DEBUG] Arbiter booted  # 建议master
[2021-02-23 18:01:12httpwatch +0800] [5django布置0462] [INFO] Listening apython123t: http://127.0.0.1:8000 (50462)  # 监听端口
[2021-02-23 18:01:12 +0800] [50462] [INFO] Using worker: sync
[2021-02-23 18:01:12 +0800]python基础教程 [50464] [INFO] Booting worker with pid: 50464 # 建议worker
[2021-02-23 18:01:12 +0800] [50465] [INFO] Booting worker with pid: 50465
[2021-02-23 18:01:12 +0800] [50466] [INFO] BLinuxooting worker with p源码超市id: 50466
[2021源码网站-02-23 18:01:12 +0800] [50467] [INFO] Booting worker with pid: 50467
[2021-python是什么意思02-23 18:01:12 +0800] [50462] [DEBUG] 4 workers

运用 curl 检验服务

# curl http://http://192.168.1.1登录127.0.0.1:8000
Hello, World!

一起gunicorn中能够看到 worHTTPker=50465 处理了这个http央求

[2021-02-24 16:09:39 +0800] [50465] [DEBUG] GET /

运转时分，还能够通过发送信号，手动扩展work节点数

# kill -TTIN 504linux中文乱码视频62

调查服务日志，会发现 master=50462 进程处理了 ttin 信号，并且扩展worker节点数到5

...
[2021-02-24 18:02:56 +0800] [50462] [INFO] Handling signal源码本钱: ttin
[2021-HTTP02-24 18:02:56 +0800] [75918] [INFhttpclientO] Bootpython基础教程ing worker with pid: 75918
[2021-02-24 18:02:56 +0800] [50462] [DEBUG] 5 workers

运用 Ctrl+C 封闭服务，能够看到也是 master=50462 进程处理了 int 信号，并且在httpclient封闭worhttpclientker节点django怎样读后封闭自己

^C[2021-02-25 15:06:54 +0800] [5django面试题0462] [INFO] Handling signal: int
[2021-02-25 15:06:54 +0800] [50464] [INFO] Worker exiting (pid: 504http://192.168.1.1登录64)
[python教程2021-02-25 15:06:54 +0800] [5python培训班膏火一般多少0465] [INFO] Worker exiting (pidhttps和http的差异: 50465)
[2021-02-25 15:06:54 +0800] [50466] [INFO] Worker elinux指令xiting (pid: 50466)
[2021-02-25 15:06:54 +0800] [50467] [INFO] Worker exiting (pid: 50467)
[2021-02-25 15:06:54 +0800] [75918] [INFO] Worker exiting (pid: 75918)
[2021-02-25 15:06:54 +0800] [50462] [INFO] Shutting downLinux: Master

假如对gunicon的参数不了解，能够linux装备ip地址运用下面指令查看帮助

# gunicorn -h
usage: gunicorn [OPTIONS] [APP_MODULE]
optional alinuxrguments:
-h, --help            show this heDjangolp message and exit
...
-w INT, --python为什么叫爬虫workers INT
The number of worker pro源码超市cesses f源码之家or handling requests. [1]

帮助运用咱们了解的 arglinux是什么操作体系parse 完毕。

class Setting(object):
def add_option(self, parselinux中文乱码视频r):
args = tuple(self.cli)
help_txt = "%s [%s]" % (self.short, self.httpwatchdefault)
help_txt = help_txt.replace("%", "%%")
kwargs = {
"dest": self.name,
"action": self.action or "store",
"type": self.type or str,
"default": None,
"help": help_txt
}
...
parser.add_argument(*args, **kwa源码编辑器rgs)  # 增加选项
classhttp://192.168.1.1登录 Workers(Setting):  # --workers 的选项类
name = "workers"
section = "Wpython怎样读orker Processes"django菜鸟教程
cli = ["-w", "--workepython教程rs"]
meta = "INT"
validator = validate_pos_int
type = int
default = int(os.environhttpclient.get("WEB_CONCURRENCY", 1django布置))
desc = """
The number of worker processes for handling request源码年代s.
A positive integer generall源码编辑器编程猫下载y in the ``2-4 x $(NUM_CORES)`` range.
You'll want to vary this a bit to find the bepython下载装置教程st for your particular
application's work loa源码d.
By default, the value ofpython下载装置教程 thPythone ``WEB_CONCURRENCY`` environment variable.
If it isdjango结构过期了吗 not defined, the defau源码lt is ``1``.
"""
def parser(self源码本钱):
kwargs = {
"usage": self.usage,
"prog": sellinux是什么操作体系f.prog
}
parser = argparse.linux体系ArgumentParsHTTPer(**kwargs)
parser.add_源码编辑器编程猫下载argument("-v", "--verslinux指令ion",
action="version", d源码分享网efault=argparse.SUPPREpython教程SS,
version="django结构过期了吗%(prog)s (version " + __version__ + ")n",
help=linux体系装置"show program's version number and exit")
ppython教程arser.add_a源码之家rgument("args", nahttp 302rgs="*", help=argparse.SUPPRESS)
keys = sorted(self.sepython教程ttings, key=self.settings.__getitem__)  # 动态增加参数选项
for k in keys:
self.settings[k].add_option(parser)
return parse源码编辑器编程猫下载r

gunicorn-application 完毕

gunicorn的app源码年代lication首要是下面三个类完毕python为什么叫爬虫。需求注意的是这儿的application能够理解为web-server的application；bottle/flask/django等完毕的是web-framework的applicaitdjango面试题on。源码本钱前者动态加载后者，前者处理http服务，后者处理单次的http央求。

• BaseApplicationlinux指令
  • Applicat源码之家ion
    • WSGIApplication

3个Application收拾后，大约的代码模版如下:

class WSDjangoGIApplication(Applicapython怎样读tion)
def __init__(self, usag源码本钱e=None, prog=None):源码年代
self.do_lhttp 404oad_config()  # 加载装备
def do_load_config():
...
cfg = self.init(parser, args, args.args)  # 初始化装备
...
def init(源码分享网...):
...
self.app_uri = args[0]  # 获取wsgi-application参数
def load(...):
util.import_app(self.applinux中文乱码视频_uri)  # 动态加载wsgi-application
..django教程.
def run(...):
self.load()
Arbiter(self).run(python怎样读)  # 建议master，也就是Arbiter
def run():  # 运转服务
"""
The ``gunicorn`` command line runner for launching Gunicorn with
generic WSGI applications.
"""
from ghttp 404unicorn.app.wsgiapp import WSGIApplication
WSGIApplication("%(python怎样读prog)linux体系装置s [OPTIONS] [APhttpclientP_MODULE]").run()
if __name__ == '__main__':
run()

application部分的完毕，相对比较简单，源码编辑器就不再赘述linux常用指令。

arbiter完毕

Arbiter 仲裁者，事实上的master进程中心，收拾后代码模版如下：

classhttp协议 Arbiter(object):
def __init__(self, app):
self.worker_class = self.cfhttp协议g.worker_class  # worker类
self.num_workepython怎样读rs = self.cfg.worker  # worker数量
...
def start():
self.init_signals()  #源码编辑器 初始化信号监听
...
sock.crdjango是什么意思eate_socket(.httpclient.源码本钱.) # 创立socket服务
def run(self):
selhttp://www.baidu.comf.start()
try:
self.manage_wpython培训班膏火一般多少orkers() # 建议节点
while True: #  无django结构过期了吗限循环
...
sig = self.SIG_QUEUE.pop(0) if self.SIG_QUEUE els源码交易网站源码e None
if sig is None:
self.sleep()  # 持续休眠
self.murder_workers()
self.manage_workers()
continue
if sig not in self.SIG_NAMES:
self.loghttp://www.baidu.com.info("Ignoring unknown sign源码年代al: %s", sig)
continpython怎样读ue
# 处理信源码编辑器号
signame = self.SIG_NAMES.get(s源码ig)
handler = getattr(self, "handle_%s" % silinux体系装置g源码网站name, None)
...
hanPythondler()
self.wakeup()  # 唤醒
except (StopIteration, Keyboalinux体系rdInterrupt):
...

在了解Arbiter作业前先了解一下信号, linux 体系能够https和http的差异运用下面指令查看信号源码之家清单

# kill -lhttp://www.baidu.com
1) SIGHUP	 2) SIGINT	 3) SIGQUIT	 4) SIGILLhttp 500	 5) SIGTRAP
6) SIG源码ABRT	 7) SIGBUS	 8) SIGFPE	 9) SIGKILL	10) SIGUSR1
11) SIGSEGV	12) SIGUSR2	13)linux SIGPIPE	14) SIGALRM	15) SIGTERM
...

• 1 (SIGHUP): terminate a connection, or reload the confi源码编辑器编程猫下载guration for daemons
• 2 (SIGINT): interrupt thelinux体系装置 sdjango面试题ession fhttp://www.baidu.comrom the dialogue station
• 3 (SIGHTTPQUIT): terminate the shttp署理ession from the dialdjango是什么意思ogue station
• 4 (SIGILL): illegal instru源码年代ction was executed
• 5 (SIGTRAPlinux关机指令): do a single instrucDjangotion (trap)
• 6http://192.168.1.1登录 (SIGABRT): abnormal termination
• 7 (SIG源码编辑器编程猫下载BUS): error on the system bus
• 8 (SI源码之家GFPE):源码 floadjango结构首要用来做什么ting point error
• 9 (SIGKILL): immmediatelinux指令ly terminate the p源码年代rocess
• 10 (SIGUSR1): user-defined signal
• 11 (SIGSEGV): segmentation fault due to illegal access of a memory segment
• 12 (SIGUSR2): user-defined signal
• 13 (SIGPIPE): writing into a pipe, and nobody is reaPythonding from it
• 14 (SIGALRM): the timer termlinux装备ip地址inated (alarm)
• 15 (SIGTERM): termina源码网站te the process in a soft way

信号是操作体系供给的事情，能够用来进行跨进程的通讯。Arbiter.init_signals 做的作业如下:

SIGNALS = [getattr(signal, "SIG%s" % x)
for x in "HUP QUIT INT TERM TTIN TTOU USR1 USR2 WINCH".split(源码之家)]
def init_sipython怎样读gnals(self):
...
# initi源码编辑器alize all signals
for s in slinux体系elf.SIGNALS:
signal.signal(s, self.signal)
signal.signdjango怎样读al(signal.SIGCHLD, self.handle_chld)  # 增加信号监听器
def signallinux中文乱码视频(self, sig, frame)https和http的差异:
if len(self.http://192.168.1.1登录SIG_QUEUE) < 5:
self.SIG_QUEUE.append(sig)django教程
self.wakeup()

之前演示的扩容信号 TTIN 是这样处理的 :

def handle_ttin(self):
"""
SIGTTIN handling.
Increases the number of workers by one.
"""
self.num_workdjango菜鸟教程ers += 1  # 扩容
self.manage_workers()  # 处理workPythoner

Arbiter的sleep和warkeup是这样完毕的:

self.PIPE = pair = os.pipe()  # 创立管道
def sleep(self):
"""
Sleep until PIPE is readablhttpcliente or we timeout.
A readable PIPE means a signal occulinux重启指令rred.
"""
try:
ready = select.select([self.PIPE[linux常用指令0]], [], [], 1.0)  # 运用select监听管道的数据改变
if not ready[0]:
return
whilhttp 404e os.read(self.PIPE[0], 1):  # 读取管道数据
pass
except (select.error, OSError) as e:
...
def wakeup(self):
"""
Wake up the arbiter by writing to the PIPE
"""
try:
os.write(self.PIPE[1], b'.')  # 管道写入
exHTTPcept IOError as e:
...

需求说明的是Arbiter通过 sock.create_sockets 创立了socket，并绑定端口和监听，然后在fork-worker的时分，将socket传递给了子进程。

worker = spython培训班膏火一般多少elf.wpython是什么意思orker_class(self.worker_age, self.pid, self.LISTENEDjangoRS,
self.app, self.timeout / 2.0,
self.cfg, self.log)
shttp 404elf.cfg.pre_fork(self, worker)
pid = oshttp 302.fork()
if pid != 0:
worker.pid = pid  # 记载worker的pid
sehttp协议lf.WORKERShttp署理[pid] = workpython基础教程er # 增加到worker调集
return pid

毁掉worker是运用信号:

ddjango结构过期了吗ef kill_workershttp 302(self, sig):
"""
Kill all workers with the signal `sig`
:att源码编辑器编程猫下载r sig: `signal.SIG*` value
"django菜鸟教程""
worker_plinux体系装置ids = list(self.WORKERS.keys())
for pid in worker_pids:
os.k源码超市ipython怎样读ll(pid, sig)

sync-worker完毕

接下来，咱们看看worker，首要是sync-worker的完毕。workpython是什么意思er的关系首要如下:

• Worker 处理信linux常用指令号
  • SyncWorker 同步处理http央求
  • ThreadWorker 运https和http的差异用线程处理http央求

接之前Arbiter中fork-workerLinux的代码，创立完毕的work进入 init_process

# Process Child
worker.pid = olinux体系s.getpid()
try:
util._setprodjango是什么意思ctitle("worker [%s]" % self.proc_name)
self.log.info("Booting worker with pid: %s", worker.pid)
sepython基础教程lf.cfg.post_fork(self, worker)
worker.init_process()
sys.exit(0)

work的init_proc源码网站ess模版如下:

def init_process(self):
"""
If you override this method in a subclass, th源码交易网站源码e last statement
in the function should be to call this method with
super().init_process() so thapython为什么叫爬虫t the ``run()`` loop is initiated.
"""
# For waking ourselves up
self.PIPE = os.pipe()  # 创立管道
...
selinux重启指令lf.wait_fds = self.sockets + [self.PIPE[0]]  # 监听管道和socket
...
self.init_signals()http://www.baidu.com  # 初始化信号监听
...
self.load_python教程wsgi()  # 加载wsgi的使用
...
# Enter main run loop
self.booted = True
self.run()  # 作业循环

work一样的进行信号监听:

SIGNALS = [getattr(signal, "SIG%s" % x)
for x in "ABRT Hpython能够做什么工作UP QUIT INT TERM USR1 USR2 WINCH CHLD".split()]
def inithttpclient_signals(self):
# reset signaling
for s in源码怎样做成app软件 self.SIGNALS:
signal.signal(s, signal.SIG_DFL)
# indjango布置it new signaling
signal.signal(signal.Shttp://192.168.1.1登录IGQUIT, self.handl源码之家e_quit)
signa源码之家l.signal(signal.SIGTERM, self.handlpython基础教程e_exit)
signal.signal(signal.SIGINT, self.handle_qu源码怎样做成app软件it)
...
if hasattr(signal, 'set_wakeup_fd'):
siglinux关机指令nal.set_wakeup_fd(self.PIPE[1])  # 等候select唤醒

work最重要的run循环:

 def run(self, timeout):
lilinux常用指令stener = self.sockets[0]
while self.alive:
...
# Accept a connection. If we get an error telling us
# that no connepython培训班膏火一般多少ction is waiting we fall down to the
# select which is where we'll wait for a bit for new
# workers to源码分享网 come give us some love.
try:
self.achttp://192.168.1.1登录cept(listener)  # 接受客户端链接
# Keedjango怎样读p processipython培训班膏火一般多少ng clients until no one is waiting. Thishttp://192.168.1.1登录
# prevents the need to selecdjango教程t() for every client that we
# process.
continue
exceptHTTP EnvironmentError as e:
...
try:
selpython是什么意思f.wait(timeout源码网站)  # 休眠等候
except StopWaiting:
return

处理客户端衔接，这一部分和之前介绍http比较类似，也不再赘述。

def acchttpwatchept(self, listener):
client, addr = listener.accept()
client.setblocking(1)
util.close_on_exec(client)
self.handle(listener,linux常用指令 client, addr)

work处理完毕央求后进入等候

def wait(self, timeout):
try:
ret = select.select(self.wahttp协议it_fds, [], [], timeout)
if retlinux中文乱码视频[0]:
if self.linuxPIPE[0] in ret[0]:
os.read(self.PIPE[0], 1)
return ret[0]
except select.error as e:
if e.args[0] == errno.EINTR:
redjango怎样读turn self.sockets
if e.args[0] == errno.EBADF:
if self.nr < 0:
return self.sockets
else:
raise StopWaiting
raise

小结

能够用下python123面一张图展现gunicorn的作业流程，作为咱们的小定论

小技巧python教程

能够运用thread，完毕一个定时器

# reloader.py
class Reloader(threading.Thread):
def __init__(self, extra_files=None, interhttp 302val=1, callback=None):
super().python下载装置教程__init__()
self.setDaemon(True)
self._interval = interval
self._callback = callback
def run(self):
mtimes = {}
while True:
for filename in self.get_files():
try:
mtime = os.stat(filename).st_mtime
except OSError:源码超市
continue
old_time = mtimes.get(filename)
if old_time is None:
mtimes[filename] = mtime
continue
elif mtime > old_time:
if selfdjango教程._callback:
self._callback(filename)
time.sleep(self._indjango布置terval)

在运用 gunicorn myapp:app 指令的时分， myapp:adjango怎样读pp 没有静python下载装置教程态的 import ，而是这样动http协议态加载的:

# util.py
klass = components.pop(-1)
mod = importllinux体系装置ib.import_m源码年代odule('.'.join(components))
return getattr(mod, klass)

参看链接

• Gunicorn Design d源码网站ocs.gunicorn.org/en/stable/d…
• 阅读gunicorn 代码文档 gunicorn.readtheHTTPdocs.io/en/ldjango菜鸟教程ahttp署理test/i…
• Handling Unix Signals in Python stackabuslinuxe.com/handling-un…
• How To Use Signal Driven Programming In Applications medium.com/fintechexpl…
• Django with Nginx, Gunicorn medium.com/analytics-v…