hanjm's blog

Python(Tornado Tornado+uvloop Tornado+uvloop2 Flask Sanic Django) Go(http Httprouter Fasthttp) RPS性能对比

发表于 2017-03-21 |

在v2ex上发现了一个Python的新web框架, Sanic, 基于uvloop, 其在GitHub上的性能数据十分耀眼, 于是想跟其他用过的web框架对比一下性能, 跑个分嘛.

测试环境:

Python:3.6.0 Go:1.8.0

测试脚本:

app用tmux启动, 然后使用Apache bench测试, n=1000, c=100, 每个app测5次

for session in test_tornado test_tornadouv test_tornadouv2 test_flask test_sanic
do
tmux kill-session -t $session > /dev/null 2>&1
sleep 1
tmux new -s $session -d "/root/.pyenv/versions/3.6.0/bin/python $session.py"
done
# django
tmux kill-session -t test_django
sleep 1
tmux new -s test_django -d "/root/.pyenv/versions/3.6.0/bin/python test_django/manage.py runserver 127.0.0.1:8893"

for session in test_go_http test_go_httprouter test_go_fasthttp
do
tmux kill-session -t $session
sleep 1
tmux new -s $session -d "go run $session.go"
done


for ((i=0;i<5;i++))
do
echo $i
ab -n 1000 -c 100 http://127.0.0.1:8888/ 2> /dev/null |grep "Requests per second" |awk '{printf "tornado-epoll  :%s\n",$0}'
ab -n 1000 -c 100 http://127.0.0.1:8889/ 2> /dev/null |grep "Requests per second" |awk '{printf "tornado-uvloop :%s\n",$0}'
ab -n 1000 -c 100 http://127.0.0.1:8890/ 2> /dev/null |grep "Requests per second" |awk '{printf "tornado-uvloop2:%s\n",$0}'
ab -n 1000 -c 100 http://127.0.0.1:8891/ 2> /dev/null |grep "Requests per second" |awk '{printf "flask          :%s\n",$0}'
ab -n 1000 -c 100 http://127.0.0.1:8892/ 2> /dev/null |grep "Requests per second" |awk '{printf "sanic          :%s\n",$0}'
ab -n 1000 -c 10  http://127.0.0.1:8893/ 2> /dev/null |grep "Requests per second" |awk '{printf "django         :%s\n",$0}' # django自带的wsgi server 无法处理concurrent 100
ab -n 1000 -c 100 http://127.0.0.1:8894/ 2> /dev/null |grep "Requests per second" |awk '{printf "go_http        :%s\n",$0}'
ab -n 1000 -c 100 http://127.0.0.1:8895/ 2> /dev/null |grep "Requests per second" |awk '{printf "go_httprouter  :%s\n",$0}'
ab -n 1000 -c 100 http://127.0.0.1:8896/ 2> /dev/null |grep "Requests per second" |awk '{printf "go_fasthttp    :%s\n",$0}'
done

测试结果:

本机 i5-6600k@3.5GHZ 四核16GB

0
tornado-epoll  :Requests per second:    1714.60 [#/sec] (mean)
tornado-uvloop :Requests per second:    1988.30 [#/sec] (mean)
tornado-uvloop2:Requests per second:    1977.77 [#/sec] (mean)
flask          :Requests per second:    2167.29 [#/sec] (mean)
sanic          :Requests per second:    14221.72 [#/sec] (mean)
django         :Requests per second:    1157.64 [#/sec] (mean)
go_http        :Requests per second:    13847.92 [#/sec] (mean)
go_httprouter  :Requests per second:    14595.34 [#/sec] (mean)
go_fasthttp    :Requests per second:    12730.74 [#/sec] (mean)
1
tornado-epoll  :Requests per second:    2011.02 [#/sec] (mean)
tornado-uvloop :Requests per second:    1788.50 [#/sec] (mean)
tornado-uvloop2:Requests per second:    1980.40 [#/sec] (mean)
flask          :Requests per second:    2237.84 [#/sec] (mean)
sanic          :Requests per second:    13614.70 [#/sec] (mean)
django         :Requests per second:    1145.03 [#/sec] (mean)
go_http        :Requests per second:    13811.01 [#/sec] (mean)
go_httprouter  :Requests per second:    13441.94 [#/sec] (mean)
go_fasthttp    :Requests per second:    12619.25 [#/sec] (mean)
2
tornado-epoll  :Requests per second:    2056.31 [#/sec] (mean)
tornado-uvloop :Requests per second:    1831.49 [#/sec] (mean)
tornado-uvloop2:Requests per second:    2015.39 [#/sec] (mean)
flask          :Requests per second:    2124.87 [#/sec] (mean)
sanic          :Requests per second:    12485.80 [#/sec] (mean)
django         :Requests per second:    1160.55 [#/sec] (mean)
go_http        :Requests per second:    14014.43 [#/sec] (mean)
go_httprouter  :Requests per second:    13949.34 [#/sec] (mean)
go_fasthttp    :Requests per second:    12419.12 [#/sec] (mean)
3
tornado-epoll  :Requests per second:    2017.32 [#/sec] (mean)
tornado-uvloop :Requests per second:    1731.04 [#/sec] (mean)
tornado-uvloop2:Requests per second:    2014.46 [#/sec] (mean)
flask          :Requests per second:    2100.02 [#/sec] (mean)
sanic          :Requests per second:    12459.04 [#/sec] (mean)
django         :Requests per second:    1164.06 [#/sec] (mean)
go_http        :Requests per second:    13239.77 [#/sec] (mean)
go_httprouter  :Requests per second:    14127.49 [#/sec] (mean)
go_fasthttp    :Requests per second:    12824.46 [#/sec] (mean)
4
tornado-epoll  :Requests per second:    1810.77 [#/sec] (mean)
tornado-uvloop :Requests per second:    1975.12 [#/sec] (mean)
tornado-uvloop2:Requests per second:    1962.81 [#/sec] (mean)
flask          :Requests per second:    2234.08 [#/sec] (mean)
sanic          :Requests per second:    13881.95 [#/sec] (mean)
django         :Requests per second:    1154.90 [#/sec] (mean)
go_http        :Requests per second:    14399.47 [#/sec] (mean)
go_httprouter  :Requests per second:    13688.69 [#/sec] (mean)
go_fasthttp    :Requests per second:    12672.99 [#/sec] (mean)

阿里云ECS 双核8GB

0
tornado-epoll  :Requests per second:    542.60 [#/sec] (mean)
tornado-uvloop :Requests per second:    930.95 [#/sec] (mean)
tornado-uvloop2:Requests per second:    811.54 [#/sec] (mean)
flask          :Requests per second:    897.00 [#/sec] (mean)
sanic          :Requests per second:    5681.50 [#/sec] (mean)
django         :Requests per second:    447.41 [#/sec] (mean)
go_http        :Requests per second:    8551.98 [#/sec] (mean)
go_httprouter  :Requests per second:    9655.31 [#/sec] (mean)
go_fasthttp    :Requests per second:    11469.21 [#/sec] (mean)
1
tornado-epoll  :Requests per second:    969.97 [#/sec] (mean)
tornado-uvloop :Requests per second:    970.14 [#/sec] (mean)
tornado-uvloop2:Requests per second:    990.24 [#/sec] (mean)
flask          :Requests per second:    900.50 [#/sec] (mean)
sanic          :Requests per second:    5789.89 [#/sec] (mean)
django         :Requests per second:    435.28 [#/sec] (mean)
go_http        :Requests per second:    8076.63 [#/sec] (mean)
go_httprouter  :Requests per second:    9340.99 [#/sec] (mean)
go_fasthttp    :Requests per second:    6024.82 [#/sec] (mean)
2
tornado-epoll  :Requests per second:    946.98 [#/sec] (mean)
tornado-uvloop :Requests per second:    975.64 [#/sec] (mean)
tornado-uvloop2:Requests per second:    955.25 [#/sec] (mean)
flask          :Requests per second:    1021.28 [#/sec] (mean)
sanic          :Requests per second:    6361.36 [#/sec] (mean)
django         :Requests per second:    440.23 [#/sec] (mean)
go_http        :Requests per second:    7601.67 [#/sec] (mean)
go_httprouter  :Requests per second:    8123.21 [#/sec] (mean)
go_fasthttp    :Requests per second:    8500.00 [#/sec] (mean)
3
tornado-epoll  :Requests per second:    944.39 [#/sec] (mean)
tornado-uvloop :Requests per second:    921.12 [#/sec] (mean)
tornado-uvloop2:Requests per second:    980.30 [#/sec] (mean)
flask          :Requests per second:    946.53 [#/sec] (mean)
sanic          :Requests per second:    5686.99 [#/sec] (mean)
django         :Requests per second:    498.31 [#/sec] (mean)
go_http        :Requests per second:    6297.78 [#/sec] (mean)
go_httprouter  :Requests per second:    8078.91 [#/sec] (mean)
go_fasthttp    :Requests per second:    6523.84 [#/sec] (mean)
4
tornado-epoll  :Requests per second:    953.86 [#/sec] (mean)
tornado-uvloop :Requests per second:    983.97 [#/sec] (mean)
tornado-uvloop2:Requests per second:    955.04 [#/sec] (mean)
flask          :Requests per second:    994.77 [#/sec] (mean)
sanic          :Requests per second:    6877.11 [#/sec] (mean)
django         :Requests per second:    510.76 [#/sec] (mean)
go_http        :Requests per second:    8243.14 [#/sec] (mean)
go_httprouter  :Requests per second:    7055.97 [#/sec] (mean)
go_fasthttp    :Requests per second:    6183.83 [#/sec] (mean)

###结论:

Sanic确实叼, 拉出Flask一大截. 由于这个测试只是简单返回”Hello, world”, Tornado的优势要在使用了aio,AsyncHTTPClient的时候才能发挥出来. 如果拿tornado当纯同步使用的话, 性能会比flask弱一点点.
Django自带的wsgi server无法完成此任务.
fasthttp表现的很奇怪, 可能是handler太简单的情况下sync/pool的代价超过了gc的影响, httprouter比http快一点.

###测试代码:

1.test_tornado.py Tornado 4.4.2

import tornado.ioloop
import tornado.web


class MainHandler(tornado.web.RequestHandler):
    def get(self):
        self.write("Hello, world")


def make_app():
    return tornado.web.Application([
        (r"/", MainHandler),
    ])


if __name__ == "__main__":
    app = make_app()
    app.listen(8888)
    tornado.ioloop.IOLoop.current().start()

2.test_tornadouv.py Tornado+uvloop(pip install tornadouvloop)

import tornado.ioloop
import tornado.web
from tornadouvloop import TornadoUvloop


class MainHandler(tornado.web.RequestHandler):
    def get(self):
        self.write("Hello, world")


def make_app():
    return tornado.web.Application([
        (r"/", MainHandler),
    ])


if __name__ == "__main__":
    app = make_app()
    app.listen(8889)
    tornado.ioloop.IOLoop.configure(TornadoUvloop)
    tornado.ioloop.IOLoop.current().start()

3.test_tornadouv2.py Tornado+uvloop2(pip install tornaduv)

import tornado.ioloop
import tornado.web
from tornaduv import UVLoop


class MainHandler(tornado.web.RequestHandler):
    def get(self):
        self.write("Hello, world")


def make_app():
    return tornado.web.Application([
        (r"/", MainHandler),
    ])


if __name__ == "__main__":
    app = make_app()
    app.listen(8890)
    tornado.ioloop.IOLoop.configure(UVLoop)
    tornado.ioloop.IOLoop.current().start()

4.test_flask.py Flask 0.12

from flask import Flask

app = Flask(__name__)


@app.route("/")
def hello():
    return "Hello, world"


if __name__ == "__main__":
    app.run(port=8891)

5.test_sanic.py Sanic 0.4.1

from sanic import Sanic
from sanic.response import HTTPResponse

app = Sanic()


@app.route("/")
async def test(request):
    return HTTPResponse("Hello, world")


if __name__ == "__main__":
    app.run(port=8892)

6.test_django.py Django

# views.py
from django.http import HttpResponse


def hello(request):
    return HttpResponse("Hello, world")

7.test_go_http.go Go http

package main

import (
	"net/http"
	"runtime"
)

func main() {
	runtime.GOMAXPROCS(runtime.NumCPU())
	http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
		w.Write([]byte("Hello, world"))
	})
	http.ListenAndServe("127.0.0.1:8894", nil)
}

8.test_go_httprouter.go GO httprouter

package main

import (
	"net/http"
	"github.com/julienschmidt/httprouter"
	"runtime"
)

func main() {
	runtime.GOMAXPROCS(runtime.NumCPU())
	router := httprouter.New()
	router.HandlerFunc("GET", "/", func(w http.ResponseWriter, r *http.Request) {
		w.Write([]byte("Hello, world"))
	})
	http.ListenAndServe("127.0.0.1:8895", router)
}

9.test_go_fasthttp.go Go fasthttp

package main

import (
	"github.com/valyala/fasthttp"
	"runtime"
)

func main() {
	runtime.GOMAXPROCS(runtime.NumCPU())
	requestHandler := func(ctx *fasthttp.RequestCtx) {
		ctx.Write([]byte("Hello, world"))
	}
	fasthttp.ListenAndServe("127.0.0.1:8896", requestHandler)
}

Go快速笔记

发表于 2016-12-31 |

1. := 与var定义

在函数中，:= 简洁赋值语句在明确类型的地方，可以用于替代 var 定义。

函数外的每个语句都必须以关键字开始（var、func、等等），:= 结构不能使用在函数外。

常量用const定义

slice channle map 必须使用make函数来定义

new 分配内存后置零,返回指针

make 分配内存后初始化, 返回对象

在满足下列条件时，已被声明的变量 v 可出现在:= 声明中：

本次声明与已声明的 v 处于同一作用域中,（若 v 已在外层作用域中声明过，则此次声明会创建一个新的变量v）
在初始化中与其类型相应的值才能赋予 v，且在此次声明中至少另有一个变量是新声明的。

2. 基本数据类型

bool
string
int  int8  int16  int32  int64
uint uint8 uint16 uint32 uint64 uintptr
byte // uint8 的别名
rune // int32 的别名
     // 代表一个Unicode码
float32 float64
complex64 complex128

单引号字符常量表示 Unicode Code Point，⽀支持 \uFFFF、\U7FFFFFFF、\xFF 格式。 对应 rune 类型，UCS-4。
var c1, c2 rune = '\u6211', '们'

3.Channel

channel 操作符 <- ,操作符只有这一个,这样就不会搞错位置了

发送者可以 close 一个 channel 来表示再没有值会被发送了。接收者可以通过赋值语句的第二参数来测试 channel 是否被关闭：当没有值可以接收并且 channel 已经被关闭，那么经过

v, ok := <-ch
之后 ok 会被设置为 false。

循环 for i := range c 会不断从 channel 接收值，直到它被关闭。

注意：只有发送者才能关闭 channel，而不是接收者。向一个已经关闭的 channel 发送数据会引起 panic。

还要注意： channel 与文件不同；通常情况下无需关闭它们。只有在需要告诉接收者没有更多的数据的时候才有必要进行关闭，例如中断一个 range。

4.select

select 语句使得一个 goroutine 在多个通讯操作上等待。

select 会阻塞，直到条件分支中的某个可以继续执行，这时就会执行那个条件分支。当多个都准备好的时候，会随机选择一个。

当 select 中的其他条件分支都没有准备好的时候，default 分支会被执行。

为了非阻塞的发送或者接收，可使用 default 分支：

select {
case i := <-c:
    // 使用 i
default:
    // 从 c 读取会阻塞
}

5.for range 遍历 range 会复制对象

for key, value := range oldMap {
	newMap[key] = value
}

//若你只需要该遍历中的第一个项（键或下标），去掉第二个就行了：

for key := range m {
	if key.expired() {
		delete(m, key)
	}
}

6.Switch

switch 可以什么都不写当多级if else使用还可以当做判断类型使用 t.(type) 只能在switch使用

switch t := t.(type)

7.Map

key不存在不会报错，会返回与map的value的类型对应的零值. 用if _,ok:=someMap[key];ok {} 来判断有没有值

8.Print

字符串函数（Sprintf 等）会返回一个字符串，而非填充给定的缓冲区。
fmt.Fprint 一类的格式化打印函数可接受任何实现了io.Writer接口的对象作为第一个实参:如os.Stdout`os.Stderr`
fmt.Printf
- %v（对应“值”）map中的键可能按任意顺序输出。
- 当打印结构体时，改进的格式 %+v 会为结构体的每个字段添上字段名，而另一种格式 %#v 将完全按照Go的语法打印值。
- 当遇到 string 或 []byte 值时，可使用 %q 产生带引号的字符串；而格式 %#q 会尽可能使用反引号。
- %T 类型
- 若你想控制自定义类型的默认格式，只需为该类型定义一个具有 String() string 签名的方法。

9.append 中extend

x := []int{1,2,3}
y := []int{4,5,6}
x = append(x, y...)
fmt.Println(x)

10.类型转换

要提取我们知道在该值中的字符串，可以这样：

str := value.(string)
但若它所转换的值中不包含字符串，该程序就会以运行时错误崩溃。为避免这种情况，需使用“逗号, ok”惯用测试它能安全地判断该值是否为字符串：

str, ok := value.(string) 
if ok { 
	fmt.Printf("字符串值为 %q\n", str)
 } else { 
 	fmt.Printf("该值非字符串\n") 
 }

若类型断言失败，str 将继续存在且为字符串类型，但它将拥有零值，即空字符串。

11. x:=x

req := req
但在Go中这样做是合法且惯用的。你用相同的名字获得了该变量的一个新的版本，以此来局部地刻意屏蔽循环变量，使它对每个Go程保持唯一。
比如gorm中

12.函数

变参变参本质上就是 slice。只能有⼀一个，且必须是最后⼀一个。

func test(s string, n ...int) string {
 var x int
 for _, i := range n {
 	x += i 
 }
return fmt.Sprintf(s, x)
}
func main() { 
println(test("sum: %d", 1, 2, 3)) 
}

使⽤用 slice 对象做变参时，必须展开。

func main() { 
	s := []int{1, 2, 3
}
println(test("sum: %d", s...)) }

匿名函数可赋值给变量，做为结构字段，或者在 channel ⾥里传送。

13.Defer

func add(x, y int) (z int) {
	defer func() { 
		println(z) // 输出: 203 
	}()
	z = x + y return z + 200
	// 执⾏行顺序: (z = z + 200) -> (call defer) -> (ret)
}

多个 defer 注册，按 FILO 次序执⾏行。哪怕函数或某个延迟调⽤用发⽣生错误，这些调⽤用依旧会被执⾏行。

func test(x int) {
	defer println("a") defer println("b")
	defer func() { println(100 / x) }()
	defer println("c")
	// div0 异常未被捕获，逐步往外传递，最终终⽌止进程。
}
func main() { test(0) }

输出：
c b a panic: runtime error: integer divide by zero

14. error

捕获函数 recover 只有在延迟调⽤用内直接调⽤用才会终⽌止错误，否则总是返回 nil。任何未捕获的错误都会沿调⽤用堆栈向外传递。

导致关键流程出现不可修复性错误的使⽤用 panic，其他使⽤用 error。

15. 数组

a := [3]int{1, 2}// 未初始化元素值为 0。 
b := [...]int{1, 2, 3, 4}// 通过初始化值确定数组⻓长度。
c := [5]int{2: 100, 4:200} // 使⽤用索引号初始化元素。
d := [...]struct { name string age uint8 }{ {"user1", 10}, {"user2", 20}, } // 可省略元素类型。 // 别忘了最后⼀一⾏行的逗号。
⽀支持多维数组。
a := [2][3]int{{1, 2, 3}, {4, 5, 6}} 
b := [...][2]int{{1, 1}, {2, 2}, {3, 3}}  // 第 2 纬度不能⽤用 "..."。

16. slice

array和slice是不同的类型,不同长度的array类型不同 slice是[]int array是[n]int

官方文档中说

In Go, array is a fixed length of continuous memory with specified type, while slice is just a reference which points to an underlying array. Since they are different types, they can't assign value each other directly.