文章目录
1、概述
开发简单 web 服务程序 cloudgo，了解 web 服务器工作原理。
任务目标
- 熟悉 go 服务器工作原理
- 基于现有 web 库，编写一个简单 web 应用类似 cloudgo。
- 使用 curl 工具访问 web 程序
- 对 web 执行压力测试
2、任务要求
基本要求
- 编程 web 服务程序 类似 cloudgo 应用。
- 支持静态文件服务
- 支持简单 js 访问
- 提交表单，并输出一个表格（必须使用模板）
- 使用 curl 测试，将测试结果写入 README.md
- 使用 ab 测试，将测试结果写入 README.md。并解释重要参数。
基本要求的实现
根据课件HTTP协议 与 web/http 库使用指南，直接可以仿照得到main.go和service/server.go。
main.go
package main
import (
"os"
"github.com/github-user/cloudgo/service"
flag "github.com/spf13/pflag"
)
const (
PORT string = "8080"
)
func main() {
port := os.Getenv("PORT")
if len(port) == 0 {
port = PORT
}
pPort := flag.StringP("port", "p", PORT, "PORT for httpd listening")
flag.Parse()
if len(*pPort) != 0 {
port = *pPort
}
server := service.NewServer()
server.Run(":" + port)
}
service/server.go
package service
import (
"net/http"
"os"
"github.com/codegangsta/negroni"
"github.com/gorilla/mux"
"github.com/unrolled/render"
)
// NewServer configures and returns a Server.
func NewServer() *negroni.Negroni {
formatter := render.New(render.Options{
Directory: "templates",
Extensions: []string{".html"},
IndentJSON: true,
})
n := negroni.Classic()
mx := mux.NewRouter()
initRoutes(mx, formatter)
n.UseHandler(mx)
return n
}
func initRoutes(mx *mux.Router, formatter *render.Render) {
webRoot := os.Getenv("WEBROOT")
if len(webRoot) == 0 {
if root, err := os.Getwd(); err != nil {
panic("Could not retrive working directory")
} else {
webRoot = root
//fmt.Println(root)
}
}
mx.HandleFunc("/api/test", apiTestHandler(formatter)).Methods("GET")
mx.PathPrefix("/static").Handler(http.StripPrefix("/static/", http.FileServer(http.Dir(webRoot+"/assets/"))))
mx.HandleFunc("/", homeHandler(formatter)).Methods("GET")
mx.HandleFunc("/login", loginHandler(formatter)).Methods("GET")
mx.HandleFunc("/login", tableform).Methods("POST")
}
此处把Handler函数分离到了service/handlers.go。
再根据下一个课件golang web 服务器端编程，完善service/handlers.go里面的各个handler函数。
package service
import (
"html/template"
"net/http"
"github.com/unrolled/render"
)
func apiTestHandler(formatter *render.Render) http.HandlerFunc {
return func(w http.ResponseWriter, req *http.Request) {
formatter.JSON(w, http.StatusOK, struct {
ID string `json:"id"`
Content string `json:"content"`
}{ID: "8675309", Content: "Hello from Go!"})
}
}
func homeHandler(formatter *render.Render) http.HandlerFunc {
return func(w http.ResponseWriter, req *http.Request) {
formatter.HTML(w, http.StatusOK, "index", struct {}{})
}
}
func loginHandler(formatter *render.Render) http.HandlerFunc {
return func(w http.ResponseWriter, req *http.Request) {
formatter.HTML(w, http.StatusOK, "login", struct {}{})
}
}
func tableform(w http.ResponseWriter, r *http.Request) {
r.ParseForm()
username := template.HTMLEscapeString(r.Form.Get("username"))
password := template.HTMLEscapeString(r.Form.Get("password"))
t := template.Must(template.New("detail.html").ParseFiles("./templates/detail.html"))
err := t.Execute(w, struct {
Username string
Password string
}{Username: username, Password: password})
if err != nil {
panic(err)
}
}
不过，要想程序能够运行，还需要执行三条指令。
go get github.com/codegangsta/negroni
go get github.com/gorilla/mux
go get github.com/unrolled/render
支持静态文件服务
mx.PathPrefix("/static").Handler(http.StripPrefix("/static/", http.FileServer(http.Dir(webRoot+"/assets/"))))
访问http://localhost:8080/static/js/hello.js得到
访问http://localhost:8080/static/css/main.css得到
支持简单 js 访问支持简单 js 访问
func apiTestHandler(formatter *render.Render) http.HandlerFunc {
return func(w http.ResponseWriter, req *http.Request) {
formatter.JSON(w, http.StatusOK, struct {
ID string `json:"id"`
Content string `json:"content"`
}{ID: "8675309", Content: "Hello from Go!"})
}
}
结果就是获得一个ID跟Content。
同样地主页也可以通过hello.js来获取ID跟Content。
hello.js
$(document).ready(function() {
$.ajax({
url: "/api/test"
}).then(function(data) {
$('.greeting-id').append(data.id);
$('.greeting-content').append(data.content);
});
});
func homeHandler(formatter *render.Render) http.HandlerFunc {
return func(w http.ResponseWriter, req *http.Request) {
formatter.HTML(w, http.StatusOK, "index", struct {}{})
}
}
提交表单，并输出一个表格（必须使用模板）
可以参考文档4.1 处理表单的输入。
模板
login.html
<html>
<head>
<link rel="stylesheet" href="static/css/main.css"/>
</head>
<body>
<form action="/login" method="post">
用户名:<input type="text" name="username">
密码:<input type="text" name="password">
<input type="submit" value="登录">
</form>
</body>
</html>
detail.html
<html>
<head>
<link rel="stylesheet" href="static/css/main.css"/>
</head>
<body>
<table>
<tr>
<td>Username</td>
<td>Password</td>
</tr>
<tr>
<td>{{.Username}}</td>
<td>{{.Password}}</td>
</tr>
</table>
</body>
</html>
mx.HandleFunc("/login", loginHandler(formatter)).Methods("GET")
mx.HandleFunc("/login", tableform).Methods("POST")
func loginHandler(formatter *render.Render) http.HandlerFunc {
return func(w http.ResponseWriter, req *http.Request) {
formatter.HTML(w, http.StatusOK, "login", struct {}{})
}
}
func tableform(w http.ResponseWriter, r *http.Request) {
r.ParseForm()
username := template.HTMLEscapeString(r.Form.Get("username"))
password := template.HTMLEscapeString(r.Form.Get("password"))
t := template.Must(template.New("detail.html").ParseFiles("./templates/detail.html"))
err := t.Execute(w, struct {
Username string
Password string
}{Username: username, Password: password})
if err != nil {
panic(err)
}
}
提交前：
提交后：
使用 curl 测试
curl -v http://localhost:8080
curl -v http://localhost:8080/login
使用 ab 测试
首先需要安装 Apache web 压力测试程序。
sudo yum -y install httpd-tools --nogpgcheck
ab性能指标：
Document Path: / ###请求的资源
Document Length: 50679 bytes ###文档返回的长度，不包括相应头
Concurrency Level: 3000 ###并发个数
Time taken for tests: 30.449 seconds ###总请求时间
Complete requests: 3000 ###总请求数
Failed requests: 0 ###失败的请求数
Write errors: 0
Total transferred: 152745000 bytes
HTML transferred: 152037000 bytes
Requests per second: 98.52 [#/sec] (mean) ###平均每秒的请求数
Time per request: 30449.217 [ms] (mean) ###平均每个请求消耗的时间
Time per request: 10.150 [ms] (mean, across all concurrent requests) ###上面的请求除以并发数
Transfer rate: 4898.81 [Kbytes/sec] received ###传输速率
Connection Times (ms)
min mean[+/-sd] median max
Connect: 2 54 27.1 55 98
Processing: 51 8452 5196.8 7748 30361
Waiting: 50 6539 5432.8 6451 30064
Total: 54 8506 5210.5 7778 30436
Percentage of the requests served within a certain time (ms)
50% 7778 ###50%的请求都在7778Ms内完成
66% 11059
75% 11888
80% 12207
90% 13806
95% 18520
98% 24232
99% 24559
100% 30436 (longest request)
那么通过指令ab -n 1000 -c 100 http://localhost:8080/则有
完整代码提交
我的完整代码
扩展要求
选择以下一个或多个任务，以博客的形式提交。
- 通过源码分析、解释一些关键功能实现
- 选择简单的库，如 mux 等，通过源码分析、解释它是如何实现扩展的原理，包括一些 golang 程序设计技巧。
net/http库源码分析
web的工作原理分为创建 ServerSocket，绑定并 listen，accept 连接，创建 go 线程服务一个连接这四个步骤， 首先从入口函数ListenAndServe开始。
它创建了一个server对象，并调用server对象的ListenAndServe方法。
server的结构如下：
// A Server defines parameters for running an HTTP server.
// The zero value for Server is a valid configuration.
type Server struct {
// Addr optionally specifies the TCP address for the server to listen on,
// in the form "host:port". If empty, ":http" (port 80) is used.
// The service names are defined in RFC 6335 and assigned by IANA.
// See net.Dial for details of the address format.
Addr string
Handler Handler // handler to invoke, http.DefaultServeMux if nil
// TLSConfig optionally provides a TLS configuration for use
// by ServeTLS and ListenAndServeTLS. Note that this value is
// cloned by ServeTLS and ListenAndServeTLS, so it's not
// possible to modify the configuration with methods like
// tls.Config.SetSessionTicketKeys. To use
// SetSessionTicketKeys, use Server.Serve with a TLS Listener
// instead.
TLSConfig *tls.Config
// ReadTimeout is the maximum duration for reading the entire
// request, including the body.
//
// Because ReadTimeout does not let Handlers make per-request
// decisions on each request body's acceptable deadline or
// upload rate, most users will prefer to use
// ReadHeaderTimeout. It is valid to use them both.
ReadTimeout time.Duration
// ReadHeaderTimeout is the amount of time allowed to read
// request headers. The connection's read deadline is reset
// after reading the headers and the Handler can decide what
// is considered too slow for the body. If ReadHeaderTimeout
// is zero, the value of ReadTimeout is used. If both are
// zero, there is no timeout.
ReadHeaderTimeout time.Duration
// WriteTimeout is the maximum duration before timing out
// writes of the response. It is reset whenever a new
// request's header is read. Like ReadTimeout, it does not
// let Handlers make decisions on a per-request basis.
WriteTimeout time.Duration
// IdleTimeout is the maximum amount of time to wait for the
// next request when keep-alives are enabled. If IdleTimeout
// is zero, the value of ReadTimeout is used. If both are
// zero, there is no timeout.
IdleTimeout time.Duration
// MaxHeaderBytes controls the maximum number of bytes the
// server will read parsing the request header's keys and
// values, including the request line. It does not limit the
// size of the request body.
// If zero, DefaultMaxHeaderBytes is used.
MaxHeaderBytes int
// TLSNextProto optionally specifies a function to take over
// ownership of the provided TLS connection when an ALPN
// protocol upgrade has occurred. The map key is the protocol
// name negotiated. The Handler argument should be used to
// handle HTTP requests and will initialize the Request's TLS
// and RemoteAddr if not already set. The connection is
// automatically closed when the function returns.
// If TLSNextProto is not nil, HTTP/2 support is not enabled
// automatically.
TLSNextProto map[string]func(*Server, *tls.Conn, Handler)
// ConnState specifies an optional callback function that is
// called when a client connection changes state. See the
// ConnState type and associated constants for details.
ConnState func(net.Conn, ConnState)
// ErrorLog specifies an optional logger for errors accepting
// connections, unexpected behavior from handlers, and
// underlying FileSystem errors.
// If nil, logging is done via the log package's standard logger.
ErrorLog *log.Logger
// BaseContext optionally specifies a function that returns
// the base context for incoming requests on this server.
// The provided Listener is the specific Listener that's
// about to start accepting requests.
// If BaseContext is nil, the default is context.Background().
// If non-nil, it must return a non-nil context.
BaseContext func(net.Listener) context.Context
// ConnContext optionally specifies a function that modifies
// the context used for a new connection c. The provided ctx
// is derived from the base context and has a ServerContextKey
// value.
ConnContext func(ctx context.Context, c net.Conn) context.Context
inShutdown atomicBool // true when when server is in shutdown
disableKeepAlives int32 // accessed atomically.
nextProtoOnce sync.Once // guards setupHTTP2_* init
nextProtoErr error // result of http2.ConfigureServer if used
mu sync.Mutex
listeners map[*net.Listener]struct{}
activeConn map[*conn]struct{}
doneChan chan struct{}
onShutdown []func()
}
server结构存储了服务器处理请求常见的字段。其中Handler字段也保留Handler接口。如果Server接口没有提供Handler结构对象，那么会使用DefautServeMux做multiplexer。
Server的ListenAndServe方法中，会初始化监听地址Addr，同时调用Listen方法设置监听。最后将监听的TCP对象传入Serve方法。
// Serve accepts incoming connections on the Listener l, creating a
// new service goroutine for each. The service goroutines read requests and
// then call srv.Handler to reply to them.
//
// HTTP/2 support is only enabled if the Listener returns *tls.Conn
// connections and they were configured with "h2" in the TLS
// Config.NextProtos.
//
// Serve always returns a non-nil error and closes l.
// After Shutdown or Close, the returned error is ErrServerClosed.
func (srv *Server) Serve(l net.Listener) error {
if fn := testHookServerServe; fn != nil {
fn(srv, l) // call hook with unwrapped listener
}
origListener := l
l = &onceCloseListener{Listener: l}
defer l.Close()
if err := srv.setupHTTP2_Serve(); err != nil {
return err
}
if !srv.trackListener(&l, true) {
return ErrServerClosed
}
defer srv.trackListener(&l, false)
baseCtx := context.Background()
if srv.BaseContext != nil {
baseCtx = srv.BaseContext(origListener)
if baseCtx == nil {
panic("BaseContext returned a nil context")
}
}
var tempDelay time.Duration // how long to sleep on accept failure
ctx := context.WithValue(baseCtx, ServerContextKey, srv)
for {
rw, err := l.Accept()
if err != nil {
select {
case <-srv.getDoneChan():
return ErrServerClosed
default:
}
if ne, ok := err.(net.Error); ok && ne.Temporary() {
if tempDelay == 0 {
tempDelay = 5 * time.Millisecond
} else {
tempDelay *= 2
}
if max := 1 * time.Second; tempDelay > max {
tempDelay = max
}
srv.logf("http: Accept error: %v; retrying in %v", err, tempDelay)
time.Sleep(tempDelay)
continue
}
return err
}
connCtx := ctx
if cc := srv.ConnContext; cc != nil {
connCtx = cc(connCtx, rw)
if connCtx == nil {
panic("ConnContext returned nil")
}
}
tempDelay = 0
c := srv.newConn(rw)
c.setState(c.rwc, StateNew) // before Serve can return
go c.serve(connCtx)
}
}
监听开启之后，一旦客户端请求到底，go就开启一个协程处理请求，主要逻辑都在serve方法之中。
// Serve a new connection.
func (c *conn) serve(ctx context.Context) {
c.remoteAddr = c.rwc.RemoteAddr().String()
ctx = context.WithValue(ctx, LocalAddrContextKey, c.rwc.LocalAddr())
defer func() {
if err := recover(); err != nil && err != ErrAbortHandler {
const size = 64 << 10
buf := make([]byte, size)
buf = buf[:runtime.Stack(buf, false)]
c.server.logf("http: panic serving %v: %v\n%s", c.remoteAddr, err, buf)
}
if !c.hijacked() {
c.close()
c.setState(c.rwc, StateClosed)
}
}()
if tlsConn, ok := c.rwc.(*tls.Conn); ok {
if d := c.server.ReadTimeout; d != 0 {
c.rwc.SetReadDeadline(time.Now().Add(d))
}
if d := c.server.WriteTimeout; d != 0 {
c.rwc.SetWriteDeadline(time.Now().Add(d))
}
if err := tlsConn.Handshake(); err != nil {
// If the handshake failed due to the client not speaking
// TLS, assume they're speaking plaintext HTTP and write a
// 400 response on the TLS conn's underlying net.Conn.
if re, ok := err.(tls.RecordHeaderError); ok && re.Conn != nil && tlsRecordHeaderLooksLikeHTTP(re.RecordHeader) {
io.WriteString(re.Conn, "HTTP/1.0 400 Bad Request\r\n\r\nClient sent an HTTP request to an HTTPS server.\n")
re.Conn.Close()
return
}
c.server.logf("http: TLS handshake error from %s: %v", c.rwc.RemoteAddr(), err)
return
}
c.tlsState = new(tls.ConnectionState)
*c.tlsState = tlsConn.ConnectionState()
if proto := c.tlsState.NegotiatedProtocol; validNextProto(proto) {
if fn := c.server.TLSNextProto[proto]; fn != nil {
h := initALPNRequest{ctx, tlsConn, serverHandler{c.server}}
fn(c.server, tlsConn, h)
}
return
}
}
// HTTP/1.x from here on.
ctx, cancelCtx := context.WithCancel(ctx)
c.cancelCtx = cancelCtx
defer cancelCtx()
c.r = &connReader{conn: c}
c.bufr = newBufioReader(c.r)
c.bufw = newBufioWriterSize(checkConnErrorWriter{c}, 4<<10)
for {
w, err := c.readRequest(ctx)
if c.r.remain != c.server.initialReadLimitSize() {
// If we read any bytes off the wire, we're active.
c.setState(c.rwc, StateActive)
}
if err != nil {
const errorHeaders = "\r\nContent-Type: text/plain; charset=utf-8\r\nConnection: close\r\n\r\n"
switch {
case err == errTooLarge:
// Their HTTP client may or may not be
// able to read this if we're
// responding to them and hanging up
// while they're still writing their
// request. Undefined behavior.
const publicErr = "431 Request Header Fields Too Large"
fmt.Fprintf(c.rwc, "HTTP/1.1 "+publicErr+errorHeaders+publicErr)
c.closeWriteAndWait()
return
case isUnsupportedTEError(err):
// Respond as per RFC 7230 Section 3.3.1 which says,
// A server that receives a request message with a
// transfer coding it does not understand SHOULD
// respond with 501 (Unimplemented).
code := StatusNotImplemented
// We purposefully aren't echoing back the transfer-encoding's value,
// so as to mitigate the risk of cross side scripting by an attacker.
fmt.Fprintf(c.rwc, "HTTP/1.1 %d %s%sUnsupported transfer encoding", code, StatusText(code), errorHeaders)
return
case isCommonNetReadError(err):
return // don't reply
default:
publicErr := "400 Bad Request"
if v, ok := err.(badRequestError); ok {
publicErr = publicErr + ": " + string(v)
}
fmt.Fprintf(c.rwc, "HTTP/1.1 "+publicErr+errorHeaders+publicErr)
return
}
}
// Expect 100 Continue support
req := w.req
if req.expectsContinue() {
if req.ProtoAtLeast(1, 1) && req.ContentLength != 0 {
// Wrap the Body reader with one that replies on the connection
req.Body = &expectContinueReader{readCloser: req.Body, resp: w}
w.canWriteContinue.setTrue()
}
} else if req.Header.get("Expect") != "" {
w.sendExpectationFailed()
return
}
c.curReq.Store(w)
if requestBodyRemains(req.Body) {
registerOnHitEOF(req.Body, w.conn.r.startBackgroundRead)
} else {
w.conn.r.startBackgroundRead()
}
// HTTP cannot have multiple simultaneous active requests.[*]
// Until the server replies to this request, it can't read another,
// so we might as well run the handler in this goroutine.
// [*] Not strictly true: HTTP pipelining. We could let them all process
// in parallel even if their responses need to be serialized.
// But we're not going to implement HTTP pipelining because it
// was never deployed in the wild and the answer is HTTP/2.
serverHandler{c.server}.ServeHTTP(w, w.req)
w.cancelCtx()
if c.hijacked() {
return
}
w.finishRequest()
if !w.shouldReuseConnection() {
if w.requestBodyLimitHit || w.closedRequestBodyEarly() {
c.closeWriteAndWait()
}
return
}
c.setState(c.rwc, StateIdle)
c.curReq.Store((*response)(nil))
if !w.conn.server.doKeepAlives() {
// We're in shutdown mode. We might've replied
// to the user without "Connection: close" and
// they might think they can send another
// request, but such is life with HTTP/1.1.
return
}
if d := c.server.idleTimeout(); d != 0 {
c.rwc.SetReadDeadline(time.Now().Add(d))
if _, err := c.bufr.Peek(4); err != nil {
return
}
}
c.rwc.SetReadDeadline(time.Time{})
}
}
serve方法比较长，其主要职能就是，创建一个上下文对象，然后调用Listener的Accept方法用来 获取连接数据并使用newConn方法创建连接对象。最后使用goroutein协程的方式处理连接请求。因为每一个连接都开起了一个协程，请求的上下文都不同，同时又保证了go的高并发。
使用defer定义了函数退出时，连接关闭相关的处理。然后就是读取连接的网络数据，并处理读取完毕时候的状态。接下来就是调用serverHandler{c.server}.ServeHTTP(w, w.req)方法处理请求了。
serverHandler是一个重要的结构，它实现了Handler接口方法ServeHTTP，并在该接口方法中初始化multiplexer路由多路复用器。如果server对象没有指定Handler，则使用默认的DefaultServeMux作为路由Multiplexer。并调用初始化Handler的ServeHTTP方法。
这里DefaultServeMux的ServeHTTP方法其实也是定义在ServeMux结构中的，相关代码如下：
mux的ServeHTTP方法通过调用其Handler方法寻找注册到路由上的handler函数，并调用该函数的ServeHTTP方法。
mux的Handler方法对URL简单的处理，然后调用handler方法，后者会创建一个锁，同时调用match方法返回一个handler和pattern。
在match方法中，mux的m字段是map[string]muxEntry图，后者存储了pattern和handler处理器函数，因此通过迭代m寻找出注册路由的patten模式与实际url匹配的handler函数并返回。
上述返回的结构一直传递到mux的ServeHTTP方法，接下来调用handler函数的ServeHTTP方法，即IndexHandler函数，然后把response写到http.RequestWirter对象返回给客户端。
至此，Golang中一个完整的http服务就基本完成了，包括注册路由，开启监听，处理连接，路由处理函数。
多数的web应用都是基于HTTP协议的，客户端和服务器通过request-response的方式交互。路由注册和连接处理是一个server不可缺少的两部分。Golang通过一个ServeMux实现了的multiplexer路由多路复用器来管理路由。同时提供一个Handler接口提供ServeHTTP，用于实现handler处理函数，后者可以处理实际request并构造response。
ServeMux和handler处理器函数的连接桥梁就是Handler接口。ServeMux的ServeHTTP方法实现了寻找注册路由的handler的函数，并调用该handler的ServeHTTP方法。ServeHTTP方法就是真正处理请求和构造响应的地方。
272
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
