gibhub地址:针对negroni的gzip
作业内容:支持了gzip的服务器小程序
这是一个为Negroni设计的gzip压缩处理中间件,需要用到已有的compress中的gzip。源码共有一百多行并不多,下面记录一下对gzip源码的分析与理解。
首先gzip是干啥用的?gzip是一种压缩方式。浏览器和web服务器之间为了减少传输链路上的文件的大小,浏览器和服务器在传送数据的时候会将数据以某种方式进行压缩,使用什么压缩方式则记录在报文头里面。这里的gzip便是一种压缩方式。并不是所有的浏览器和服务器都能支持所有的压缩方式。比如说浏览器支持gzip压缩方式,即浏览器能够解析gzip压缩后的内容,那么他会在请求中发送Accept-Encoding请求报头值为"gzip" 表明浏览器支持gzip这种压缩方式,web服务器根据读取Accept-Encoding请求报头的值来判断浏览器是否接受压缩的内容,服务器发现浏览器能够解析gzip压缩后的内容之后就会对要发送的数据进行gzip压缩再发送到客户端,同时设置Content-Encoding实体报头值为gzip以告知浏览器实体正文采用了gzip的压缩编码。
那么这里的negroni-gzip就是一个中间件,用来使regroni搭建的服务器能够支持gzip压缩。
源码分析:
- const数据内容
// These compression constants are copied from the compress/gzip package. const ( encodingGzip = "gzip" headerAcceptEncoding = "Accept-Encoding" headerContentEncoding = "Content-Encoding" headerContentLength = "Content-Length" headerContentType = "Content-Type" headerVary = "Vary" headerSecWebSocketKey = "Sec-WebSocket-Key" BestCompression = gzip.BestCompression BestSpeed = gzip.BestSpeed DefaultCompression = gzip.DefaultCompression NoCompression = gzip.NoCompression )
上面的数据代表了接下来的代码中各种const常量代表的值。里面使用了compress/gzip里面的值,其中NoCompression = 0,BestSpeed = 1,BestCompression = 9,DefaultCompression = -1。这些值代表压缩的level,不能超BestCompression。
-
// gzipResponseWriter is the ResponseWriter that negroni.ResponseWriter is // wrapped in. type gzipResponseWriter struct { w *gzip.Writer negroni.ResponseWriter wroteHeader bool }
结构体gzipResponseWriter,wroteHeader代表response(即响应内容)是否已经编码 。
-
// Check whether underlying response is already pre-encoded and disable // gzipWriter before the body gets written, otherwise encoding headers func (grw *gzipResponseWriter) WriteHeader(code int) { headers := grw.ResponseWriter.Header() if headers.Get(headerContentEncoding) == "" { headers.Set(headerContentEncoding, encodingGzip) headers.Add(headerVary, headerAcceptEncoding) } else { grw.w.Reset(ioutil.Discard) grw.w = nil } // Avoid sending Content-Length header before compression. The length would // be invalid, and some browsers like Safari will report // "The network connection was lost." errors grw.Header().Del(headerContentLength) grw.ResponseWriter.WriteHeader(code) grw.wroteHeader = true }
WriteHeader函数,如果要发送给客户端的响应内容未预编码,则采用gzip压缩方式压缩后再发送到客户端,同时设置Content-Encoding实体报头值为gzip。否则在写之前令gzipWriter失效,使得它对任何写调用无条件成功。
-
// Write writes bytes to the gzip.Writer. It will also set the Content-Type // header using the net/http library content type detection if the Content-Type // header was not set yet. func (grw *gzipResponseWriter) Write(b []byte) (int, error) { if !grw.wroteHeader { grw.WriteHeader(http.StatusOK) } if grw.w == nil { return grw.ResponseWriter.Write(b) } if len(grw.Header().Get(headerContentType)) == 0 { grw.Header().Set(headerContentType, http.DetectContentType(b)) } return grw.w.Write(b) }
向gzip.Writer中写入字节流。如果报头没写的话就写报头,如果不是使用的gzip压缩的话就用ResponseWriter写。如果头的Content-Type还没有被设置,则用net/http库中的类型检测来完成设置。
-
type gzipResponseWriterCloseNotifier struct { *gzipResponseWriter }
一个简单的数据类型定义。
-
func (rw *gzipResponseWriterCloseNotifier) CloseNotify() <-chan bool { return rw.ResponseWriter.(http.CloseNotifier).CloseNotify() } func newGzipResponseWriter(rw negroni.ResponseWriter, w *gzip.Writer) negroni.ResponseWriter { wr := &gzipResponseWriter{w: w, ResponseWriter: rw} if _, ok := rw.(http.CloseNotifier); ok { return &gzipResponseWriterCloseNotifier{gzipResponseWriter: wr} } return wr }
gzipResponseWriter,用于写入gzip编码后的数据。
-
// handler struct contains the ServeHTTP method type handler struct { pool sync.Pool }
一个sync.Pool对象就是一组临时对象的集合,Pool用于存储那些被分配了但是没有被使用,而未来可能会使用的值,以减小垃圾回收的压力。
-
// Gzip returns a handler which will handle the Gzip compression in ServeHTTP. // Valid values for level are identical to those in the compress/gzip package. func Gzip(level int) *handler { h := &handler{} h.pool.New = func() interface{} { gz, err := gzip.NewWriterLevel(ioutil.Discard, level) if err != nil { panic(err) } return gz } return h }
Gzip返回一个handler来处理在ServeHTTP中的压缩,需要调用gzip库的NewWriterLevel方法。
-
// ServeHTTP wraps the http.ResponseWriter with a gzip.Writer. func (h *handler) ServeHTTP(w http.ResponseWriter, r *http.Request, next http.HandlerFunc) { // Skip compression if the client doesn't accept gzip encoding. if !strings.Contains(r.Header.Get(headerAcceptEncoding), encodingGzip) { next(w, r) return } // Skip compression if client attempt WebSocket connection if len(r.Header.Get(headerSecWebSocketKey)) > 0 { next(w, r) return } // Retrieve gzip writer from the pool. Reset it to use the ResponseWriter. // This allows us to re-use an already allocated buffer rather than // allocating a new buffer for every request. // We defer g.pool.Put here so that the gz writer is returned to the // pool if any thing after here fails for some reason (functions in // next could potentially panic, etc) gz := h.pool.Get().(*gzip.Writer) defer h.pool.Put(gz) gz.Reset(w) // Wrap the original http.ResponseWriter with negroni.ResponseWriter // and create the gzipResponseWriter. nrw := negroni.NewResponseWriter(w) grw := newGzipResponseWriter(nrw, gz) // Call the next handler supplying the gzipResponseWriter instead of // the original. next(grw, r) gz.Close() }
处理handler中压缩请求的函数:如果客户端不支持gzip编码则跳过并不压缩。如果客户端在尝试WebSocket连接时,也会不压缩。接下来从pool中遍历writer,如果之后遇到的错误,就通过defer的方法,返回pool,用ResponseWriter重置,这让我们可以再利用已经被分配的buffer,而不是为每一个单独的请求开辟新的buffer。最后用negroni.ResponseWriter打包原来的ResponseWriter,并创建一个新的gzipResponseWriter,并且调用下一个handler。最后关闭gz。