golang笔记13--go语言 http 及其它标准库
1 介绍
本文继上文 golang笔记12–迷宫的广度优先搜索, 进一步了解 go 语言 http 及其它标准库,以及相应注意事项。
具体包括: http 标准库、其它标准库、gin 框架介绍、为 gin 增加 middleware 等内容。
2 http及其它标准库
2.1 http 标准库
go http 库功能比较完善,此处通过服务端 和 客户端两个层面进行案例说明。
服务端:
vim 13.1.1.go
package main
import (
"fmt"
"html"
"net/http"
_ "net/http/pprof"
"sync"
)
type countHandler struct {
mu sync.Mutex // guards n
n int
}
func (h *countHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
h.mu.Lock()
defer h.mu.Unlock()
h.n++
fmt.Fprintf(w, "count is %d\n", h.n)
}
func main() {
http.Handle("/foo", new(countHandler))
http.HandleFunc("/bar", func(w http.ResponseWriter, r *http.Request) {
fmt.Fprintf(w, "Hello, %q", html.EscapeString(r.URL.Path))
})
http.ListenAndServe(":8888", nil)
}
输出:
http://127.0.0.1:8888/foo
1 (多次访问,其值逐渐增加)
http://127.0.0.1:8888/debug/pprof 可以访问其各种性能数据,具体如下图;
对于较复杂的http服务,可以通过命令行pprof来收集数据,然后通过其 pprof 的 web 功能输出各个模块调用时间关系图;具体使用方式如下(30s 采集完成后可以通过交互界面的web功能输出对于关系图):
chapter13/13.1-1$ go tool pprof http://localhost:8888/debug/pprof/profile
Fetching profile over HTTP from http://localhost:8888/debug/pprof/profile
Saved profile in /home/xg/pprof/pprof.___go_build_learngo_chapter13_13_1_1.samples.cpu.001.pb.gz
File: ___go_build_learngo_chapter13_13_1_1
Type: cpu
Time: Feb 18, 2021 at 1:07pm (CST)
Duration: 30s, Total samples = 0
No samples were found with the default sample value type.
Try "sample_index" command to analyze different sample values.
Entering interactive mode (type "help" for commands, "o" for options)
(pprof) web
(pprof) quit
客户端:
package main
import (
"fmt"
"net/http"
"net/http/httputil"
)
func serverV1() {
resp, err := http.Get("http://www.imooc.com")
if err != nil {
panic(err)
}
defer resp.Body.Close()
s, err := httputil.DumpResponse(resp, true)
if err != nil {
panic(err)
}
fmt.Printf("%s\n", s)
}
func serverV2() {
// 获取手机版本的网页信息
request, err := http.NewRequest(http.MethodGet, "http://www.imooc.com", nil)
request.Header.Add("User-Agent", "Mozilla/5.0 (iPhone; CPU iPhone OS 13_2_3 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/13.0.3 Mobile/15E148 Safari/604.1")
resp, err := http.DefaultClient.Do(request)
if err != nil {
panic(err)
}
defer resp.Body.Close()
s, err := httputil.DumpResponse(resp, true)
if err != nil {
panic(err)
}
fmt.Printf("%s\n", s)
}
func main() {
// serverV1()
serverV2()
}
输出:
serverV1 输出PC网页版本主页信息,内容过多此处省略
serverV2 输出手机网页版本主页信息,内容过多此处省略
2.2 其它标准库
go 语言中包含很多常见标准库,例如 bufio、log、encoding/json、regexp、time、strings|math|rand 等;可以直接在官网文档查看,也可以通过 godoc -http :8080 起一个本地文档服务器查看文档。
package main
import (
"bufio"
"encoding/json"
"fmt"
"io"
"log"
"os"
"regexp"
"strings"
)
func testBufio() {
// 打印输入的每行字符串
scanner := bufio.NewScanner(os.Stdin)
for scanner.Scan() {
fmt.Println(scanner.Text()) // Println will add back the final '\n'
}
if err := scanner.Err(); err != nil {
fmt.Fprintln(os.Stderr, "reading standard input:", err)
}
}
func testLog() {
fmt.Println("test log")
log.Print("test log")
}
func testJson() {
fmt.Println("test json")
const jsonStream = `
{"Name": "Ed", "Text": "Knock knock."}
{"Name": "Sam", "Text": "Who's there?"}
{"Name": "Ed", "Text": "Go fmt yourself!"}`
type Message struct {
Name, Text string
}
dec := json.NewDecoder(strings.NewReader(jsonStream))
for {
var m Message
if err := dec.Decode(&m); err == io.EOF {
break
} else if err != nil {
log.Fatal(err)
}
fmt.Printf("%s: %s\n", m.Name, m.Text)
}
}
func testRegexp() {
fmt.Println("test regexp")
matched, err := regexp.MatchString(`foo.*`, "seafood")
fmt.Println(matched, err)
matched, err = regexp.MatchString(`bar.*`, "seafood")
fmt.Println(matched, err)
matched, err = regexp.MatchString(`a(b`, "seafood")
fmt.Println(matched, err)
}
func main() {
//testBufio()
testLog()
testJson()
testRegexp()
}
输出:
test log
2021/02/18 14:23:52 test log
test json
Ed: Knock knock.
Sam: Who's there?
Ed: Go fmt yourself!
test regexp
true <nil>
false <nil>
false error parsing regexp: missing closing ): `a(b`
2.3 json 数据格式处理
go 语言中可以通过 json.Marshal 将结构体转换为 json 字符串;也可以通过 json.Unmarshal 将json字符串转为具体的结构体对象。
需要注意的是结构体多为首字母大写的,而实际中习惯将json 首字母小写,因此需要通过 json 的标签来标识 json 的小写 key;如需要丢弃为空的key,则可以在 json的标签后加 omitempty 。
package main
import (
"encoding/json"
"fmt"
)
type Order struct {
ID string
Name string
Quantity int
TotalPrice float64
}
func test01() {
fmt.Println("test01")
o := Order{
ID: "1234",
Name: "learn go",
Quantity: 3,
TotalPrice: 30,
}
fmt.Printf("%v\n%+v\n", o, o)
b, err := json.Marshal(o)
if err != nil {
panic(err)
}
fmt.Printf("%s\n", b)
}
type Order2 struct {
// 使用json 的tag,可以大写转为tag中的小写
ID string `json:"id"`
Name string `json:"name,omitempty"`
Quantity int `json:"quantity"`
TotalPrice float64 `json:"total_price"`
}
func test02() {
fmt.Println("test02")
o := Order2{
ID: "1234",
Name: "learn go",
Quantity: 3,
TotalPrice: 30,
}
b, err := json.Marshal(o)
if err != nil {
panic(err)
}
fmt.Printf("%s\n", b)
// 默认情况下省略某个key后,json中会输出该key的空值;使用omitempty后,当省略某个key的时候,json中不会输出该key
o2 := Order2{
ID: "1234",
Quantity: 3,
TotalPrice: 30,
}
b2, err2 := json.Marshal(o2)
if err2 != nil {
panic(err2)
}
fmt.Printf("%s\n", b2)
}
type OrderItem struct {
ID string `json:"id"`
Name string `json:"name"`
Price float64 `json:"price"`
}
type Order3 struct {
ID string `json:"id"`
Item []OrderItem `json:"item"`
Quantity int `json:"quantity"`
TotalPrice float64 `json:"total_price"`
}
func test03() {
fmt.Println("test03")
o := Order3{
ID: "1234",
Item: []OrderItem{
{
ID: "item_1",
Name: "learn go",
Price: 15,
},
{
ID: "item_2",
Name: "learn python",
Price: 16,
},
},
Quantity: 3,
TotalPrice: 30,
}
b, err := json.Marshal(o)
if err != nil {
panic(err)
}
fmt.Printf("%s\n", b)
}
func unmarshal() {
fmt.Println("test unmarshal")
s := `{"id":"1234","item":[{"id":"item_1","name":"learn go","price":15},{"id":"item_2","name":"learn python","price":16}],"quantity":3,"total_price":30}`
var o Order3
err := json.Unmarshal([]byte(s), &o)
if err != nil {
panic(err)
}
fmt.Printf("%+v", o)
}
func main() {
test01()
test02()
test03()
unmarshal()
}
输出:
test01
{1234 learn go 3 30}
{ID:1234 Name:learn go Quantity:3 TotalPrice:30}
{"ID":"1234","Name":"learn go","Quantity":3,"TotalPrice":30}
test02
{"id":"1234","name":"learn go","quantity":3,"total_price":30}
{"id":"1234","quantity":3,"total_price":30}
test03
{"id":"1234","item":[{"id":"item_1","name":"learn go","price":15},{"id":"item_2","name":"learn python","price":16}],"quantity":3,"total_price":30}
test unmarshal
{ID:1234 Item:[{ID:item_1 Name:learn go Price:15} {ID:item_2 Name:learn python Price:16}] Quantity:3 TotalPrice:30}
2.4 解析第三方api数据格式技巧
解析其它应用api 的 json数据时候,我们可以使用map 和 自定义结构体两种方式。具体案例如下:
package main
import (
"encoding/json"
"fmt"
)
var res = `{
"data": [
{
"synonym":"",
"weight":"0.6",
"word": "真丝",
"tag":"材质"
},
{
"synonym":"",
"weight":"0.8",
"word": "韩都衣舍",
"tag":"品牌"
},
{
"synonym":"连身裙;联衣裙",
"weight":"1.0",
"word": "连衣裙",
"tag":"品类"
}
]
}`
func parseNLP() {
// 解析json的方法:使用map
fmt.Println("parseNLP1")
m := make(map[string]interface{})
err := json.Unmarshal([]byte(res), &m)
if err != nil {
panic(err)
}
fmt.Printf("%+v\n", m)
// 获取第二个item 的 synonym 字段, 1)获取data,并告知其为切片;2)拿到第2个item,并告知其为map,获取其指定字段 "synonym"
fmt.Printf("%+v\n", m["data"].([]interface{})[2].(map[string]interface{})["synonym"])
}
func parseNLP2() {
// 解析json的方法:定义一个对应的结构体,
fmt.Println("parseNLP2")
m := struct {
Data []struct {
Synonym string `json:"synonym"`
Tag string `json:"tag"`
} `json:"data"`
}{}
err := json.Unmarshal([]byte(res), &m)
if err != nil {
panic(err)
}
fmt.Printf("%+v %+v\n", m.Data[2].Synonym, m.Data[2].Tag)
}
func main() {
parseNLP()
parseNLP2()
}
输出:
parseNLP1
map[data:[map[synonym: tag:材质 weight:0.6 word:真丝] map[synonym: tag:品牌 weight:0.8 word:韩都衣舍] map[synonym:连身裙;联衣裙 tag:品类 裙]]]
连身裙;联衣裙
parseNLP2
连身裙;联衣裙 品类
2.5 gin 框架介绍
Gin 是使用go写的一个web 框架,其具备很高性能,比 httprouter 快40倍。
1) 安装方法
go get -u github.com/gin-gonic/gin
2) 案例
vim 13.3.go
package main
import "github.com/gin-gonic/gin"
func main() {
r := gin.Default()
r.GET("/ping", func(c *gin.Context) {
c.JSON(200, gin.H{"message": "pong"})
})
r.Run(":8888")
}
输出:
http://127.0.0.1:8888/ping
{"message": "pong"}
2.6 为 gin 增加 middleware
本案例使用 middleware ,对每个请求都打印相关的日志,并记录每次请求所发的时间;
本案例依赖 gin 和 zap 的log模块
go get -u github.com/gin-gonic/gin
go get -u go.uber.org/zap
vim 13.4.go
package main
import (
"math/rand"
"time"
"github.com/gin-gonic/gin"
"go.uber.org/zap"
)
const keyRequestId = "requestId"
func main() {
r := gin.Default()
logger, err := zap.NewProduction()
if err != nil {
panic(err)
}
r.Use(func(c *gin.Context) {
s := time.Now()
c.Next()
logger.Warn("incoming request", zap.String("path", c.Request.URL.Path), zap.Int("status", c.Writer.Status()), zap.Duration("elapsed", time.Now().Sub(s)))
}, func(c *gin.Context) {
c.Set(keyRequestId, rand.Int())
c.Next()
})
r.GET("/ping", func(c *gin.Context) {
h := gin.H{"message": "pong"}
if rid, exists := c.Get(keyRequestId); exists {
h[keyRequestId] = rid
}
c.JSON(200, h)
})
r.GET("/hello", func(c *gin.Context) {
c.String(200, "hello golang")
})
r.Run(":8888")
}
输出:
http://127.0.0.1:8888/ping
{"message": "pong","requestId": 605394647632969758}
http://127.0.0.1:8888/hello
hello golang
3 注意事项
- debug http 性能情况时,需要使用 _ “net/http/pprof” (前面需要添加一个下横杠,以确保该server 具备 pprof 等功能,不添加会导致保存时候被idea删除)
4 说明
- 软件环境
go版本:go1.15.8
操作系统:Ubuntu 20.04 Desktop
Idea:2020.01.04 - 参考文档
golang 轻量级框架 --Gin入门
github gin 官方文档