Golang 开启黑盒测试

最新推荐文章于 2023-11-27 18:31:15 发布

u013433591

最新推荐文章于 2023-11-27 18:31:15 发布

阅读量195

点赞数 3

分类专栏： Golang 文章标签： golang 开发语言后端

本文链接：https://blog.csdn.net/u013433591/article/details/126277683

版权

Golang 专栏收录该内容

21 篇文章 1 订阅

订阅专栏

Golang 开启黑盒测试

黑盒测试必要性

单元测试具有局限性，因为单元测试用例的输入是开发人员增加的。黑盒测试的好处是可以识别并增加边缘性的测试case。

测试Demo

开发代码

package main

import "fmt"

func main() {
	input := "The quick brown fox jumped over the lazy dog"
	rev := Reverse(input)
	doubleRev := Reverse(rev)
	fmt.Printf("original: %q\n", input)
	fmt.Printf("reversed: %q\n", rev)
	fmt.Printf("reversed again: %q\n", doubleRev)
}

// Reverse 字符串逆序函数
func Reverse(s string) string {
	b := []byte(s)
	for i, j := 0, len(b)-1; i < len(b)/2; i, j = i+1, j-1 {
		b[i], b[j] = b[j], b[i]
	}
	return string(b)
}

单元测试

在同级目录下新增 reverse_test.go 文件代码如下

package main

import (
    "testing"
)

func TestReverse(t *testing.T) {
    testcases := []struct {
        in, want string
    }{
        {"Hello, world", "dlrow ,olleH"},
        {" ", " "},
        {"!12345", "54321!"},
    }
    for _, tc := range testcases {
        rev := Reverse(tc.in)
        if rev != tc.want {
                t.Errorf("Reverse: %q, want %q", rev, tc.want)
        }
    }
}

进行单元测试输出如下

$ go test
PASS
ok      example/fuzz  0.013s

黑盒测试

在reverse_test.go文件里面增加以下函数，golang中的黑盒测试同样也存在部分限制：

单元测试中需要对输出结果做出判断并验证是否跟预期一致, 如 Hello 的输出结果为 olleH
黑盒测试不能判断期望的输出结果因为无法控制输入参数

无论如何，针对Reverse函数也有一些基本的属性可以进行验证，如下两点:

反转2次后需要跟原始的数据一致
UTF-8编码的字符串需要正常执行

需要注意黑盒测试的语法跟单元测试也有一点不同：

函数名称从 TestXxx(单元测试) 变成 FuzzXxx(黑盒测试)，参数由 *testing.T 变成 *testing.F
在期望执行 t.Run 的地方，将会有 f.Fuzz 执行目标函数。由 f.Add 函数将输入放入到待测试的数据集中

func FuzzReverse(f *testing.F) {
    testcases := []string{"Hello, world", " ", "!12345"}
    for _, tc := range testcases {
        f.Add(tc)  // Use f.Add to provide a seed corpus
    }
    f.Fuzz(func(t *testing.T, orig string) {
        rev := Reverse(orig)
        doubleRev := Reverse(rev)
        if orig != doubleRev {
            t.Errorf("Before: %q, after: %q", orig, doubleRev)
        }
        if utf8.ValidString(orig) && !utf8.ValidString(rev) {
            t.Errorf("Reverse produced invalid UTF-8 string %q", rev)
        }
    })			
}

执行黑盒测试，发现并没有执行成功，中途报错。

PS C:\Users\User36\GolandProjects\awesomeProject\fuzz\day01> go test -fuzz=Fuzz
fuzz: elapsed: 0s, gathering baseline coverage: 0/3 completed
fuzz: elapsed: 0s, gathering baseline coverage: 3/3 completed, now fuzzing with 8 workers
fuzz: minimizing 34-byte failing input file
fuzz: elapsed: 0s, minimizing
--- FAIL: FuzzReverse (0.09s)
    --- FAIL: FuzzReverse (0.00s)
        reverse_test.go:36: Reverse produced invalid UTF-8 string "\xb6\xd5"
    
    Failing input written to testdata\fuzz\FuzzReverse\8fd8b02db1ce4b6cd20b03c7447f9b21a278f5931241b61e0437b14b7e5a6b0e
    To re-run:
    go test -run=FuzzReverse/8fd8b02db1ce4b6cd20b03c7447f9b21a278f5931241b61e0437b14b7e5a6b0e
FAIL
exit status 1
FAIL    awesomeProject/fuzz/day01       0.143s

在当前目录下 testdata/fuzz/FuzzReverse/8fd8b02db1ce4b6cd20b03c7447f9b21a278f5931241b61e0437b14b7e5a6b0e文件，内容:

go test fuzz v1
string("泃")

再次执行单元测试命令，发现也是失败的，golang的内部机制决定了只要测试失败应该Debug问题

PS C:\Users\User36\GolandProjects\awesomeProject\fuzz\day01> go test
--- FAIL: FuzzReverse (0.00s)
    --- FAIL: FuzzReverse/8fd8b02db1ce4b6cd20b03c7447f9b21a278f5931241b61e0437b14b7e5a6b0e (0.00s)
        reverse_test.go:36: Reverse produced invalid UTF-8 string "\xb6\xd5"
FAIL
exit status 1
FAIL    awesomeProject/fuzz/day01       0.031s

问题跟踪

考虑到字符串反转函数的输入有可能是中文，因此需要做一些调整，代码如下

func Reverse(s string) string {
    fmt.Printf("input: %q\n", s)
    r := []rune(s)
    fmt.Printf("runes: %q\n", r)
    for i, j := 0, len(r)-1; i < len(r)/2; i, j = i+1, j-1 {
        r[i], r[j] = r[j], r[i]
    }
    return string(r)
}

利用之前的测试用例，再次执行发现已经解决了，完美

PS C:\Users\User36\GolandProjects\awesomeProject\fuzz\day01> go test -run=FuzzReverse/8fd8b02db1ce4b6cd20b03c7447f9b21a278f5931241b61e0437b14b7e5a6b0e
input: "ն"
runes: ['ն']
input: "ն"
runes: ['ն']
PASS
ok      awesomeProject/fuzz/day01       0.032s

等等不要高兴的太早了，再次执行黑盒测试脚本，特么的问题又出现了, 日志显示还是一些UTF8编码的字符串搞得鬼

PS C:\Users\User36\GolandProjects\awesomeProject\fuzz\day01> go test -fuzz=Fuzz
input: "Hello, world"
runes: ['H' 'e' 'l' 'l' 'o' ',' ' ' 'w' 'o' 'r' 'l' 'd']      
input: " "                                                    
runes: [' ']                                                  
input: "!12345"                                               
runes: ['!' '1' '2' '3' '4' '5']                              
fuzz: elapsed: 0s, gathering baseline coverage: 0/10 completed
fuzz: minimizing 46-byte failing input file                                                                          
failure while testing seed corpus entry: FuzzReverse/0df90bdec6befb47c1c0a8064e9af09e83c4f50f61c948026f3a46152302792d
fuzz: elapsed: 0s, gathering baseline coverage: 4/10 completed
--- FAIL: FuzzReverse (0.05s)                         
    --- FAIL: FuzzReverse (0.00s)                     
        reverse_test.go:33: Before: "\xbc", after: "�"
                                                      
FAIL                                                  
exit status 1                                         
FAIL    awesomeProject/fuzz/day01       0.079s

再次修改 Reverse 函数，判断为UTF8的字符串直接返回注意修改后的函数返回了两个值了

func Reverse(s string) (string, error) {
    if !utf8.ValidString(s) {
        return s, errors.New("input is not valid UTF-8")
    }
    r := []rune(s)
    for i, j := 0, len(r)-1; i < len(r)/2; i, j = i+1, j-1 {
        r[i], r[j] = r[j], r[i]
    }
    return string(r), nil
}

再次执行黑盒测试脚本,发现会一直不停的执行下去，知道按住ctrl-C中断执行

PS C:\Users\User36\GolandProjects\awesomeProject\fuzz\day01> go test -fuzz=Fuzz
fuzz: elapsed: 0s, gathering baseline coverage: 0/10 completed
fuzz: elapsed: 0s, gathering baseline coverage: 10/10 completed, now fuzzing with 8 workers
fuzz: elapsed: 3s, execs: 391042 (130032/sec), new interesting: 25 (total: 35)
fuzz: elapsed: 6s, execs: 1068358 (226007/sec), new interesting: 27 (total: 37)
fuzz: elapsed: 9s, execs: 1574658 (167653/sec), new interesting: 27 (total: 37)
fuzz: elapsed: 12s, execs: 2242408 (223348/sec), new interesting: 28 (total: 38)

也可以执行黑盒测试时指定测试时间

PS C:\Users\User36\GolandProjects\awesomeProject\fuzz\day01> go test -fuzz=Fuzz -fuzztime 10s
fuzz: elapsed: 0s, gathering baseline coverage: 0/42 completed
fuzz: elapsed: 0s, gathering baseline coverage: 42/42 completed, now fuzzing with 8 workers
fuzz: elapsed: 3s, execs: 601251 (200011/sec), new interesting: 1 (total: 43)
fuzz: elapsed: 6s, execs: 1208050 (202093/sec), new interesting: 1 (total: 43)
fuzz: elapsed: 9s, execs: 1810927 (200932/sec), new interesting: 1 (total: 43)
fuzz: elapsed: 10s, execs: 2031601 (201306/sec), new interesting: 1 (total: 43)
PASS
ok      awesomeProject/fuzz/day01       10.165s

最终代码

main.go

package main

import (
	"errors"
	"fmt"
	"unicode/utf8"
)

func main() {
	input := "The quick brown fox jumped over the lazy dog"
	rev, _ := Reverse(input)
	doubleRev, _ := Reverse(rev)
	fmt.Printf("original: %q\n", input)
	fmt.Printf("reversed: %q\n", rev)
	fmt.Printf("reversed again: %q\n", doubleRev)
}

// Reverse 字符串逆序函数
func Reverse(s string) (string, error) {
	if !utf8.ValidString(s) {
		return s, errors.New("input is not valid UTF-8")
	}
	r := []rune(s)
	for i, j := 0, len(r)-1; i < len(r)/2; i, j = i+1, j-1 {
		r[i], r[j] = r[j], r[i]
	}
	return string(r), nil
}

reverse_test.go

package main

import (
	"testing"
	"unicode/utf8"
)

func TestReverse(t *testing.T) {
	testcases := []struct {
		in, want string
	}{
		{"Hello, world", "dlrow ,olleH"},
		{" ", " "},
		{"!12345", "54321!"},
	}
	for _, tc := range testcases {
		rev, _ := Reverse(tc.in)
		if rev != tc.want {
			t.Errorf("Reverse: %q, want %q", rev, tc.want)
		}
	}
}

func FuzzReverse(f *testing.F) {
	testcases := []string{"Hello, world", " ", "!12345"}
	for _, tc := range testcases {
		f.Add(tc) // Use f.Add to provide a seed corpus
	}
	f.Fuzz(func(t *testing.T, orig string) {
		rev, _ := Reverse(orig)
		doubleRev, _ := Reverse(rev)
		if orig != doubleRev {
			t.Errorf("Before: %q, after: %q", orig, doubleRev)
		}
		if utf8.ValidString(orig) && !utf8.ValidString(rev) {
			t.Errorf("Reverse produced invalid UTF-8 string %q", rev)
		}
	})
}

总结一下，确实比单元测试香多了