此前,我们提到了模型的使用包含了将它转换为gguf格式的步骤,后续的流程都是讨论基于gguf格式,如何实现推理。但是,我们并没有对于对于其中的细节深究。
那么,gguf格式的数据到底是如何转换得到的,保存的内容具体包含哪些呢?
可以关注 ggml/examples/yolo/convert-pth-to-ggml.py, 在main函数中,gguf_writer这个对象扮演了很关键的角色。
其中,save_conv2d_layer的作用很重要,它把tensor的信息存储到gguf_writer中,存储的信息包括: 1. 卷积层的名称 2. 卷积层的权重参数 3. 卷积层的尺寸。
在存储了conv层的信息后,gguf_writer写入header。kv data。最后才是tensor。可以看出来,下列代码存储了13个卷积层的参数,而实际上,对照Part3的网络结构,实际上模型主要就是包含13个卷积层的参数。
if __name__ == '__main__':
if len(sys.argv) != 2:
print("Usage: %s <yolov3-tiny.weights>" % sys.argv[0])
sys.exit(1)
outfile = 'yolov3-tiny.gguf'
gguf_writer = gguf.GGUFWriter(outfile, 'yolov3-tiny')
f = open(sys.argv[1], 'rb')
f.read(20) # skip header
save_conv2d_layer(f, gguf_writer, "l0", 3, 16, 3)
save_conv2d_layer(f, gguf_writer, "l1", 16, 32, 3)
save_conv2d_layer(f, gguf_writer, "l2", 32, 64, 3)
save_conv2d_layer(f, gguf_writer, "l3", 64, 128, 3)
save_conv2d_layer(f, gguf_writer, "l4", 128, 256, 3)
save_conv2d_layer(f, gguf_writer, "l5", 256, 512, 3)
save_conv2d_layer(f, gguf_writer, "l6", 512, 1024, 3)
save_conv2d_layer(f, gguf_writer, "l7", 1024, 256, 1)
save_conv2d_layer(f, gguf_writer, "l8", 256, 512, 3)
save_conv2d_layer(f, gguf_writer, "l9", 512, 255, 1, batch_normalize=False)
save_conv2d_layer(f, gguf_writer, "l10", 256, 128, 1)
save_conv2d_layer(f, gguf_writer, "l11", 384, 256, 3)
save_conv2d_layer(f, gguf_writer, "l12", 256, 255, 1, batch_normalize=False)
f.close()
gguf_writer.write_header_to_file()
gguf_writer.write_kv_data_to_file()
gguf_writer.write_tensors_to_file()
gguf_writer.close()
print("{} converted to {}".format(sys.argv[1], outfile))
针对yolov3,关于gguf的模型转换其实不复杂,主要就是保存卷积层的参数。
而针对卷积层的保存,还是需要区分为包含Batch Norm,和不包含BN两种情况,当包含BN时,需要包含的参数明显增加,包括每个通道的scales,mean,variance。此外,必须要存储的是模型的bias和卷积核参数。
def save_conv2d_layer(f, gguf_writer, prefix, inp_c, filters, size, batch_normalize=True):
biases = np.fromfile(f, dtype=np.float32, count=filters)
gguf_writer.add_tensor(prefix + "_biases", biases, raw_shape=(1, filters, 1, 1))
if batch_normalize:
scales = np.fromfile(f, dtype=np.float32, count=filters)
gguf_writer.add_tensor(prefix + "_scales", scales, raw_shape=(1, filters, 1, 1))
rolling_mean = np.fromfile(f, dtype=np.float32, count=filters)
gguf_writer.add_tensor(prefix + "_rolling_mean", rolling_mean, raw_shape=(1, filters, 1, 1))
rolling_variance = np.fromfile(f, dtype=np.float32, count=filters)
gguf_writer.add_tensor(prefix + "_rolling_variance", rolling_variance, raw_shape=(1, filters, 1, 1))
weights_count = filters * inp_c * size * size
l0_weights = np.fromfile(f, dtype=np.float32, count=weights_count)
## ggml doesn't support f32 convolution yet, use f16 instead
l0_weights = l0_weights.astype(np.float16)
gguf_writer.add_tensor(prefix + "_weights", l0_weights, raw_shape=(filters, inp_c, size, size))
但是,目前来看,gguf header的具体格式还是不清晰,需要进一步探索。