三星堆文字点选验证码
抢票过程中往往需要对验证码进行自动识别,本文对三星堆博物馆的抢票验证码进行分析,并训练模型进行识别,准确率可以达到99%对上,基本破解此类验证码。>
前言
文字点选类的验证码通常选择yolo+siamese孪生神经网络算法进行识别,通过本人多种验证码的实践发现,普通的siamese孪生神经网络算法因为需要逐个对比,耗时较长,因此基于孪生神经网络进行改进,提升推理速度。
一、数据标注
首先在网站上批量采集验证码图像,然后使用标注工具进行标注,注意和普通yolo模型标注不同的是,除了要标注框的类别之外,还需要标注文字,为后续孪生神经网络数据集做准备。
二、模型训练
1.目标检测模型
当然是大家都熟悉的yolov5了,代码就不放了,网上一大把:
2.相似度识别模型
代码如下(核心模型部分):
"""
训练模型的时候分为两个部分,特征提取器和分类器
采用组合loss进行训练: Classification Loss 和 Metric learning Loss
"""
import paddle
import paddle.nn as nn
class SiameseNet18(nn.Layer):
def __init__(self, feature_num, pretrained=True):
super().__init__()
self.backbone = paddle.vision.resnet18(pretrained=True)
self.fc = nn.Linear(512, feature_num)
self.backbone.fc = self.fc
self.backbone.relu = nn.Silu()
def forward(self, inputs):
x = self.backbone(inputs)
features = paddle.nn.functional.normalize(x, axis=1)
return features
# insightface loss
class Classifier(nn.Layer):
def __init__(self, input_dim, class_nums, margin1=1.0, margin2=0.5, margin3=0.0, scale=64):
super().__init__()
self.weight = paddle.normal(0, 0.01, (input_dim, class_nums))
self.sub_weight = paddle.create_parameter(shape=self.weight.shape, dtype='float32',
default_initializer=paddle.nn.initializer.Assign(self.weight))
self.sub_weight.stop_gradient = False
self.margin1 = margin1
self.margin2 = margin2
self.margin3 = margin3
self.logit_scale = scale
def forward(self, total_feature, total_label):
norm_weight = paddle.nn.functional.normalize(self.sub_weight, axis=0)
local_logit = paddle.matmul(total_feature, norm_weight)
loss, softmax = paddle.nn.functional.margin_cross_entropy(
local_logit,
total_label,
margin1=self.margin1,
margin2=self.margin2,
margin3=self.margin3,
scale=self.logit_scale,
return_softmax=True,
reduction='mean', )
return loss, softmax
if __name__ == '__main__':
model = SiameseNet18(64)
paddle.summary(model, (2, 3, 64, 64))
三、模型部署
推理部署和调用方法
使用flask或者fastapi框架进行部署,推理部署采用openvino(只支持Intel CPU)或者onnxruntime(支持CPU和GPU)。
接口URL:http://127.0.0.1:8050/sxd/
请求方法:POST
请求参数
- 参数类型:playload生成验证码后的返回响应,为json格式,示例:
{
"id": "86d4faecc1bc434c99b0ffc2725a34ea",
"captcha": {
"type": "WORD_IMAGE_CLICK",
"backgroundImage": "data:image/jpeg;base64,/9j/4AAQSkZJRgABAgAAAQABAAD/2wBDAAgGBgcGBQgHBwcJCQgKDBQNDAsLDBkSEw8UHRofHh0aHBwgJC4nICIsIxwcKDcpLDAxNDQ0Hyc5PTgyPC4zNDL/2wBDAQkJfpWqeo+pnU006kNMoeh+R/p/UUIcb/wDdIoT7r/T+opUHD/7ppiIR1obrR3FB6mgYlFFFAAKcKaKc2Q==",
"templateImage": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAR4AAABMCAYAAAC7+NieAAAHcklEQVR42u2du60VSxBFj4s3dUBWwHIrqC9/Ljmaej9I9AgwIQol2Y+gFLmdjGBwQW3hVwNAiRFRydpyPoXKWdDR9Em5QcZAP9aX3g==",
"backgroundImageTag": "default",
"templateImageTag": null,
"backgroundImageWidth": 590,
"backgroundImageHeight": 360,
"templateImageWidth": 286,
"templateImageHeight": 76,
"data": null
}
}
响应格式
- 响应体:返回json格式
{
"id": "86d4faecc1bc434c99b0ffc2725a34ea",
"data": {
"bgImageWidth": 590,
"bgImageHeight": 360,
"startSlidingTime": "2024-07-17T14:34:23.593Z",
"endSlidingTime": "2024-07-17T14:34:23.613000Z",
"trackList": [
{
"x": 535,
"y": 145,
"type": "move",
"t": 4
},
.....
{
"x": 443,
"y": 181,
"type": "click",
"t": 20
}
]
}
}
- 过验证码:
将上面得到的json,作为负载发送至https://ticket.sxd.cn/api-api/captcha/check,注意需要带入tk,当得到如下响应时,验证成功;
{
"code": 200,
"msg": "OK",
"data": {
"id": "5ad886ddc5af44ef907e82cb0bf53c3e"
},
"success": true
}