紧跟MiniCPM脚步，升级MiniCPM-V 2.6

1.版本升级/安装

随着最新版本的发布，第一时间对服务器的多模态模型进行了升级。

git地址：https://github.com/OpenBMB/MiniCPM-V

如果已经是 MiniCPM-V 2.5的小伙伴直接做升级即可，将 requirements.txt 文件中只保留如下内容

gradio==4.22.0
gradio_client
http://thunlp.oss-cn-qingdao.aliyuncs.com/multi_modal/never_delete/modelscope_studio-0.4.0.9-py3-none-any.whl
decord

然后执行

pip3 install -r requirements.txt -t /home/aistudio/external-libraries

具体路径替换成之前的py包路径即可。

如果是全新安装的话，就按照git官网上的安装步骤一步一步来吧~~不搬了。

2.新加内容-视频解析

一直在等这个功能，它终于出现了~在原来的开发的py基础上，做了点小改动。

接收video，这里面我单独在外层定义了个sessionMap，用来缓存加载之后的video。

@app.route('/omnilmm/uploadvideo', methods=['POST'])
def upload_video():
    file = request.files['video']
    if file :
        now = datetime.datetime.now()
        timestamp = now.timestamp()
        filename = ('{}'+file.filename).format(timestamp)
        file.save(os.path.join(app.config['UPLOAD_FOLDER'], filename))
        def uniform_sample(l, n):
            gap = len(l) / n
            idxs = [int(i * gap + gap / 2) for i in range(n)]
            return [l[i] for i in idxs]
        vr = VideoReader(app.config['UPLOAD_FOLDER'] + filename)
        sample_fps = round(vr.get_avg_fps() / 1)  # FPS
        frame_idx = [i for i in range(0, len(vr), sample_fps)]
        if len(frame_idx)>MAX_NUM_FRAMES:
            frame_idx = uniform_sample(frame_idx, MAX_NUM_FRAMES)
        video = vr.get_batch(frame_idx).asnumpy()
        video = [Image.fromarray(v.astype('uint8')) for v in video]
        b = uuid.uuid4()
        c = b.hex
        sessionMap[c] = video
        #from PIL import Image
        #print(res)
        if os.path.exists((app.config['UPLOAD_FOLDER'] + filename)):
            os.remove((app.config['UPLOAD_FOLDER'] + filename))
            print(f"文件 {(app.config['UPLOAD_FOLDER'] + filename)} 删除成功！")
        else:
            print(f"文件 {(app.config['UPLOAD_FOLDER'] + filename)} 不存在。")

        return jsonify({'message': {"key":c , "value" : "解析完成，可继续提问"}, 'success': True}), 200

根据前端传回来的key，进行提问。

@app.route('/omnilmm/askvideo', methods=['POST'])
def ask_video():
    imgkey = request.form.get("key")
    if imgkey in sessionMap:
        question = request.form.get("q")
        msgs = [{'role': 'user', 'content': sessionMap[imgkey] + [question]}]
        default_params = {"use_image_id":False, "max_slice_nums": 1}
        res = model.chat(
            image=None,
            msgs=msgs,
            tokenizer=tokenizer,
            **default_params
        )
        return jsonify({'message': res, 'success': True}), 200
    else:
        return jsonify({'message': "", 'success': True}), 200

至此，关于视频的保存及解析的调用就完成了~

感兴趣的赶紧试试最新的MiniCPM-V 2.6，真心好用~