python视频分析系统_基于智能多模式学习的系统，用于视频，产品和广告分析

Hysia Video to Retail Platform

An intelligent multimodal-learning based system for video, product and ads analysis. You can build various downstream applications with the system, such as product recommendation, video retrieval. Several examples are provided.

V2 is under active development currently. You are welcome to create a issue, pull request here. We will credit them into V2.

Showcase

👉 Full list of showcase.

Upload video and process it by selecting different models

Display video processing result

Search scene by image and text

Insert product advertisement and display insertion result

Features

Multimodal learning-based video analysis:

Scene / Object / Face detection and recognition

Multimodality data pre-processing

Results align and store

Downstream applications:

Intelligent ads insertion

Content-product match

Visualized testbed

Visualize multimodality results

Can be installed separately

Setup Environment

1. Download Data

# Make sure this script is run from project root

bash scripts/download-data.sh

2. Installation

👉 Install with Docker

🐳

docker pull hysia/hysia:v2o

Configuration

Change decoder and model server running devices at device_placement.yml:

decoder: CPU

visual_model_server: CUDA:1

audio_model_server: CUDA:2

feature_model_server: CUDA:3

product_search_server: CUDA:2

scene_search_server: CUDA:3

Device value format: cpu, cuda or cuda:.

Demo

Run with docker

🐳

docker run --rm \

--gpus all -d -p 8000:8000 \

-v ${PWD}/server/config/device_placement.yml:/content/server/config/device_placement.yml \

hysia/hysia:v2o

Then you can go to http://localhost:8000. Use username: admin and password: admin to login.

Some Useful Tools

Large dataset preprocessing

Video/audio decoding

Model profiling

Multimodality data testbed

Contributing

You are welcome to contribute to Hysia! Please refer to here to get start.

Paper Citation

Coming soon!

About Us

Maintainers

Huaizheng Zhang

Yuanming Li

Qiming Ai

Previous Contributors

Shengsheng Zhou

Wenbo Jiang (Now, Shopee)

Ziyuan Liu (Now, Tencent)

Yongjie Wang (Now, NTU PhD)