dHydra是一个主要用于量化投资的二次开发框架(用以进行多进程多数据源的实时计算) 特点: (Important: dHydra的目的不是生产数据,而是制作一个二次开发框架与架构,面向的对象主要还是以开发者为主,不对网络获取的数据源拥有最终解释权)
dHydra Github地址
简要介绍
1.采用Redis作为消息队列分发
2.每个Worker都可以被独立开启
3.集成Tornado Web server给每个Worker分配了动态的url映射,让开发者可以自己定制可视化数据界面
自带数据获取的Vendor类有:
1.获取雪球等数据
2.获取新浪Level-2数据(新浪普及版与标准版通用。包含10档盘口与逐笔数据等Level2,详细介绍可以参考上交所的Level2数据使用说明书与新浪网页版Level2)
3.对新浪Level2行情提供的数据提供现成的存储方案
4.获取3秒/条实时获取数据并持久化存储
5.CTP期货行情与下单
6.调用easytrader进行股票的下单
TODO:
1.数据可视化(TODO)
2.基于Level-2高频数据的回测系统
3.IMPORTANT: dHydra只支持python3.4以上的版本
运行环境
1.python 3.4以上 (开发环境Ubuntu 15.10, python 3.5),不对python2.7提供支持,多版本虚拟环境安装请参考安装dHydra文档
2.mongodb
3.Redis
Mongodb环境配置
1. 配置软件包管理系统:导入公共GPG密钥包:
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv 7F0CEB10
2.创建一个10gen.list文件:
echo 'deb http://downloads-distro.mongodb.org/repo/ubuntu-upstart dist 10gen' | sudo tee /etc/apt/sources.list.d/10gen.list
3.重新加载更新:
apt-get update
4.安装mongodb:
sudo apt-get install mongodb-10gen
Redis安装
1.在自己工作目录依次键入以下命令:
git clone https://codeload.github.com/antirez/redis/tar.gz/4.0-rc2.git
tar -xzvf redis-4.0-rc2.tar.gz
cd redis-4.0-rc2/
make install
2.开一个Terminal,键入redis-server,如果有以下提示证明安装成功:
1711:C 11 Dec 12:16:20.653 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
1711:M 11 Dec 12:16:20.653 * Increased maximum number of open files to 10032 (it was originally set to 1024).
_._
_.-``__ ''-._
_.-`` `. `_. ''-._ Redis 3.9.102 (00000000/0) 64 bit
.-`` .-```. ```\/ _.,_ ''-._
( ' , .-` | `, ) Running in standalone mode
|`-._`-...-` __...-.``-._|'` _.-'| Port: 6379
| `-._ `._ / _.-' | PID: 1711
`-._ `-._ `-./ _.-' _.-'
|`-._`-._ `-.__.-' _.-'_.-'|
| `-._`-._ _.-'_.-' | http://redis.io
`-._ `-._`-.__.-'_.-' _.-'
|`-._`-._ `-.__.-' _.-'_.-'|
| `-._`-._ _.-'_.-' |
`-._ `-._`-.__.-'_.-' _.-'
`-._ `-.__.-' _.-'
`-._ _.-'
`-.__.-'
dHydra安装运行
1.在自己的工作目录依次键入以下命令:
git clone https://github.com/Emptyset110/dHydra.git
cd dHydra/
pip install --editable .
2.然后开启Redis和mongodb,打开两个Terminal,分别键入:
redis-server
和
mongod
如果发现启动mongodb提示如下错误:
MongoDB: couldn't connect to server 127.0.0.1:27017 src/mongo/shell/mongo.js:91
可通过键入下面命令解决:
mkdir -p /data/db/
然后在第三个Terminal键入:
hail dHydra
即可成功启动dHydra!
2016-12-11 12:16:34,124 - DB - 41 - INFO - Trying to connect to redis
2016-12-11 12:16:34,124 - DB - 41 - INFO - Trying to connect to redis
Welcome to dHydra! Following is the Architecture of dHydra
"hail dHydra"
|
|
┌────┴─────┐ ┌────────────┐
| dHydra | | Tornado |
| Server ├─────────┤ Web Server ├──http://127.0.0.1:5000────┐
└────┬─────┘ └──────┬─────┘ |
| | 默认两种url映射规则,例如:
┌────┴────┐ | /api/Worker/BackTest/method/
| Redis ├─────────────────┘ /Worker/BackTest/index
└──┬──────┘ 详情参考文档
| |
├─────Publish────┬─────Subscribe──────┬─────Publish───┐────┤
| | | |
┌──────┴──┐ ┌────┴─────┐ ┌────┴─────┐ ┌────┴─────┐
| (Worker)| | (Worker) | | (Worker) | | (Worker) |
| CTP | | Strategy | | BackTest | | Sina L2 |
└─────────┘ └──────────┘ └──────────┘ └──────────┘
2016-12-11 12:16:34,142 - DB - 41 - INFO - Trying to connect to redis
2016-12-11 12:16:34,142 - DB - 28 - INFO - 尝试连接到Mongodb
2016-12-11 12:16:34,163 - DB - 32 - INFO - 已经成功连接到mongodb
2016-12-11 12:16:34,178 - console - 168 - INFO - Monitor has started
2016-12-11 12:16:34,189 - console - 176 - INFO - Tornado webserver has started
2016-12-11 12:16:34,191 - DB - 41 - INFO - Trying to connect to redis
2016-12-11 12:16:34,213 - DB - 41 - INFO - Trying to connect to redis
注意:需要提前开启Redis和Mongodb服务!