本文主要是使用Python求不同的股票之间的相关性，并绘制成可视图片，参考来源https://scikit-learn.org/stable/auto_examples/applications/plot_stock_market.html#sphx-glr-auto-examples-applications-plot-stock-market-py
这是他的实现效果，他的数据来自平台接口，本文实现本地数据的读取和绘制，数据来源于sqlserver(wind数据），Python版本为2.7，需要安装相应的包，见代码
代码如下：
#-*-encoding=utf-8-*-
"""
=======================================
Visualizing the stock market structure
=======================================
This example employs several unsupervised learning techniques to extract
the stock market structure from variations in historical quotes.
The quantity that we use is the daily variation in quote price: quotes
that are linked tend to cofluctuate during a day.
.. _stock_market:
Learning a graph structure
--------------------------
We use sparse inverse covariance estimation to find which quotes are
correlated conditionally on the others. Specifically, sparse inverse
covariance gives us a graph, that is a list of connection. For each
symbol, the symbols that it is connected too are those useful to explain
its fluctuations.
Clustering
----------
We use clustering to group together quotes that behave similarly. Here,
amongst the :ref:`various clustering techniques <clustering>` available
in the scikit-learn, we use :ref:`affinity_propagation` as it does
not enforce equal-size clusters, and it can choose automatically the
number of clusters from the data.
Note that this gives us a different indication than the graph, as the
graph reflects conditional relations between variables, while the
clustering reflects marginal properties: variables clustered together can
be considered as having a similar impact at the level of the full stock
market.
Embedding in 2D space
---------------------
For visualization purposes, we need to lay out the different symbols on a
2D canvas. For this we use :ref:`manifold` techniques to retrieve 2D
embedding.
Visualization
-------------
The output of the 3 models are combined in a 2D graph where nodes
represents the stocks and edges the:
- cluster labels are used to define the color of the nodes
- the sparse covariance model is used to display the strength of the edges
- the 2D embedding is used to position the nodes in the plan
This example has a fair amount of visualization-related code, as
visualization is crucial here to display the graph. One of the challenge
is to position the labels minimizing overlap. For this we use an
heuristic based on the direction of the nearest neighbor along each
axis.
(tensorflow) F:\faceswap-master>pip install lxml -i https://pypi.tuna.tsinghua.edu.cn/simple/
"""
from __future__ import print_function
# Author: Gael Varoquaux gael.varoquaux@normalesup.org
# License: BSD 3 clause
import sys
reload(sys)
sys.setdefaultencoding('utf-8')
import numpy as np
import matplotlib.pyplot as plt
from matplotlib.collections import LineCollection
#import matplotlib.pyplot as plt
import pandas as pd
import pymssql
from sklearn import cluster, covariance, manifold
import sklearn.covariance.graph_lasso_ as gl
print(__doc__)
# #############################################################################
# Retrieve the data from Internet
# The data is from 2003 - 2008. This is reasonably calm: (not too long ago so
# that we get high-tech firms, and before the 2008 crash). This kind of
# historical data can be obtained for from APIs like the quandl.com and
# alphavantage.co ones.
#沪深三百
symbol_dict = {
'000002':u'深万科Ａ',
'000009':u'深宝安Ａ',
'000012':u'南玻科控',
'000021':u'深科技Ａ',
'000022':u'深赤湾Ａ',
'000027':u'深能源Ａ',
'000029':u'深深房Ａ',
'000031':u'深宝恒Ａ',
'000036':u'华联控股',
'000039':u'中集集团',
'000059':u'辽通化工',
'000060':u'中金岭南',
'000061':u'农产品',
'000063':u'中兴通讯',
'000066':u'长城电脑',
'000068':u'赛格三星',
'000069':u'华侨城Ａ',
'000088':u'盐田港Ａ',
'000089':u'深圳机场',
'000099':u'中信海直',
'000157':u'中联重科',
'000400':u'许继电气',
'000401':u'冀东水泥',
'000402':u'金融街',
'000410':u'沈阳机床',
'000422':u'湖北宜化',
'000423':u'东阿阿胶',
'000425':u'徐工科技',
'000488':u'晨鸣纸业',
'000503':u'海虹控股',
'000518':u'四环生物',
'000520':u'中国凤凰',
'000528':u'桂柳工Ａ',
'000538':u'云南白药',
'000539':u'粤电力Ａ',
'000541':u'佛山照明',
'000550':u'ST江铃',
'000559':u'万向钱潮',
'000568':u'泸州老窖',
'000581':u'威孚高科',
'000601':u'韶能股份',
'000607':u'华立控股',
'000617':u'石油济柴',
'000623':u'吉林敖东',
'000625':u'长安汽车',
'000630':u'铜都铜业',
'000636':u'风华高科',
'000651':u'格力电器',
'000652':u'泰达股份',
'000659':u'珠海中富',
'000680':u'山推股份',
'000682':u'ST东方',
'000698':u'沈阳化工',
'000707':u'双环科技',
'000708':u'大冶特钢',
'000709':u'唐钢股份',
'000717':u'韶钢松山',
'000725':u'京东方Ａ',
'000726':u'鲁泰Ａ',
'000729':u'燕京啤酒',
'000733':u'振华科技',
'000758':u'中色建设',
'000761':u'ST板材',
'000767':u'漳泽电力',
'000768':u'西飞国际',
'000778':u'新兴铸管',
'000786':u'北新建材',
'000792':u'盐湖钾肥',
'000793':u'燃气股份',
'000800':u'一汽轿车',
'000806':u'银河科技',
'000807':u'云铝股份',
'000822':u'山东海化',
'000825':u'太钢不锈',
'000828':u'福地科技',
'000839':u'中信国安',
'000858':u'五粮液',
'000869':u'张裕A',
'000875':u'吉电股份',
'000878':u'云南铜业',
'000895':u'双汇发展',
'000898':u'鞍钢新轧',
'000900':u'现代投资',
'000912':u'泸天化',
'000917':u'电广传媒',
'000920':u'南方汇通',
'000927':u'ST夏利',
'000930':u'丰原生化',
'000932':u'华菱管线',
'000933':u'神火股份',
'000937':u'金牛能源',
'000939':u'凯迪电力',
'000959':u'首钢股份',
'000960':u'锡业股份',
'000962':u'东方钽业',
'000968':u'神州股份',
'000969':u'安泰科技',
'000970':u'中科三环',
'000983':u'西山煤电',
'000997':u'新大陆',
'600000':u'浦发银行',
'600004':u'白云机场',
'600006':u'东风汽车',
'600008':u'首创股份',
'600009':u'上海机场',
'600010':u'钢联股份',
'600011':u'华能国际',
'600012':u'皖通高速',
'600015':u'华夏银行',
'600016':u'民生银行',
'600018':u'上港集箱',
'600019':u'宝钢股份',
'600020':u'中原高速',
'600026':u'中海发展',
'600028':u'中国石化',
'600029':u'南方航空',
'600030':u'中信证券',
'600031':u'三一重工',
'600033':u'福建高速',
'600036':u'招商银行',
'600037':u'歌华有线',
'600050':u'中国联通',
'600057':u'厦新电子',
'600058':u'龙腾科技',
'600060':u'海信电器',
'600062':u'双鹤药业',
'600073':u'上海梅林',
'600078':u'澄星股份',
'600085':u'同仁堂',
'600088':u'中视传媒',
'600089':u'特变电工',
'600091':u'明天科技',
'600096':u'云天化',
'600098':u'广州控股',
'600100':u'清华同方',
'600104':u'上海汽车',
'600108':u'亚盛集团',
'600110':u'中科英华',
'600115':u'东方航空',
'600117':u'西宁特钢',
'600121':u'郑州煤电',
'600123':u'兰花科创',
'600125':u'铁龙股份',
'600126':u'杭钢股份',
'600132':u'重庆啤酒',
'600138':u'中青旅',
'600151':u'航天机电',
'600153':u'厦门建发',
'600166':u'福田汽车',
'600170':u'上海建工',
'600171':u'上海贝岭',
'600177':u'雅戈尔',
'600183':u'生益科技',
'600188':u'兖州煤业',
'600196':u'复星实业',
'600198':u'大唐电信',
'600207':u'安彩高科',
'600210':u'紫江企业',
'600215':u'长春经开',
'600220':u'江苏阳光',
'600221':u'海南航空',
'600231':u'凌钢股份',
'600236':u'桂冠电力',
'600256':u'广汇股份',
'600266':u'北京城建',
'600267':u'海正药业',
'600269':u'赣粤高速',
'600270':u'外运发展',
'600271':u'航天信息',
'600276':u'恒瑞医药',
'600282':u'南钢股份',
'600299':u'星新材料',
'600307':u'酒钢宏兴',
'600308':u'华泰股份',
'600309':u'烟台万华',
'600320':u'振华港机',
'600331':u'宏达股份',
'600333':u'长春燃气',
'600348':u'国阳新能',
'600350':u'山东基建',
'600361':u'华联综超',
'600362':u'江西铜业',
'600377':u'宁沪高速',
'600383':u'金地集团',
'600399':u'抚顺特钢',
'600408':u'安泰集团',
'600415':u'小商品城',
'600418':u'江淮汽车',
'600428':u'中远航运',
'600456':u'宝钛股份',
'600460':u'士兰微',
'600498':u'烽火通信',
'600500':u'中化国际',
'600508':u'上海能源',
'600519':u'贵州茅台',
'600521':u'华海药业',
'600535':u'天士力',
'600548':u'深高速',
'600549':u'厦门钨业',
'600550':u'天威保变',
'600569':u'安阳钢铁',
'600578':u'京能热电',
'600581':u'八一钢铁',
'600583':u'海油工程',
'600585':u'海螺水泥',
'600588':u'用友软件',
'600597':u'光明乳业',
'600598':u'北大荒',
'600600':u'青岛啤酒',
'600601':u'方正科技',
'600602':u'广电电子',
'600616':u'第一食品',
'600621':u'上海金陵',
'600628':u'新世界',
'600635':u'大众公用',
'600637':u'广电信息',
'600639':u'浦东金桥',
'600642':u'申能股份',
'600643':u'爱建股份',
'600649':u'原水股份',
'600652':u'爱使股份',
'600653':u'申华控股',
'600654':u'飞乐股份',
'600655':u'豫园商城',
'600660':u'福耀玻璃',
'600662':u'上海强生',
'600663':u'陆家嘴',
'600674':u'川投控股',
'600675':u'中华企业',
'600688':u'上海石化',
'600690':u'青岛海尔',
'600694':u'大商股份',
'600717':u'天津港',
'600718':u'东软股份',
'600724':u'宁波富达',
'600739':u'辽宁成大',
'600740':u'山西焦化',
'600741':u'巴士股份',
'600747':u'大显股份',
'600754':u'新亚股份',
'600770':u'综艺股份',
'600779':u'全兴股份',
'600780':u'通宝能源',
'600795':u'国电电力',
'600797':u'浙大网新',
'600808':u'马钢股份',
'600809':u'山西汾酒',
'600811':u'东方集团',
'600812':u'华北制药',
'600820':u'隧道股份',
'600834':u'申通地铁',
'600835':u'上菱电器',
'600839':u'四川长虹',
'600851':u'海欣股份',
'600854':u'春兰股份',
'600863':u'内蒙华电',
'600868':u'梅雁股份',
'600871':u'仪征化纤',
'600874':u'创业环保',
'600875':u'东方电机',
'600879':u'火箭股份',
'600881':u'亚泰集团',
'600884':u'杉杉股份',
'600886':u'ST华靖',
'600887':u'伊利股份',
'600894':u'广钢股份',
'600895':u'张江高科',
}
server ='服务器名称'
#conn = pymssql.connect(server, user, password, database)
conn = pymssql.connect('服务器名称','数据库名称','数据库密码','数据库表名')
cursor = conn.cursor()
symbols, names = np.array(sorted(symbol_dict.items())).T
quotes = []
close_prices=[]
open_prices=[]
for symbol in symbols:
print('Fetching quote history for %r' % symbol, file=sys.stderr)
# url = ('https://raw.githubusercontent.com/scikit-learn/examples-data/'
# 'master/financial-data/{}.csv')
#历史数据从2016年到2018年
sql = 'SELECT * FROM TB_TICKDATE where cd=\'{}\'and dt>\'2016-01-01 00:00:00.000\' order by dt'
cursor.execute(sql.format(symbol))
row = cursor.fetchone()
close = []
open = []
while row:
#print(row[1])
close.append(row[7])
open.append(row[4])
row = cursor.fetchone()
# quotes.append(row)
close_prices.append(close)
open_prices.append(open)
# quotes.append(pd.read_csv(url.format(symbol)))
conn.close()
#close_prices = np.vstack([q['close'] for q in quotes])
#open_prices = np.vstack([q['open'] for q in quotes])
close_prices1 = np.vstack(close_prices)
open_prices1 = np.vstack(open_prices)
# The daily variations of the quotes are what carry most information
variation = close_prices1 - open_prices1
# #############################################################################
# Learn a graphical structure from the correlations
#edge_model = covariance.GraphicalLassoCV(cv=5)
#edge_model = covariance.graph_lasso_(cv=5)
edge_model = covariance.GraphLassoCV(cv=5)
#covariance.GraphLassoCV()
#covariance.graph_lasso()
# standardize the time series: using correlations rather than covariance
# is more efficient for structure recovery
X = variation.copy().T
X /= X.std(axis=0)
edge_model.fit(X)
# #############################################################################
# Cluster using affinity propagation
_, labels = cluster.affinity_propagation(edge_model.covariance_)
n_labels = labels.max()
for i in range(n_labels + 1):
print('Cluster %i: %s' % ((i + 1), ', '.join(names[labels == i])))
# #############################################################################
# Find a low-dimension embedding for visualization: find the best position of
# the nodes (the stocks) on a 2D plane
# We use a dense eigen_solver to achieve reproducibility (arpack is
# initiated with random vectors that we don't control). In addition, we
# use a large number of neighbors to capture the large-scale structure.
node_position_model = manifold.LocallyLinearEmbedding(
n_components=2, eigen_solver='dense', n_neighbors=6)
embedding = node_position_model.fit_transform(X.T).T
# #############################################################################
# Visualization
plt.figure(1, facecolor='w', figsize=(10, 8))
plt.clf()
ax = plt.axes([0., 0., 1., 1.])
plt.axis('off')
# Display a graph of the partial correlations
partial_correlations = edge_model.precision_.copy()
d = 1 / np.sqrt(np.diag(partial_correlations))
partial_correlations *= d
partial_correlations *= d[:, np.newaxis]
non_zero = (np.abs(np.triu(partial_correlations, k=1)) > 0.02)
# Plot the nodes using the coordinates of our embedding
plt.scatter(embedding[0], embedding[1], s=100 * d ** 2, c=labels,
cmap=plt.cm.nipy_spectral)
# Plot the edges
start_idx, end_idx = np.where(non_zero)
# a sequence of (*line0*, *line1*, *line2*), where::
# linen = (x0, y0), (x1, y1), ... (xm, ym)
segments = [[embedding[:, start], embedding[:, stop]]
for start, stop in zip(start_idx, end_idx)]
values = np.abs(partial_correlations[non_zero])
lc = LineCollection(segments,
zorder=0, cmap=plt.cm.hot_r,
norm=plt.Normalize(0, .7 * values.max()))
lc.set_array(values)
lc.set_linewidths(15 * values)
ax.add_collection(lc)
# Add a label to each node. The challenge here is that we want to
# position the labels to avoid overlap with other labels
for index, (name, label, (x, y)) in enumerate(
zip(names, labels, embedding.T)):
dx = x - embedding[0]
dx[index] = 1
dy = y - embedding[1]
dy[index] = 1
this_dx = dx[np.argmin(np.abs(dy))]
this_dy = dy[np.argmin(np.abs(dx))]
if this_dx > 0:
horizontalalignment = 'left'
x = x + .002
else:
horizontalalignment = 'right'
x = x - .002
if this_dy > 0:
verticalalignment = 'bottom'
y = y + .002
else:
verticalalignment = 'top'
y = y - .002
plt.text(x, y, name, size=10,
horizontalalignment=horizontalalignment,
verticalalignment=verticalalignment,
bbox=dict(facecolor='w',
edgecolor=plt.cm.nipy_spectral(label / float(n_labels)),
alpha=.6))
plt.xlim(embedding[0].min() - .15 * embedding[0].ptp(),
embedding[0].max() + .10 * embedding[0].ptp(),)
plt.ylim(embedding[1].min() - .03 * embedding[1].ptp(),
embedding[1].max() + .03 * embedding[1].ptp())
plt.show()
效果图如下： 
局部放大图：
图中两个股票之间的线条越黑表示相关性越强，可见沪深三百之间的相关性不是很强，下面按股票代码排序选择前50多个股票测试下相关性：
比沪深300的相关性要复杂些
比较下里面比较黑的深物业和深深宝的历史数据：
前半部分相关性确实很强
扫一扫
3076
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
