“巧妇难为无米之炊”,很多人在学习数据挖掘时,总是感觉没有数据,然后就不了了之了。事实上,在互联网迅猛发展的今天,我们并不缺少数据,而是被数据所淹没,从而迷失了方向。在此总结一下获取数据的途径,我将其分为屠龙式和倚天式。所谓屠龙式也就是动态获取,可以写一些爬虫程序,从互联网上抓取数据,或者是通过程序自动生成数据;而倚天式就是静态获取,现在有很多公开测试数据挖掘算法的数据,还有一些数据挖掘相关竞赛会提供数据,并且有些竞赛有商业性质,所以数据真实性也很可靠。
在KDNuggets上有Datasets栏目,提供一些数据集,网址为:http://www.kdnuggets.com/datasets/
还有另外一个很好的资源网址为:http://kdd.ics.uci.edu/,里面包含的数据资源如下(按应用领域划分):
Direct Marketing
KDD CUP 1998 Data
GIS
Forest CoverType
Indexing
Corel Image Features
Pseudo Periodic Synthetic Time Series
Intrusion Detection
KDD CUP 1999 Data
Process Control
Synthetic Control Chart Time Series
Recommendation Systems
Entree Chicago Recommendation Data
Robots
Pioneer-1 Mobile Robot Data
Robot Execution Failures
Sign Language Recognition
Australian Sign Language Data
High-quality Australian Sign Language Data
Text Categorization
20 Newsgroups Data
Reuters-21578 Text Categorization Collection
NSF Research Awards Abstracts 199 0-2003
World Wide Web
Microsoft Anonymous Web Data
MSNBC Anonymous Web Data
Syskill Webert Web Data
http://www.cs.toronto.edu/~roweis/data.html
http://www.cs.toronto.edu/~roweis/data.html
http://kdd.ics.uci.edu/summary.task.type.html
http://www-2.cs.cmu.edu/afs/cs.cmu.edu/project/theo-20/www/data/
http://www-2.cs.cmu.edu/afs/cs.cmu.edu/project/theo-11/www/wwkb/
http://www.phys.uni.torun.pl/~duch/software.html
在下面的网址可以找到reuters数据集http://www.research.att.com/~lewis/reuters21578.html
以下网址上有各种数据集:
http://kdd.ics.uci.edu/summary.data.type.html
进行文本分类,还有一个数据集是可以用的,即rainbow的数据集
http://www-2.cs.cmu.edu/afs/cs/project/theo-11/www/naive-bayes.html
UCI收集的机器学习数据集
ftp://pami.sjtu.edu.cn/
http://www.ics.uci.edu/~mlearn//MLRepository.htm
statlib
http://liama.ia.ac.cn/SCILAB/scilabindexgb.htm
http://lib.stat.cmu.edu/
样本数据库
http://kdd.ics.uci.edu/
http://www.ics.uci.edu/~mlearn/MLRepository.html
关于基金的数据挖掘的网站
http://www.gotofund.com/index.asp
http://lans.ece.utexas.edu/~strehl/
各种数据集:
http://kdd.ics.uci.edu/summary.data.type.html
http://www.mlnet.org/cgi-bin/mlnetois.pl/?File=datasets.html
http://lib.stat.cmu.edu/datasets/
http://dctc.sjtu.edu.cn/adaptive/datasets/
http://fimi.cs.helsinki.fi/data/
http://www.almaden.ibm.com/software/quest/Resources/index.shtml
http://miles.cnuce.cnr.it/~palmeri/datam/DCI/
进行文本分类&WEB
http://www-2.cs.cmu.edu/afs/cs/project/theo-11/www/naive-bayes.html
http://www.w3.org/TR/WD-logfile-960221.html
http://www.w3.org/Daemon/User/Config/Logging.html#AccessLog
http://www.w3.org/1998/11/05/WC-workshop/Papers/bala2.html
http://www-2.cs.cmu.edu/afs/cs.cmu.edu/project/theo-11/www/wwkb/
http://www.web-caching.com/traces-logs.html
http://www-2.cs.cmu.edu/webkb
http://www.cs.auc.dk/research/DP/tdb/TimeCenter/TimeCenterPublications/TR-75.pdf
http://www.cs.cornell.edu/projects/kddcup/index.html
时间序列数据的网址
http://www.stat.wisc.edu/~reinsel/bjr-data/
apriori算法的测试数据
http://www.almaden.ibm.com/cs/quest/syndata.html
数据生成器的链接
http://www.cse.cuhk.edu.hk/~kdd/data_collection.html
http://www.almaden.ibm.com/cs/quest/syndata.html
关联:
http://flow.dl.sourceforge.net/sourceforge/weka/regression-datasets.jar
http://www.almaden.ibm.com/software/quest/Resources/datasets/syndata.html#assocSynData
WEKA:
http://flow.dl.sourceforge.net/sourceforge/weka/regression-datasets.jar
1。A jarfile containing 37 classification problems, originally obtained from the UCI repository
http://prdownloads.sourceforge.net/weka/datasets-UCI.jar
2。A jarfile containing 37 regression problems, obtained from various sources
http://prdownloads.sourceforge.net/weka/datasets-numeric.jar
3。A jarfile containing 30 regression datasets collected by Luis Torgo
http://prdownloads.sourceforge.net/weka/regression-datasets.jar
癌症基因:
http://www.broad.mit.edu/cgi-bin/cancer/datasets.cgi
金融数据:
http://lisp.vse.cz/pkdd99/Challenge/chall.htm
以下网址上有各种数据集:
http://kdd.ics.uci.edu/summary.data.type.html
数据挖掘相关比赛以及数据集
2005 University of California data mining contest, predicting bad accounts and their churn date using real-world CRM data, deadline June 30, 2005.
·
ILP 2005 Challenge, on the prediction of functional classes of genes.
·
KDD Cup 2005, on classifying internet user search queries, deadline July 8.
·
Data Mining Cup 2005 (Chemnitz, Germany), for students; topic: How data mining can ascertain the risk of loss of payments andreduce this risk.
·
KDD Cup 2004, focuses on data-mining for a several performance criteria using datasets from bioinformatics and quantum physics.
·
InfoVis 2004 Contest, The History of InfoVis.
·
DATA MINING CUP 2004 (Chemnitz, Germany), for students.
·
InfoVis 2003 Contest: Visualization and Pair Wise Comparison of Trees, results announced Sep 5, 2003.
·
KDD Cup 2003, focuses on problems motivated by network mining and the analysis of usage logs.
·
DATA MINING CUP 2003 (Chemnitz, Germany). The task is to identify spam emails before they reach the user′s mailbox.
·
KDD Cup 2002, focus on data mining in molecular biology.
·
Student Data Mining Cup (2002), Chemnitz University and Prudential Systems.