Advanced Topics in Data Mining Spring 2011

Books (PDFs):

• Mining Massive Datasets by A. Rajaraman, J. Ullman.

• Networks, Crowds, and Markets: Reasoning About a Highly Connected World by D. Easley, J. Kleinberg.

• Data-Intensive Text Processing with MapReduce by J. Lin, C. Dyer.

Datasets:

SNAP network datasets

• 60 large social and information network datasets

Wikipedia

• Complete edit history of Wikipedia articles: Which user edited what article at what time.
• Wikipedia page to page link data
• DBpedia: A richly labeled graph of Wikipedia entities.
• Freebase: An entity graph of people, places and things.

Ratings and purchases (movies, music, etc.)

• Amazon product co-purchasing network: 600k products and all their metadata.
• KDD Cup 2011: 300M ratngs from 1M users on 600k songs, albums and artists.
• IMDB database: Everything about every movie ever made.
• Movielens: User movie rating data.

Yahoo! Webscope Catalog of datasets

• Yahoo! Webscope dataset collection. Cotains Language Data, Graph and Social Data, Ratings Data, Advertising and Market Data, Competition Data
• Note: Jure Leskovec will have to apply for any sets you want, and we must agree not to distribute them further. There may be a delay, so get requests in early.

Co-authorship and Citation Networks

• DBLP: Digital Bibliography & Library Project. More info.
• Arxiv citation and co-authorship networks: Data is from KDD 2003 Cup.

Internet (Autonomous Systems) topology

• AS Graphs

Who trusts whom data at Trustlet

• Trust network datasets from Trustlet.org

Stanford only datasets

• Instant messenger buddy graph from March 2005. There are 227 million nodes and 7.3 billion undirected edges.
• Altavista web graph from 2002. 1.4 billion nodes, 5.5 billion edges.
• Memetracker2. 1 million blog posts, news media articles, tweets and facebook wall posts per hour for a period from August 1 to August 31 2010. 181GB of compressed data.
• The New York Times Annotated Corpus: over 1.8 million articles written and published by the New York Times between January 1, 1987 and June 19, 2007 with article metadata.
• TheFind: product information data (price, category, related products) extracted from 239 different websites.
• Twitter: About 500 million tweets over a 7 month period. Data description.
• Wikipedia: Complete revision history of Wikipedia -- every edit of every article with full content.
• Wikipedia webserver logs: Hourly Wikipedia page access statistics.
• Yahoo! Messenger: Instant Messenger graph with some additional information

Data can be accessed here. Email Jure if you do not have a password.

Other Datasets

• Yannis Antonellis and Jawed Karim offer a file that contains information about the search queries that were used to reach pages on the Stanford Web server. See http://www.stanford.edu/~antonell/tags_dataset.html
• The Stanford WebBase project provides a crawl, and may even be talked into providing a specialized crawl if you have a need. Find description here. Find how to access web pages in the repository here.