NLP之路-Dataset大全

The Enron corpus with an extensive annotation of organizational hierarchy.

http://www1.ccls.columbia.edu/~rambow/enron/

======================================================================================================================

CMU AI Repository (非常全！！！)

http://www.cs.cmu.edu/afs/cs/project/ai-repository/ai/util/areas/

Names Corpus

http://www.cs.cmu.edu/afs/cs/project/ai-repository/ai/util/areas/nlp/corpora/names/

======================================================================================================================

Email Datasets

http://www.cs.cmu.edu/~einat/datasets.html

Email Datasets

---

1. Personal Name Annotation

Due to privacy issues, it is very hard to get hold of large and realistic email corpora. Here you can find
a couple of email datasets, as well as a dataset of news groups text - annotated with personal names spans.

The full description of these datasets, including relevant statistics and references, is available in:

Einat Minkov, Richard C. Wang &William W. Cohen, Extracting Personal Names from Emails:
Applying Named Entity Recognition to Informal Text, in HLT/EMNLP 2005(PDF)

Some fast details:

• The email corpora given here were extracted from the Enron corpus, made public by the Federal
  Agency Regulatory commission. A version of this data was later purchased by the CALO project,
  and made available for research purposes.
• The first dataset, 'Enron-Meetings', consists of all messages located in folders named "meetings"
  or "calendar" (excluding a few very large files). Most of these messages are meeting related. The second
  subset, 'Enron-Random', was formed by uniformly sampling a user name (out of 158 users) and then
  randomly sampling an email from that user.
• As a second type of informal text, we also annotated a collection of newsgroups postings. The
  'Newsgroups' dataset was extracted from the 20Newsgroups corpus, by Vitor R. Carvalho.
  
• These datasets are given here in a Minorthird format (plain text, with separate labels files), as well as
  in a 'general' format, where the personal labels are embedded in the text using XML tags.
• The given zipped files construct a directory tree. The separation into train and test folders corresponds
  to the data splits described in the abovementioned paper. Further separation is for convenience purposes.

Download:

• Enron Meetings: Minorthird format , XML tags
  Enron - random : Minorthird format , XML tags
  NewsGroups : Minorthird format , XML tags
  

---

2. Person name disambiguation and threading

Here you can download Enron corpora and datasets, used for the general problems of entity disambiguation
and the extraction of inter-entity relations. Email here is represented as a relational database, which includes
text. Specifically, the tasks considered in these subsets of the Enron corpus are person name disambiguation
in email and intelligent message threading.

Two variations of the data are provided:

A. row email essages, and the corresponding datasets (queries and correct answers), as used in

• Einat Minkov, William W. Cohen and Andrew Y. Ng,
  "Contextual Search and Name Disambiguation in Email using Graphs", SIGIR 2006 (PDF)
  
  Download: Person name diambiguation corpora, datasets
  Threading corpora , datasets
  

B. graph files (net relations and entity declarations), and the corresponding datasets, as used in

• Einat Mi