STS 数据分析

2012 - train

- MSR-Paraphrase, Microsoft Research Paraphrase Corpus
  http://research.microsoft.com/en-us/downloads/607d14d9-20cd-47e3-85bc-a2f65cd28042/
  750 pairs of sentences.


- MSR-Video, Microsoft Research Video Description Corpus
  http://research.microsoft.com/en-us/downloads/38cf15fd-b8df-477e-a4e4-a4680caa75af/
  750 pairs of sentences.


- SMTeuroparl: WMT2008 develoment dataset (Europarl section)
  http://www.statmt.org/wmt08/shared-evaluation-task.html
  734 pairs of sentences.

2012 - test

- MSR-Paraphrase, Microsoft Research Paraphrase Corpus
  http://research.microsoft.com/en-us/downloads/607d14d9-20cd-47e3-85bc-a2f65cd28042/
  750 pairs of sentences.


- MSR-Video, Microsoft Research Video Description Corpus
  http://research.microsoft.com/en-us/downloads/38cf15fd-b8df-477e-a4e4-a4680caa75af/
  750 pairs of sentences.


- SMTeuroparl: WMT2008 develoment dataset (Europarl section)
  http://www.statmt.org/wmt08/shared-evaluation-task.html
  459 pairs of sentences.


In addition, it contains two surprise datasets comprising the
following collections:


- SMTnews: news conversation sentence pairs from WMT
  399 pairs of sentences.


- OnWN: pairs of sentences where the first comes from Ontonotes and
  the second from a WordNet definition.
  750 pairs of sentences.

2013 - test

- STS.input.headlines.txt: we used headlines mined from several news
  sources by European Media Monitor using the RSS feed. 
  http://emm.newsexplorer.eu/NewsExplorer/home/en/latest.html


- STS.input.OnWN.txt: The sentences are sense definitions from WordNet
  and OntoNotes. 


- STS.input.FNWN.txt: The sentences are sense definitions from WordNet
  and FrameNet. Note that some FrameNet definitions involve more than
  one sentence.


// 丢失
- STS.input.SMT.txt: This SMT dataset comes from DARPA GALE HTER and
  HyTER, where one sentence is a MT output and the other is a
  reference translation where a reference is generated based on human
  post editing (provided by LDC) or an original human reference
  (provided by LDC) or a human generated reference based on FSM as
  described in (Dreyer and Marcu, NAACL 2012). The reference comes
  from post edited translations.

2014 - test

- STS.input.image.txt: The Image Descriptions data set is a subset of
  the PASCAL VOC-2008 data set (Rashtchian et al., 2010) . PASCAL
  VOC-2008 data set consists of 1,000 images and has been used by a
  number of image description systems. The image captions of the data
  set are released under a CreativeCommons Attribution-ShareAlike
  license, the descriptions itself are free.


- STS.input.OnWN.txt: The sentences are sense definitions from WordNet
  and OntoNotes. 5 pairs of sentences.


- STS.input.tweet-news.txt: The tweet-news data set is a subset of the
  Linking-Tweets-to-News data set (Guo et al., 2013), which consists
  of 34,888 tweets and 12,704 news articles.  The tweets are the
  comments on the news articles.  The news sentences are the titles of
  news articles.


- STS.input.deft-news.txt: A subset of news article data in the DEFT
  project.


- STS.input.deft-forum.txt: A subset of discussion forum data in the
  DEFT project.


- STS.input.headlines.txt: we used headlines mined from several news
  sources by European Media Monitor using the RSS feed.
  http://emm.newsexplorer.eu/NewsExplorer/home/en/latest.html 

  2015 - test（with some raw data)

 

- STS.input.image.txt: The Image Descriptions data set is a subset of
  the Flickr dataset presented in (Rashtchian et al., 2010), which
  consisted on 8108 hand-selected images from Flickr, depicting
  actions and events of people or animals, with five captions per
  image. The image captions of the data set are released under a
  CreativeCommons Attribution-ShareAlike license.


- STS.input.headlines.txt: We used headlines mined from several news
  sources by European Media Monitor using their RSS feed from April 2,
  2013 to July 28, 2014. This period was selected to avoid overlap
  with STS 2014 data.
  http://emm.newsexplorer.eu/NewsExplorer/home/en/latest.html


- STS.input.answers-students.txt: The source of these pairs is the
  BEETLE corpus (Dzikovska et al., 2010), is a question-answer data
  set collected and annotated during the evaluation of the BEETLE II
  tutorial dialogue system. The BEETLE II system is an intelligent
  tutoring engine that teaches students in basic electricity and
  electronics. The corpus was used in the student response analysis
  task of semeval-2013. Given a question, a known correct "reference
  answer" and the "student answer", the goal of the task was to assess
  student answers as correct, contradictory or incorrect (partially
  correct, irrelevant or not in the domain). For STS, we selected
  pairs of answers made up by single sentences.


- STS.input.answers-forum.txt: This data set consists of paired
  answers collected from the Stack Exchange question and answer
  websites (http://stackexchange.com/). Some of the paired answers are
  in response to the same question, while others are in response to
  different questions. Each answer in the pair consists of a statement
  composed of a single sentence or sentence fragment. For
  multi-sentence answers, we extract the single sentence from the
  larger answer that appears to best summarize the answer. The Stack
  Exchange data license requires that we provide additional metadata
  that allows participants to recover the source of each paired
  answer. Systems submitted to the shared task must not make use of
  this meta-data in anyway to assign STS scores or to otherwise inform
  the operation of the system.


- STS.input.belief: The data is collected from DEFT Committed Belief
  Annotation dataset (LDC2014E55).  All source documents are English
  Discussion Forum data.