human evaluation-Totto

Totto

一共40个case，每个case包含一张表格，一个gold reference，以及两个模型分别得到的表格描述，我们希望能够对每个模型的描述进行人工评测，分别从fluency（流利度）、coverage（覆盖性，侧重和gold的重合度）、relevance（对于表格内容是否真实的反应以及关联程度）、overal quality（整体印象的评分），分数从1-5分，1分最差，5分最好。
[0, 1, 4, 5, 8, 9, 13, 17, 18, 23, 24, 26, 29, 31, 32, 40, 41, 42, 48, 49, 54, 56, 59, 63, 64, 65, 68, 73, 74, 75, 81, 82, 87, 88, 89, 94, 95, 105, 108, 111]

case 1-1

link	https://en.wikipedia.org/wiki/Lisa_Fischer
gold	Fischer’s How Can I Ease the Pain won her a Grammy Award for Best Female R&B Vocal Performance in 1992
1	Lisa Fischer won the 1992 Soul Train Music Award for Best R&B/Soul Album – Female
2	Fischer won the 1992 Grammy Award for Best Female R&B Vocal Performance for “How Can I Ease the Pain”
3	“How Can I Ease the Pain” was nominated for the 1992 Soul Train Music Award

case 2-2

link	http://en.wikipedia.org/wiki/George_Washington_Colonials_football
gold	In 1956, George Washington Colonials scored 13–0 against Texas Western at the Sun Bowl
1	George Washington Colonials won the 1956 Sun Bowl against Texas Western
2	George Washington Colonials beat Texas Western 13–0 in the Sun Bowl
3	George Washington Colonials defeated Texas Western 13–0 in the Sun Bowl

case 3-5

link	http://en.wikipedia.org/wiki/MPEG-1
gold	Part 1 of the MPEG-1 system is defined in ISO/IEC-11172-1
1	ISO/IEC 11172-2 was first published in 1993
2	In 1993, MPEG-1 was released as part of the ISO/IEC 11172-2 release
3	The first version of MPEG-1 was released in 1993

case 4-6

link	https://en.wikipedia.org/wiki/Vartholomio
gold	Vartholomio´s population was 5,899 in 2011, of which 3,603 lived in the town
1	Vartholomio has a population of 3,216
2	In 2011, Vartholomio had a population of 4,271
3	In 2011 Vartholomio had a population of 3,603

case 5-9

link	http://en.wikipedia.org/wiki/DYKR
gold	DYKR FM was broadcasted as 96.7 WRocK Bacolod
1	WRocK Bacolod is a radio station in Bacolod, Cebu
2	Retro 95.5 (formerly WRocK Davao) has 95.5 MHz radio frequency
3	The DYKR-FM station is on 96.3 WRocK Cebu and 96.7 WRocK Bacolod

case 6-10

link	http://en.wikipedia.org/wiki/Lucius_Neratius_Marcellus
gold	Marcellus was consul twice, first in 95 and again in 129
1	Lucius Neratius Marcellus was consul with Lucius Neratius Marcellus
2	Lucius Neratius Marcellus was consul with Titus Flavius Clemens
3	Lucius Neratius Marcellus was the consul with Titus Avidius Quietus in 88

case 7-14

link	http://en.wikipedia.org/wiki/Julio_Agustin
gold	In 2012, Agustin directed Sweet Charity at The New Haarlem Arts Theatre
1	Agustin directed the 2007 show, Out of Line, at the Pennsylvania Center Stage
2	In 2008, Julio Agustin played the role of Director / Choreographer in Whistle Down the Wind
3	Julio Agustin worked as a Director/Choreographer for the North Carolina Theatre, in 2008, and as Director for the 2012 show Sweet Charity at the New Haarlem Arts Theatre

case 8-18

link	http://en.wikipedia.org/wiki/Ink_(song)
gold	“Ink” peaked at number 156 on the UK Singles Chart
1	Ink (song) peaked at number 156 on the UK Singles chart
2	Ink peaked at number 156 on the UK Singles chart
3	Ink peaked at 156 on the UK Singles Chart

case 9-19

link	http://en.wikipedia.org/wiki/Singular_(band)
gold	In 2010, Singular released its first single, “24.7 (Twenty-Four Seven)”
1	Single is Bao Bao (Tender)
2	Singular’s single, Bao Bao, peaked at number 1 on the 2010 chart
3	Singular (band) reached number 3 on the Billboard 200, with 24.7 (Twenty-Four Seven)

case 10-24

link	http://en.wikipedia.org/wiki/XHCJE-TDT
gold	XHCJE broadcast the Proyecto 40 network on digital subchannel 34.3 (PSIP 1.2)
1	XHCJE 3 is a digital subchannel with 34.1 DT and 1.2 PSIP
2	The XHCJE channel Azteca 13 has a 16:9 DT on 1080i
3	The XHCJE-TDT has a resolution of 34.1 DT

case 11-25

link	http://en.wikipedia.org/wiki/Ogl%C4%99d%C3%B3w
gold	In 2002, there were 392 people residing in Oględów, of which 47.7% were male and 52.3% were female
1	According to the 2002 census, there were 392 people residing in Ogldów, of whom 50.5% were male and 49.5% were female
2	According to the 2002 census, there were 392 people residing in Ogldów, of whom 50.5% were male and 49.5% were female
3	According to the 2002 census, there were 392 people residing in Ogldów, of whom 50.8% were male and 49.2% were female

case 12-27

link	http://en.wikipedia.org/wiki/Wollongong_Showground
gold	Wollongong Showground hosted two pool games in the 2003 Rugby World Cup. Canada defeated Tonga 24–7 in front of 15,630 fans, and France defeated the United States 41–14 in front of a crowd of 17,833
1	On 29 October 2003, Canada hosted the 2003 Rugby World Cup Pool D at the Wollongong Showground
2	Canada defeated Tonga in the 2003 Rugby World Cup Pool B
3	In 2003 Rugby World Cup Pool D, Canada hosted the United States and Tonga

case 13-30

link	http://en.wikipedia.org/wiki/List_of_ambassadors_of_the_United_States_to_Italy
gold	Porter was appointed as Minister to Italy in 1889
1	George P. Marsh served as the U.S. ambassador to Italy from 1871 to 1882
2	George P. Marsh served as a U.S. ambassador to Italy from 1871 to 1882
3	George P. Marsh was the U.S. Ambassador to Italy from 1871 to 1882

case 14-32

link	http://en.wikipedia.org/wiki/German_submarine_U-60_(1939)
gold	U-60 sank three ships for a total of 7,561 (GRT) and damaging one other of 15,434 GRT
1	U-60 sank a 4,373 ton German submarine in Kobe on 19 December 1939
2	On 19 December 1939, the German submarine U-60 raided Ulva with 4,373 tons
3	U-60 sank one ship a total of 4,373 gross register tons (GRT)

case 15-33

link	http://en.wikipedia.org/wiki/Portland_Timbers%E2%80%93Seattle_Sounders_rivalry
gold	On October 7, 2012, Portland played against Seattle in MLS, with the second-largest crowd of 66,452 at CenturyLink Field
1	The Portland Timbers hosted the Seattle Sounders at CenturyLink Field on August 25, 2013
2	The Portland Timbers–Seattle Sounders rivalry was held on August 25, 2013 at the CenturyLink Field
3	The Portland Timbers and Seattle Sounders rivalry was announced on August 25, 2013 in front of 67,385 fans at CenturyLink Field

case 16-41

link	http://en.wikipedia.org/wiki/Fiat_Croma
gold	Fiat Croma petrol engines, 1.8 L with 140 PS (103 kW), followed by the 2.2 L with 147 PS (108 kW)
1	The Fiat Croma-Petrol has a displacement of 1,796 cc and 140 PS (103 kW; 138 hp) at 6,300 rpm
2	Fiat Croma has 2,198 cc of torque and 147 PS (108 kW; 145 hp) at 5,800 rpm
3	The Fiat Croma is offered with an I4 DOHC 16V engine that generates 140 PS (103 kW; 138 hp) at 6,300 rpm

case 17-42

link	http://en.wikipedia.org/wiki/Minus_the_Bear
gold	Interpretaciones del Oso, an album of remixed songs from the album Menos el Oso, was released in 2007, by Suicide Squeeze Records
1	Menos el Oso was released on Suicide Squeeze Records in 2007
2	In 2007, Minus the Bear remixed the album, Interpretaciones del Oso
3	In 2007, Minus the Bear remixed the albums Interpretaciones del Oso by Suicide Squeeze Records

case 18-43

link	http://en.wikipedia.org/wiki/Rugby_League_World_Cup_venues
gold	The Sydney Cricket Ground Stadium has a spectator capacity of 48,000 and was used as a Rugby League World Cup venue in 1957
1	Brisbane hosted the Rugby League World Cup in 1957
2	The Rugby League World Cup took place in 1957 at the Brisbane Cricket Ground in Brisbane, with 29,636 spectators
3	The Rugby League World Cup venues in Australia were Brisbane Cricket Ground, Sydney Cricket Ground and the Sydney Cricket Ground

case 19-49

link	http://en.wikipedia.org/wiki/Paradise_in_Me
gold	The Paradise in Me album reached #121 on the U.S. Billboard 200 chart
1	Paradise in Me peaked at #112 on the Billboard 200 and #112 on the Billboard 200
2	Paradise peaked at number 121 on the Billboard 200
3	Paradise in Me peaked at number 121 on the Billboard 200 chart

case 20-50

link	http://en.wikipedia.org/wiki/Qamdo_Bamda_Airport
gold	At an elevation of 4,334 m (14,219 ft) above sea level, Qamdo Airport was the highest airport in the world
1	Qamdo Bamda Airport is a reference airport to El Alto International Airport
2	Qamdo Bamda Airport was preceded by El Alto International Airport
3	Qamdo Bamda Airport is the 3rd largest airport in the United States, and is surpassed by El Alto International Airport

case 21-55

link	http://en.wikipedia.org/wiki/1906_Boston_Americans_season
gold	Jesse Tannehill with winning record, at 13–11, while Bill Dinneen had the lowest ERA, at 2.92
1	George Winter had a record of 207 23
2	Cy Young led the 1906 Boston Americans season with a 3.19 ERA
3	Cy Young led the 1906 Boston Americans season with a 3.19 ERA

case 22-57

link	http://en.wikipedia.org/wiki/1942_South_American_Championship
gold	The 1942 South American Championship featured a match between Argentina and Ecuador in which Argentina scored a 12–0 drubbing Ecuador
1	Argentina won the 1942 South American Championship, beating Ecuador 12–0
2	Argentina defeated Ecuador 12–0 in the final round of the 1942 South American Championship
3	Argentina won the 1942 South American Championship beating Ecuador 12–0 in the final round

case 23-60

link	http://en.wikipedia.org/wiki/The_Don_(Nas_song)
gold	“The Don” debuted on the UK Singles Chart at number 196
1	The Don (Nas song) peaked at number 46 on the South Korean International Singles chart
2	The Don (Nas song) peaked at number 196 in the UK Singles
3	The Don (Nas song) peaked at number 196 on the UK Singles Chart

case 24-64

link	http://en.wikipedia.org/wiki/List_of_Coptic_Orthodox_Popes_of_Alexandria
gold	Pope John XVI is Pope of Alexandria & Patriarch
1	Popes & Patriarchs of Alexandria were Popes Mark V Marcos, John XV Yoannis, and St. Matthew III Matheos
2	Popes & Patriarchs of Alexandria English were the Popes of Alexandria from 1603 to 1619
3	St. Mark V Marcos was Pope of Alexandria & Patriarch

case 25-65

link	http://en.wikipedia.org/wiki/List_of_richest_literary_prizes
gold	The award amount is €50,000, making it the largest in the world for an award of this kind
1	The RBA Prize for Crime Writing is the world’s most lucrative crime fiction prize
2	Astrid Lindgren Memorial Award was awarded as the richest children’s literature prize in the world
3	The Astrid Lindgren Memorial Award is the richest children’s literature prize in the world

case 26-66

link	http://en.wikipedia.org/wiki/Richard_Thompson_(sprinter)
gold	In 2014, Richard Thompson made a record in 100 m with a run of 9.82 s
1	Thompson’s personal best was 6.51 seconds in the 60 m at Fayetteville, United States
2	Richard Thompson had a personal best of 9.82 seconds on 30 May 2008 in Fayetteville
3	Thompson made a personal best of 6.51 seconds in the 60 m at the Fayetteville, United States

case 27-69

link	http://en.wikipedia.org/wiki/Arundel_and_South_Downs_(UK_Parliament_constituency)
gold	Arundel and South Downs is a constituency of the UK Parliament represented by Nick Herbert of the Conservative Party since 2005
1	Arundel and South Downs is a UK Parliament constituency represented since 2005 by Nick Herbert of the Conservative Party
2	The Arundel and South Downs constituency was represented by Nick Herbert in the 2005 election
3	Arundel and South Downs is a constituency of the UK Parliament represented since 2005 by Nick Herbert, a Conservative

case 28-74

link	http://en.wikipedia.org/wiki/NunSexMonkRock
gold	NunSexMonkRock peaked at number 184 on the United States Billboard 200
1	NunSexMonkRock peaked at number 27 in the German Albums chart, number 38 in New Zealand Albums chart and number 18 in Norwegian Albums chart
2	NunSexMonkRock peaked at number 184 on the Billboard 200
3	NunSexMonkRock peaked at number 184 on the Billboard 200

case 29-75

link	http://en.wikipedia.org/wiki/2010_in_Shark_Fights
gold	The Shark Fights 11: Humes vs Buentello was an event held on May 22, 2010 at the Ector County Coliseum in Odessa, Texas
1	Shark Fights 12: Unfinished Business was an event held on June 26, 2010 at the Gamboa’s Outdoor Event Center in Amarillo, Texas
2	Shark Fights 12: Unfinished Business was an event held on June 26, 2010 at the Gamboa’s Outdoor Event Center in Amarillo, Texas
3	Shark Fights 12: Unfinished Business was an event held on June 26, 2010 at the Gamboa’s Outdoor Event Center in Amarillo, Texas

case 30-76

link	http://en.wikipedia.org/wiki/Anderson_and_Wise_(song)
gold	Chase’s version of Anderson and Wise was released on HMV HR 479 in 1972
1	Nash Chase’s “Anderson And Wise” was released on HMV HR 479 in 1972
2	In 1972, Anderson and Wise’s “Anderson And Wise” / “Yo Yo Mac” was released by Ode-50
3	The 1972 Anderson and Wise song, “Yo Yo Mac”, was released on Ode-50

case 31-82

link	http://en.wikipedia.org/wiki/Nishi-eifuku_Station
gold	In the fiscal year of 2011, Nishi-eifuku Station was used by an average of 18,087 passengers daily
1	In fiscal 2011, Nishi-eifuku Station was used by an average of 18,087 passengers daily
2	In fiscal 2011, Nishi-eifuku station was used by an average of 18,087 passengers daily
3	In fiscal 2011, the Nishi-eifuku Station was used by an average of 18,087 passengers daily

case 32-83

link	http://en.wikipedia.org/wiki/Bert_Richardson_(judge)
gold	In 1999, Richardson created state judgeship numbered 379
1	Bert Richardson was the first judge in the United States
2	Bert Richardson (judge) was the judge of New judgeship
3	Bert Richardson was the first judge to serve as a consul for the 129th District of New York

case 33-88

link	http://en.wikipedia.org/wiki/TMEM106A
gold	TMEM106A has 262 amino acids
1	TMEM106B is a protein with 274 amino acids
2	TMEM106A is a protein with 262 amino acids, NP_001127704
3	TMEM106B is a tetrasodium iodide with 274 amino acids

case 34-89

link	http://en.wikipedia.org/wiki/Leopard_2E
gold	The Leopard 2’s engine output options were M60A3’s 750 horsepower (559.27 kW) and the AMX-30EM2’s 850 horsepower (633.84 kW) engines
1	Leopard 2E has a capacity of 1,500 hp (1,479 hp, 1,103 kW)
2	The Leopard 2E weighs 63 t (69.45 tons) and is 750 hp (633.84 kW)
3	The Leopard 2E was rated 63 t (69.45 tons) and 55.6 t (63.68 tons)

case 35-90

link	http://en.wikipedia.org/wiki/List_of_heads_of_state_of_Yugoslavia
gold	Alexander I, was King of Yugoslavia from 1921 to 1934
1	Paul was the Prince Regent for Peter II of Yugoslavia
2	Peter I was a heads of state of Yugoslavia from 1921 to 1921
3	Peter I (29 June 1844 – 16 August 1921) was the head of state of Yugoslavia from 1921 until his death in 1921

case 36-95

link	http://en.wikipedia.org/wiki/The_Presets
gold	In 2012, The Presets won Best Live Act in the mix Awards
1	The Presets won Best Video at the Mix Awards for 2006’s “Are You the One?” and Best Video at the Mix Awards for 2012’s “The Presets”
2	“Are You the One?” was nominated for the Best Video Award in 2012 for The Presets
3	The Presets was nominated for the “Best Live Act” at the 2012 In the Mix Awards, and won the “Are You the One?” Award in 2006

case 37-96

link	http://en.wikipedia.org/wiki/Harrawi
gold	551 dunums of land was devoted to the farming of cereals
1	Arab population of Harrawi is 1,704 and Jewish population is 1,139
2	In the Harrawi-British Mandate era, there were 330 Arabs
3	In Harrawi, there was a population of 1,704 (Arab) people in the British Mandate era

case 38-106

link	http://en.wikipedia.org/wiki/1931%E2%80%9332_FA_Cup
gold	Newcastle United beat Arsenal 2–1 at Wembley
1	Newcastle United won the 1931–32 FA Cup beating Arsenal 2–1
2	Newcastle United defeated Arsenal 2–1 in the 1931–32 FA Cup
3	Newcastle United won the 1931–32 FA Cup beating Arsenal by 2–1

case 39-109

link	http://en.wikipedia.org/wiki/Idaho_Vandals_football
gold	Vandals defeated Bowling Green by a score of 43–42 in the 2009 Humanitarian Bowl
1	In 1998, Idaho Vandals defeated Southern Miss 42–35 in the Humanitarian Bowl
2	In 2016, Paul Petrino played the Famous Idaho Potato Bowl with Colorado State
3	The Idaho Vandals won the Humanitarian Bowl in 2009, and beat Bowling Green in the Humanitarian Bowl

case 40-112

link	http://en.wikipedia.org/wiki/Alessandro_Andrei
gold	In 1987 Alessandro Andrei broke the world record three times in the same competition with successive throws of 22.72, 22.84 & 22.91 m and finishing the sequence with a throw of 22.74 m
1	Alessandro Andrei holds the world record with a time of 22.19 seconds
2	Andrei finished the world record with 22.19 seconds and 22.74 seconds
3	Alessandro Andrei has set the record of 22.19 (22.37), 22.72 (22.72), 22.84 (22.84), 22.91 (22.91), 22.74 (22.74), and 22.74 (22.74)