DIP-SumEval: A Data Set of Human Summary Evaluations

This repository contains the summaries and evaluations from the paper 'A Dataset for the Analysis of Text Quality Dimensions in Summarization Evaluation' presented at LREC 2020. In total, the data set contains over 400 automatically generated summaries for 49 topics of an data set for multi-document summarization, 1274 judgements according to 11 text and summary quality criteria on a Likert-scale (1 to 5) performed by 26 trained annotators, and 43218 pairwise judgements according to 6 criteria performed by 64 crowd-workers.

If you want to use the data please cite the paper http://www.ke.tu-darmstadt.de/bibtex/publications/show/3200

Identifier
Source https://tudatalib.ulb.tu-darmstadt.de/handle/tudatalib/2303
Related Identifier http://www.ke.tu-darmstadt.de/bibtex/publications/show/3200
Related Identifier https://github.com/keelm/DIP-SumEval
Metadata Access https://tudatalib.ulb.tu-darmstadt.de/oai/openairedata?verb=GetRecord&metadataPrefix=oai_datacite&identifier=oai:tudatalib.ulb.tu-darmstadt.de:tudatalib/2303
Provenance
Creator Kronsbein, Tim; Mieskes, Margot; Loza Mencia, Eneldo
Publisher TU Darmstadt
Contributor Deutsche Forschungsgemeinschaft
Publication Year 2019
Funding Reference Deutsche Forschungsgemeinschaft info:eu-repo/grantAgreement/DFG/GRK1994/Prof.Roth/AIPHES
Rights Creative Commons Attribution 4.0; info:eu-repo/semantics/openAccess
OpenAccess true
Contact https://tudatalib.ulb.tu-darmstadt.de/page/contact
Representation
Language English
Resource Type Dataset
Format application/zip; application/octet-stream
Version v1.0 Base Version
Discipline Other