Research papers · Paper

A dataset of rated conceptual arguments

Emery Cooper and Caspar Oesterheld and Linh Chi Nguyen and Alexander Kastner and Ethan Perez · 29 Jul 2026

Read Source at arxiv.org ↗Read hereCC0 1.0

Outline · 35 pages

Introductionp. 1
A high-level approach: multi-dimensional evaluation of contextualized argumentsp. 2
Introducing a dataset on conceptual reasoningp. 3
Details on our datasetp. 3
Scoring functionsp. 6
Some basic experimentsp. 7
Related workp. 9
Obstacles for scoring functionsp. 16
Our loss functionsp. 17
Ranking lossesp. 17
Custom weighted lossp. 18
Rating validationp. 19
Common rating issuesp. 21
Examples of rater disagreementp. 23
The full rubricp. 24
Experimental detailsp. 34
Baseline prompt for eliciting overall ratingsp. 34

How to cite

A citation names the document, the version and the place.

arxiv:2607.27499/v1/p1

https://safetypage.noahairmet.com/d/arxiv/2607.27499/v1/p1

BibTeX

@misc{perez2026,
  title = {A dataset of rated conceptual arguments},
  author = {Emery Cooper and Caspar Oesterheld and Linh Chi Nguyen and Alexander Kastner and Ethan Perez},
  year = {2026},
  url = {https://arxiv.org/abs/2607.27499v1}
}

Versions

Version	Published	Read
v1 latest	29 Jul 2026	Here

Licence

Terms: CC0 1.0
So you: Read the full document here
Evidence: Licence link on the source page: http://creativecommons.org/publicdomain/zero/1.0/
Read at: export.arxiv.org/oai2?verb=GetRecord&metadataPrefix=arXiv&identifier=oai:arXiv.org:2607.27499, 10 Nov 2026