SciTLDR: A Dataset for Scientific Extreme Summarization

SciTLDR: A Dataset for Scientific Extreme Summarization

TL;DR

SciTLDR is a scientific summarization dataset with approximately 3,200 papers and 5,400 concise summaries written by original authors and professional annotators.

Available in three JSON-lines variants with train, development, and test splits, it supports extractive and abstractive summarization, sentence selection, few-shot and zero-shot learning, and transfer learning.

What SciTLDR Is

SciTLDR is available on AIOZ AI Dataset for developing and evaluating systems that turn scientific papers into concise, informative summaries.

Its papers are paired with TLDRs from original authors and professional annotators, giving researchers multiple summary perspectives for scientific text.

The annotation workflow is designed to capture important paper contributions while reducing the effort required to create expert-derived summaries at scale.

AIOZ AI dataset documentation page for SciTLDR showing its introduction and JSON dataset structure

How the Dataset Is Organized

SciTLDR contains approximately 3,200 scientific papers and 5,400 TLDR summaries across three variants: A, AIC, FullText.

Each variant is distributed as JSON Lines with a 60/20/20 train, development, and test split:

  • train: 1,992 records
  • development: 618 records
  • test: 619 records

Each record includes:

  • source: Abstract or full-text sentences
  • source_labels: Binary oracle-sentence labels
  • rouge_scores: Precomputed ROUGE scores
  • paper_id: arXiv identifier
  • target: Author-written and expert-derived summaries
  • title: Paper title

What It Measures

SciTLDR evaluates how well models identify the key contributions of scientific papers and express them in concise summaries.

  • Extreme summarization
  • Extractive and abstractive summarization
  • Sentence selection
  • Few-shot and zero-shot evaluation
  • Multi-task and transfer learning
  • Expert annotation research

Its combination of source sentences, oracle labels, ROUGE scores, and multiple targets supports both summary generation and sentence-level analysis.

Key Dataset Details

  • Dataset: SciTLDR
  • Dataset type: Scientific extreme-summarization dataset
  • Scientific papers: ~3,200
  • TLDR summaries: 5,400
  • Summary sources: Original authors and professional annotators
  • Variants: A, AIC, FullText
  • File format: JSON Lines
  • Split ratio: 60/20/20
  • Reported split records: 1,992 train, 618 development, 619 test
  • Core tasks: Extractive and abstractive summarization
  • License: Apache License 2.0

What Builders Should Know

SciTLDR focuses on concise scientific-paper summaries, so results reflect scientific extreme summarization rather than summarization quality across every domain.

The source reports approximately 3,200 papers, while the listed split counts total 3,229 records. Use the individual split files when exact record counts matter.

Access SciTLDR on AIOZ AI

With approximately 3,200 papers, 5,400 summaries, and multiple author-written and expert-derived targets, SciTLDR provides a practical benchmark for scientific summarization.

Access SciTLDR on AIOZ AI and start evaluating how models condense scientific papers into focused summaries.

FAQ

Q1: What is SciTLDR used for?

It is used to train and evaluate scientific summarization models, sentence-selection methods, few-shot and zero-shot approaches, and transfer-learning workflows.

Q2: Who wrote the summaries in SciTLDR?

The dataset includes summaries from original paper authors and professional annotators.

Q3: How is SciTLDR distributed?

It is available in A, AIC, and FullText variants as JSON-lines files with train, development, and test splits.