---
annotations_creators:
- expert-generated
- machine-generated
language_creators:
- machine-generated
languages:
- en
licenses:
- mit
multilinguality:
- monolingual
size_categories:
- 10K<n<100K
source_datasets:
- original
task_categories:
- other
- sequence-modeling
task_ids:
- other-other-token-classification-of-text-errors
- slot-filling
paperswithcode_id: null
pretty_name: YouTube Caption Corrections
---

# Dataset Card for YouTube Caption Corrections

## Table of Contents
- [Dataset Description](#dataset-description)
  - [Dataset Summary](#dataset-summary)
  - [Supported Tasks and Leaderboards](#supported-tasks-and-leaderboards)
  - [Languages](#languages)
- [Dataset Structure](#dataset-structure)
  - [Data Instances](#data-instances)
  - [Data Fields](#data-fields)
  - [Data Splits](#data-splits)
- [Dataset Creation](#dataset-creation)
  - [Curation Rationale](#curation-rationale)
  - [Source Data](#source-data)
  - [Annotations](#annotations)
  - [Personal and Sensitive Information](#personal-and-sensitive-information)
- [Considerations for Using the Data](#considerations-for-using-the-data)
  - [Social Impact of Dataset](#social-impact-of-dataset)
  - [Discussion of Biases](#discussion-of-biases)
  - [Other Known Limitations](#other-known-limitations)
- [Additional Information](#additional-information)
  - [Dataset Curators](#dataset-curators)
  - [Licensing Information](#licensing-information)
  - [Citation Information](#citation-information)
  - [Contributions](#contributions)

## Dataset Description

- **Homepage:** https://github.com/2dot71mily/youtube_captions_corrections
- **Repository:** https://github.com/2dot71mily/youtube_captions_corrections
- **Paper:** [N/A]
- **Leaderboard:** [N/A]
- **Point of Contact:** Emily McMilin

### Dataset Summary

This dataset is built from pairs of YouTube captions where both an auto-generated and a manually-corrected caption are available for a single specified language. It currently only in English, but scripts at repo support other languages. The motivation for creating it was from viewing errors in auto-generated captions at a recent virtual conference, with the hope that there could be some way to help correct those errors.

The dataset in the repo at https://github.com/2dot71mily/youtube_captions_corrections records in a non-destructive manner all the differences between an auto-generated and a manually-corrected caption for thousands of videos. The dataset here focuses on the subset of those differences which are mutual and have the same size in token length difference, which means it excludes token insertion or deletion differences between the two captions. Therefore dataset here remains a non-destructive representation of the original auto-generated captions, but excludes some of the differences that are found in the manually-corrected captions.

### Supported Tasks and Leaderboards

- `token-classification`: The tokens in `default_seq` are from the auto-generated YouTube captions. If `diff_type` is labeled greater than `0` at a given index, then the associated token in same index in the `default_seq` was found to be different to the token in the manually-corrected YouTube caption, and therefore we assume it is an error. A model can be trained to learn when there are errors in the auto-generated captions.

- `slot-filling`: The `correction_seq` is sparsely populated with tokens from the manually-corrected YouTube captions in the locations where there was found to be a difference to the token in the auto-generated YouTube captions. These 'incorrect' tokens in the `default_seq` can be masked in the locations where `diff_type` is labeled greater than `0`, so that a model can be trained to hopefully find a better word to fill in, rather than the 'incorrect' one.

End to end, the models could maybe first identify and then replace (with suitable alternatives) errors in YouTube and other auto-generated captions that are lacking manual corrections

### Languages

English

## Dataset Structure

### Data Instances

If `diff_type` is labeled greater than `0` at a given index, then the associated token in same index in the `default_seq` was found to have a difference to the token in the manually-corrected YouTube caption. The `correction_seq` is sparsely populated with tokens from the manually-corrected YouTube captions at those locations of differences.

`diff_type` labels for tokens are as follows:
0: No difference
1: Case based difference, e.g. `hello` vs `Hello`
2: Punctuation difference, e.g. `hello` vs `hello`
3: Case and punctuation difference, e.g. `hello` vs `Hello,`
4: Word difference with same stem, e.g. `thank` vs `thanked`
5: Digit difference, e.g. `2` vs `two`
6: Intra-word punctuation difference, e.g. `autogenerated` vs `auto-generated`
7: Unknown type of difference, e.g. `laughter` vs `draft`
8: Reserved for unspecified difference

{
    'video_titles': '_QUEXsHfsA0', 
    'default_seq': ['you', 'see', "it's", 'a', 'laughter', 'but', 'by', 'the', 'time', 'you', 'see', 'this', 'it', "won't", 'be', 'so', 'we', 'have', 'a', 'big']
    'correction_seq':  ['', 'see,', '', '', 'draft,', '', '', '', '', '', 'read', 'this,', '', '', 'be.', 'So', '', '', '', '']
    'diff_type': [0, 2, 0, 0, 7, 0, 0, 0, 0, 0, 7, 2, 0, 0, 2, 1, 0, 0, 0, 0]
}

### Data Fields

- 'video_ids': Unique ID used by YouTube for each video. Can paste into `https://www.youtube.com/watch?v=<{video_ids}` to see video
- 'default_seq': Tokenized auto-generated YouTube captions for the video
- 'correction_seq':  Tokenized manually-corrected YouTube captions only at those locations, where there is a difference between the auto-generated and manually-corrected captions
- 'diff_type': A value greater than `0` at every token where there is a difference between the auto-generated and manually-corrected captions

### Data Splits

No data splits

## Dataset Creation

### Curation Rationale

It was created after viewing errors in auto-generated captions at a recent virtual conference, with the hope that there could be some way to help correct those errors.

### Source Data

#### Initial Data Collection and Normalization

All captions are requested via `googleapiclient` and `youtube_transcript_api` at the `channel_id` and language granularity, using scripts written at https://github.com/2dot71mily/youtube_captions_corrections.

The captions are tokenized on spaces and the manually-corrected sequence has here been reduced to only include differences between it and the auto-generated sequence.

#### Who are the source language producers?

Auto-generated scripts are from YouTube and the manually-corrected scripts are from creators, and any support they may have (e.g. community or software support)

### Annotations

#### Annotation process

Scripts at repo, https://github.com/2dot71mily/youtube_captions_corrections take a diff of the two captions and use this to create annotations.

#### Who are the annotators?

YouTube creators, and any support they may have (e.g. community or software support)

### Personal and Sensitive Information

All content publicly available on YouTube

## Considerations for Using the Data

### Social Impact of Dataset

[More Information Needed]

### Discussion of Biases

[More Information Needed]

### Other Known Limitations

[More Information Needed]

## Additional Information

### Dataset Curators

Emily McMilin

### Licensing Information

MIT License

### Citation Information

https://github.com/2dot71mily/youtube_captions_corrections

### Contributions

Thanks to [@2dot71mily](https://github.com/2dot71mily) for adding this dataset.
