---
annotations_creators:
- machine-generated
language_creators:
- crowdsourced
languages:
  all_languages:
  - af
  - ar
  - az
  - be
  - ber
  - bg
  - bn
  - br
  - ca
  - cbk
  - cmn
  - cs
  - da
  - de
  - el
  - en
  - eo
  - es
  - et
  - eu
  - fi
  - fr
  - gl
  - gos
  - he
  - hi
  - hr
  - hu
  - hy
  - ia
  - id
  - ie
  - io
  - is
  - it
  - ja
  - jbo
  - kab
  - ko
  - kw
  - la
  - lfn
  - lt
  - mk
  - mr
  - nb
  - nds
  - nl
  - orv
  - ota
  - pes
  - pl
  - pt
  - rn
  - ro
  - ru
  - sl
  - sr
  - sv
  - tk
  - tl
  - tlh
  - toki
  - tr
  - tt
  - ug
  - uk
  - ur
  - vi
  - vo
  - war
  - wuu
  - yue
  af:
  - af
  ar:
  - ar
  az:
  - az
  be:
  - be
  ber:
  - ber
  bg:
  - bg
  bn:
  - bn
  br:
  - br
  ca:
  - ca
  cbk:
  - cbk
  cmn:
  - cmn
  cs:
  - cs
  da:
  - da
  de:
  - de
  el:
  - el
  en:
  - en
  eo:
  - eo
  es:
  - es
  et:
  - et
  eu:
  - eu
  fi:
  - fi
  fr:
  - fr
  gl:
  - gl
  gos:
  - gos
  he:
  - he
  hi:
  - hi
  hr:
  - hr
  hu:
  - hu
  hy:
  - hy
  ia:
  - ia
  id:
  - id
  ie:
  - ie
  io:
  - io
  is:
  - is
  it:
  - it
  ja:
  - ja
  jbo:
  - jbo
  kab:
  - kab
  ko:
  - ko
  kw:
  - kw
  la:
  - la
  lfn:
  - lfn
  lt:
  - lt
  mk:
  - mk
  mr:
  - mr
  nb:
  - nb
  nds:
  - nds
  nl:
  - nl
  orv:
  - orv
  ota:
  - ota
  pes:
  - pes
  pl:
  - pl
  pt:
  - pt
  rn:
  - rn
  ro:
  - ro
  ru:
  - ru
  sl:
  - sl
  sr:
  - sr
  sv:
  - sv
  tk:
  - tk
  tl:
  - tl
  tlh:
  - tlh
  toki:
  - toki
  tr:
  - tr
  tt:
  - tt
  ug:
  - ug
  uk:
  - uk
  ur:
  - ur
  vi:
  - vi
  vo:
  - vo
  war:
  - war
  wuu:
  - wuu
  yue:
  - yue
licenses:
- cc-by-2.0
multilinguality:
- multilingual
size_categories:
  af:
  - n<1K
  all_languages:
  - 1M<n<10M
  ar:
  - 1K<n<10K
  az:
  - n<1K
  be:
  - 1K<n<10K
  ber:
  - 10K<n<100K
  bg:
  - 1K<n<10K
  bn:
  - 1K<n<10K
  br:
  - 1K<n<10K
  ca:
  - n<1K
  cbk:
  - n<1K
  cmn:
  - 10K<n<100K
  cs:
  - 1K<n<10K
  da:
  - 10K<n<100K
  de:
  - 100K<n<1M
  el:
  - 10K<n<100K
  en:
  - 100K<n<1M
  eo:
  - 100K<n<1M
  es:
  - 10K<n<100K
  et:
  - n<1K
  eu:
  - n<1K
  fi:
  - 10K<n<100K
  fr:
  - 100K<n<1M
  gl:
  - n<1K
  gos:
  - n<1K
  he:
  - 10K<n<100K
  hi:
  - 1K<n<10K
  hr:
  - n<1K
  hu:
  - 10K<n<100K
  hy:
  - n<1K
  ia:
  - 1K<n<10K
  id:
  - 1K<n<10K
  ie:
  - n<1K
  io:
  - n<1K
  is:
  - 1K<n<10K
  it:
  - 100K<n<1M
  ja:
  - 10K<n<100K
  jbo:
  - 1K<n<10K
  kab:
  - 10K<n<100K
  ko:
  - n<1K
  kw:
  - 1K<n<10K
  la:
  - 1K<n<10K
  lfn:
  - 1K<n<10K
  lt:
  - 1K<n<10K
  mk:
  - 10K<n<100K
  mr:
  - 10K<n<100K
  nb:
  - 1K<n<10K
  nds:
  - 1K<n<10K
  nl:
  - 10K<n<100K
  orv:
  - n<1K
  ota:
  - n<1K
  pes:
  - 1K<n<10K
  pl:
  - 10K<n<100K
  pt:
  - 10K<n<100K
  rn:
  - n<1K
  ro:
  - 1K<n<10K
  ru:
  - 100K<n<1M
  sl:
  - n<1K
  sr:
  - 1K<n<10K
  sv:
  - 1K<n<10K
  tk:
  - 1K<n<10K
  tl:
  - 1K<n<10K
  tlh:
  - 1K<n<10K
  toki:
  - 1K<n<10K
  tr:
  - 100K<n<1M
  tt:
  - 1K<n<10K
  ug:
  - 1K<n<10K
  uk:
  - 10K<n<100K
  ur:
  - n<1K
  vi:
  - n<1K
  vo:
  - n<1K
  war:
  - n<1K
  wuu:
  - n<1K
  yue:
  - n<1K
source_datasets:
- extended|other-tatoeba
task_categories:
- conditional-text-generation
- text-classification
task_ids:
- conditional-text-generation-other-given-a-sentence-generate-a-paraphrase-either-in-same-language-or-another-language
- machine-translation
- semantic-similarity-classification
paperswithcode_id: tapaco
pretty_name: TaPaCo Corpus
---

# Dataset Card for TaPaCo Corpus

## Table of Contents
- [Dataset Description](#dataset-description)
  - [Dataset Summary](#dataset-summary)
  - [Supported Tasks and Leaderboards](#supported-tasks-and-leaderboards)
  - [Languages](#languages)
- [Dataset Structure](#dataset-structure)
  - [Data Instances](#data-instances)
  - [Data Fields](#data-fields)
  - [Data Splits](#data-splits)
- [Dataset Creation](#dataset-creation)
  - [Curation Rationale](#curation-rationale)
  - [Source Data](#source-data)
  - [Annotations](#annotations)
  - [Personal and Sensitive Information](#personal-and-sensitive-information)
- [Considerations for Using the Data](#considerations-for-using-the-data)
  - [Social Impact of Dataset](#social-impact-of-dataset)
  - [Discussion of Biases](#discussion-of-biases)
  - [Other Known Limitations](#other-known-limitations)
- [Additional Information](#additional-information)
  - [Dataset Curators](#dataset-curators)
  - [Licensing Information](#licensing-information)
  - [Citation Information](#citation-information)
  - [Contributions](#contributions)

## Dataset Description

- **Homepage:** [TaPaCo: A Corpus of Sentential Paraphrases for 73 Languages](https://zenodo.org/record/3707949#.X9Dh0cYza3I)
- **Paper:** [TaPaCo: A Corpus of Sentential Paraphrases for 73 Languages](https://www.aclweb.org/anthology/2020.lrec-1.848.pdf)
- **Point of Contact:** [Yves Scherrer](https://blogs.helsinki.fi/yvesscherrer/)

### Dataset Summary
A freely available paraphrase corpus for 73 languages extracted from the Tatoeba database. 
Tatoeba is a crowdsourcing project mainly geared towards language learners. Its aim is to provide example sentences 
and translations for particular linguistic constructions and words. The paraphrase corpus is created by populating a 
graph with Tatoeba sentences and equivalence links between sentences “meaning the same thing”. This graph is then 
traversed to extract sets of paraphrases. Several language-independent filters and pruning steps are applied to 
remove uninteresting sentences. A manual evaluation performed on three languages shows that between half and three 
quarters of inferred paraphrases are correct and that most remaining ones are either correct but trivial, 
or near-paraphrases that neutralize a morphological distinction. The corpus contains a total of 1.9 million 
sentences, with 200 – 250 000 sentences per language. It covers a range of languages for which, to our knowledge,
no other paraphrase dataset exists.

### Supported Tasks and Leaderboards
Paraphrase detection and generation have become popular tasks in NLP
and are increasingly integrated into a wide variety of common downstream tasks such as machine translation
, information retrieval, question answering, and semantic parsing. Most of the existing datasets
 cover only a single language – in most cases English – or a small number of languages. Furthermore, some paraphrase
  datasets focus on lexical and phrasal rather than sentential paraphrases, while others are created (semi
  -)automatically using machine translation.

The number of sentences per language ranges from 200 to 250 000, which makes the dataset
more suitable for fine-tuning and evaluation purposes than
for training. It is well-suited for multi-reference evaluation
of paraphrase generation models, as there is generally not a
single correct way of paraphrasing a given input sentence.

### Languages

The dataset contains paraphrases in Afrikaans, Arabic, Azerbaijani, Belarusian, Berber languages, Bulgarian, Bengali
, Breton, Catalan; Valencian, Chavacano, Mandarin, Czech, Danish, German, Greek, Modern (1453-), English, Esperanto
, Spanish; Castilian, Estonian, Basque, Finnish, French, Galician, Gronings, Hebrew, Hindi, Croatian, Hungarian
, Armenian, Interlingua (International Auxiliary Language Association), Indonesian, Interlingue; Occidental, Ido
, Icelandic, Italian, Japanese, Lojban, Kabyle, Korean, Cornish, Latin, Lingua Franca Nova\t, Lithuanian, Macedonian
, Marathi, Bokmål, Norwegian; Norwegian Bokmål, Low German; Low Saxon; German, Low; Saxon, Low, Dutch; Flemish, ]Old
 Russian, Turkish, Ottoman (1500-1928), Iranian Persian, Polish, Portuguese, Rundi, Romanian; Moldavian; Moldovan, 
 Russian, Slovenian, Serbian, Swedish, Turkmen, Tagalog, Klingon; tlhIngan-Hol, Toki Pona, Turkish, Tatar, 
 Uighur; Uyghur, Ukrainian, Urdu, Vietnamese, Volapük, Waray, Wu Chinese and Yue Chinese

## Dataset Structure

### Data Instances
Each data instance corresponds to a paraphrase, e.g.:
```
{ 
    'paraphrase_set_id': '1483',  
    'sentence_id': '5778896',
    'paraphrase': 'Ɣremt adlis-a.', 
    'lists': ['7546'], 
    'tags': [''],
    'language': 'ber'
}
```

### Data Fields
Each dialogue instance has the following fields:
- `paraphrase_set_id`:  a running number that groups together all sentences that are considered paraphrases of each
 other
- `sentence_id`: OPUS sentence id
- `paraphrase`: Sentential paraphrase in a given language for a given paraphrase_set_id
- `lists`: Contributors can add sentences to list in order to specify the original source of the data
- `tags`: Indicates morphological or phonological properties of the sentence when available
- `language`: Language identifier, one of the 73 languages that belong to this dataset.

### Data Splits

The dataset is having a single `train` split, contains a total of 1.9 million sentences, with 200 – 250 000
 sentences per language

## Dataset Creation

### Curation Rationale

[More Information Needed]

### Source Data

#### Initial Data Collection and Normalization

[More Information Needed]

#### Who are the source language producers?

[More Information Needed]

### Annotations

#### Annotation process

[More Information Needed]

#### Who are the annotators?

[More Information Needed]

### Personal and Sensitive Information

[More Information Needed]

## Considerations for Using the Data

### Social Impact of Dataset

[More Information Needed]

### Discussion of Biases

[More Information Needed]

### Other Known Limitations

[More Information Needed]

## Additional Information

### Dataset Curators

[More Information Needed]

### Licensing Information

Creative Commons Attribution 2.0 Generic

### Citation Information

```
@dataset{scherrer_yves_2020_3707949,
  author       = {Scherrer, Yves},
  title        = {{TaPaCo: A Corpus of Sentential Paraphrases for 73 Languages}},
  month        = mar,
  year         = 2020,
  publisher    = {Zenodo},
  version      = {1.0},
  doi          = {10.5281/zenodo.3707949},
  url          = {https://doi.org/10.5281/zenodo.3707949}
}
```

### Contributions

Thanks to [@pacman100](https://github.com/pacman100) for adding this dataset.
