Summary
In the ocr_lang column of the PDF metadata (TSV file), the language information contains a mix of jp and ja. This inconsistency likely stems from confusion between the country code jp (TLD) and the language code ja.
Details
The ocr_lang column is intended to represent the language of the PDF content. However, the metadata currently includes both:
This inconsistency could lead to issues in downstream processing, such as language detection or filtering.
Suggestion
We recommend standardizing all entries in the ocr_lang column by replacing jp with the correct language code ja.
Summary
In the ocr_lang column of the PDF metadata (TSV file), the language information contains a mix of jp and ja. This inconsistency likely stems from confusion between the country code jp (TLD) and the language code ja.
Details
The ocr_lang column is intended to represent the language of the PDF content. However, the metadata currently includes both:
jp: Commonly used as a country code (top-level domain)
ja: The correct ISO 639-1 language code for Japanese
This inconsistency could lead to issues in downstream processing, such as language detection or filtering.
Suggestion
We recommend standardizing all entries in the ocr_lang column by replacing jp with the correct language code ja.