Lingvanex 翻訳品質レポート

このレポートの目的は、最も一般的な 2 つの機械翻訳評価指標に基づいて、Lingvanex 言語モデルの翻訳品質を紹介することです。

Flores は、Facebook Research によってリリースされたオープンソースで公開されているデータ テスト セットであり、最も広範囲の言語ペアをカバーしています。

品質指標の説明

BLEU

BLEU は、n-gram に基づく自動メトリックです。これは、参照と比較した機械翻訳出力の n-gram の精度を測定し、過度に短い翻訳を罰するための簡潔性ペナルティによって重み付けされます。私たちは、sacreBLEU と呼ばれる BLEU の特定の実装を使用します。これは、セグメント スコアではなく、コーパス スコアを出力します。

参考文献

  • Papineni, Kishore, S. Roukos, T. Ward and Wei-Jing Zhu. “Bleu: a Method for Automatic Evaluation of Machine Translation.” ACL (2002).
  • Post, Matt. “A Call for Clarity in Reporting BLEU Scores.” WMT (2018).

COMET

COMET (Crosslingual Optimized Metric for Evaluation of Translation) は、トークンまたは文の埋め込みを使用して機械翻訳出力と参照翻訳の類似性を計算する、機械翻訳の自動評価のメトリックです。他のメトリックとは異なり、COMET は、編集後の作業、直接評価、または翻訳エラー分析の形でさまざまな種類の人間の判断を予測するようにトレーニングされています。

参考文献

言語ペア

注: ハード ドライブ上のモデルのサイズが小さいほど、GPU メモリの消費量が少なくなり、展開コストが削減されます。モデルのサイズが小さいほど、変換時間のパフォーマンスが向上します。GPU メモリのおおよその使用量を計算すると、ハード ドライブのモデル サイズ x 1.2 になります。

言語ペアモデルの
サイズ、mb
テスト
データ
BLEU
メトリック
COMET
メトリック
Afrikaans - English113,91Flores52,3386,43
English - Afrikaans113,91Flores42,5286,92
Albanian - English113,91Lingvanex56,4387,83
English - Albanian113,91Lingvanex57,0489,03
Amharic - English113,91Flores30,4283,78
English - Amharic113,91Flores14,1387,4
Arabic - English184,11Flores45,7987,95
English - Arabic190,63Flores33,4588,28
Armenian - English113,91Flores37,4786,93
English - Armenian113,91Flores22,0989,32
Armenian - Russian190,63Flores20,9785,06
Russian - Armenian190,63Flores15,7187,00
Azerbaijani - English113,91Flores23,7685,88
English - Azerbaijani113,91Flores17,7984,93
Azerbaijani - Russian190,63Flores16,9585,23
Russian - Azerbaijani190,63Flores13,1083,64
Basque - English113,91Flores32,3486,40
English - Basque113,91Flores21,6986,99
Belarusian - English113,91Flores19,7780,89
English - Belarusian113,91Flores15,5485,16
Bengali - English113,91Flores33,9387,86
English - Bengali239,56Flores23,4186,92
Bosnian - English113,91Flores41,6287,65
English - Bosnian113,91Flores35,5790,61
Bulgarian - English184,00Flores44,9788,28
English - Bulgarian184,00Flores45,9091,22
Catalan - English113,91Flores47,5488,55
English - Catalan113,91Flores45,6287,67
Cebuano - English113,91Flores41,1779,24
English - Cebuano113,91Flores35,4671,80
Chichewa - English113,91Flores22,8669,22
English - Chichewa113,91Flores17,4464,71
Chinese (Simplified) - English184,11Flores29,9186,27
English - Chinese (Simplified)184,11Flores42,2987,70
Chinese (Traditional) - English190,63Flores30,2886,51
English - Chinese (Traditional)190,63Flores36,0688,67
Corsican - English113,91Lingvanex59,9083,86
English - Corsican113,91Lingvanex54,2176,34
Croatian - English184,00Flores 10139,8088,08
English - Croatian184,00Flores34,9591,09
Czech - English184,00Flores41,8388,33
English - Czech184,00Flores36,5390,98
Danish - English184,00Flores51,2190,25
English - Danish184,02Flores49,8891,27
Dutch - English184,00Flores34,0487,20
English - Dutch184,00Flores29,7188,10
Esperanto - English113,91Flores41,6087,49
English - Esperanto113,91Flores32,0288,92
Estonian - English113,91Flores41,0789,33
English - Estonian113,91Flores31,4891,64
Filipino - English113,91Flores47,4186,85
English - Filipino113,91Flores37,7584,61
Finnish - English184,00Flores36,8789,76
English - Finnish184,00Flores27,0690,96
French - English190,65Flores48,8289,46
English - French190,65Flores53,2588,40
Frisian - English113,91Lingvanex65,0684,99
English - Frisian113,91Lingvanex54,8281,09
Gaelic-English113,91Flores29,6076,02
English-Gaelic113,91Flores24,7280,92
Galician - English113,91Flores40,5887,52
English - Galician113,91Flores37,7187,03
Georgian - Russian190,63Flores18,6085,80
Russian - Georgian190,63Flores13,8585,77
Georgian - English113,91Flores22,0989,32
English - Georgian113,91Flores16,5687,13
German - English190,65Flores46,7689,34
English - German184,02Flores43,0988,15
Greek - English184,00Flores36,1987,38
English - Greek184,00Flores29,5489,02
Gujarati-English113,91Flores142,0989,68
English - Gujarati113,91Flores28,0088,61
Haitian Creole - English113,91Flores30,0872,24
English - Haitian Creole113,91Flores28,2767,41
Hausa - English113,91Flores33,6478,80
English - Hausa113,91Flores28,7080,76
Hawaiian - English113,91Lingvanex45,1271,92
English - Hawaiian113,91Lingvanex64,2778,56
Hebrew - English184,11Flores46,3188,32
English - Hebrew184,11Flores36,0088,53
Hindi - English113,91Flores38,1888,29
English - Hindi113,91Flores67,6281,99
English - Hmong113,91Lingvanex60,9977,35
Hmong - English184,00Lingvanex45,8775,22
Hungarian - English184,00Flores38,7488,37
English - Hungarian184,00Flores31,4089,97
Icelandic - English113,91Flores37,2785,92
English - Icelandic113,91Flores31,9086,34
Igbo - English113,91Flores26,6068,85
English - Igbo113,91Flores19,0171,94
Indonesian - English184,02Flores46,6589,60
English - Indonesian184,02Flores50,7791,90
Italian - English184,00Flores34,7487,89
English - Italian184,00Flores32,5788,19
Irish - English113,91Flores44,1585,29
English - Irish113,91Flores38,8681,81
Japanese - English190,63Flores31,0588,08
English - Japanese190,63Flores39,6291,56
Javanese - English113,91Flores30,2976,20
English - Javanese113,91Flores28,7986,59
Kannada - English113,91Flores36,0187,38
English - Kannada113,91Flores65,8386,33
Kazakh - English113,91Flores36,2587,61
English - Kazakh113,91Flores26,4890,35
Kazakh - Russian190,63Flores22,7988,12
Russian - Kazakh190,63Flores19,1889,57
Khmer - English113,91Flores33,1985,77
English - Khmer113,91Flores4,7782,37
Kinyarwanda - English113,91Flores32,9873,73
English - Kinyarwanda113,91Flores25,8466,67
Korean - English113,91Flores32,8588,09
English - Korean113,91Flores33,6689,67
Kurdish - English113,91Flores28,8179,90
English - Kurdish113,91Flores12,8580,84
Kyrgyz - English113,91Flores24,2984,79
English - Kyrgyz113,91Flores16,6588,14
Kyrgyz - Russian190,63Flores16,3886,32
Russian - Kyrgyz190,63Flores13,4387,62
Lao - English113,91Flores31,4584,11
English - Lao113,91Flores59,5483,24
Latin - English113,91Lingvanex24,2474,81
English - Latin113,91Lingvanex14,8377,64
Latvian - English113,91Flores38,9587,67
English - Latvian113,91Flores37,8890,49
Lithuanian - English113,91Flores34,9686,24
English - Lithuanian113,91Flores31,2890,11
Luxembourgish - English113,91Flores45,5680,29
English - Luxembourgish113,91Flores27,2857,29
Macedonian - English113,91Flores44,3387,43
English - Macedonian113,91Flores38,1989,28
Malagasy - English113,91Lingvanex38,6083,10
English - Malagasy113,91Lingvanex41,7485,25
Malay - English184,11Flores46,0988,84
English - Malay184,11Flores44,6389,77
Malayalam - English113,91Flores39,2688,28
English - Malayalam113,91Flores23,1988,92
Maltese - English113,91Flores53,1181,83
English - Maltese113,91Flores47,5779,52
Maori - English113,91Flores29,1369,87
English - Maori113,91Flores17,4461,51
Marathi - English113,91Flores37,9487,49
English - Marathi113,91Flores21,4275,73
Mongolian- English113,91Flores31,4085,43
English - Mongolian113,91Flores20,0789,00
Myanmar - English113,91Flores23,1382,43
English - Myanmar113,91Flores55,7287,25
Nepali - English113,91Flores41,6789,94
English - Nepali113,91Flores64,9283,27
Norwegian - English184,00Flores44,2088,78
English - Norwegian184,02Flores35,8590,25
Odia - English113,91Flores32,3486,77
English - Odia113,91Flores21,0981,84
Persian - English184,02Flores39,0487,71
English - Persian184,11Flores27,2487,82
Polish - English184,00Flores30,9485,72
English - Polish184,00Flores24,2489,21
Portuguese (Brazil) - English184,02Flores51,4289,66
English - Portuguese (Brazil)184,02Flores51,6789,40
Portuguese - English184,02Flores50,9389,27
English - Portuguese184,02Flores51,3889,71
Punjabi - English113,91Flores38,1587,74
English - Punjabi113,91Flores28,9184,28
Pushto - English113,91Flores27,0579,68
English - Pushto113,91Flores15,0477,94
Romanian - English184,00Flores45,2289,30
English - Romanian179,58Flores43,6890,29
Russian - English190,65Flores37,8686,63
English - Russian190,65Flores34,6489,37
Russian - Belarusian190,63Lingvanex57,3595,38
Samoan - English113,91Flores27,8168,62
English - Samoan113,91Flores30,0767,20
Serbian - English184,00Flores43,0686,68
English - Serbian184,00Flores36,8788,49
Sesotho - English113,91Flores32,9172,64
English - Sesotho113,91Flores20,3666,18
Shona - English113,91Flores23,0570,04
English - Shona113,91Flores13,7462,88
Sindhi - English113,91Flores31,0979,66
English - Sindhi113,91Flores25,9080,37
Sinhala - English113,91Flores36,7387,92
English - Sinhala113,91Flores21,4287,65
Slovak - English184,11Flores42,3288,33
English - Slovak184,11Flores40,0891,01
Slovenian - English113,91Flores38,2587,76
English - Slovenian113,91Flores35,3990,20
Somali - English113,91Flores29,8977,25
English - Somali113,91Flores14,5680,85
Spanish - English184,02Flores31,4886,92
English - Spanish184,02Flores30,1786,72
Sundanese - English113,91Flores34,0281,42
English - Sundanese113,91Flores20,2379,65
Swahili - English113,91Flores44,9584,91
English - Swahili113,91Flores41,1485,40
Swedish - English184,00Flores51,8990,13
English - Swedish184,00Flores49,3991,24
Tajik - English113,91Flores33,7477,46
English - Tajik113,91Flores24,9776,28
Tajik - Russian190,63Flores23,1579,69
Russian - Tajik190,63Flores19,3875,59
Tamil - English113,91Flores35,1786,48
English - Tamil113,91Flores23,1289,55
Tatar - English113,91Flores30,3779,43
English - Tatar113,91Flores19,0169,26
Telugu-English113,91Flores42,0287,74
English-Telugu113,91Flores28,3087,60
Thai - English184,02Flores31,6387,71
English - Thai113,91Flores63,6888,57
Turkish - English184,00Flores41,8589,63
English - Turkish184,00Flores35,3390,82
Turkmen - English113,91Flores37,1080,90
English - Turkmen113,91Flores22,9167,00
Ukrainian - English184,00Flores41,5486,98
English - Ukrainian184,00Flores34,3089,88
Ukrainian - Russian190,63Flores28,0791,66
Russian - Ukrainian190,63Flores25,6292,16
Urdu - English113,91Flores31,0984,46
English - Urdu113,91Flores24,8081,71
Uyghur - English113,91Flores28,1786,26
English - Uyghur113,91Flores22,6686,21
Uzbek - English113,91Flores37,4287,64
English - Uzbek113,91Flores23,0490,32
Uzbek - Russian190,63Flores22,1185,40
Russian - Uzbek190,63Flores16,8687,37
Vietnamese - English184,00Flores38,6587,53
English - Vietnamese184,11Flores46,2889,16
Welsh - English113,91Flores61,3689,32
English - Welsh113,91Flores57,5588,83
Xhosa - English113,91Flores36,6578,10
English - Xhosa113,91Flores18,9077,05
Yiddish - English113,91Flores41,8976,88
English - Yiddish113,91Flores10,8867,67
Yoruba - English113,91Flores17,5461,56
English - Yoruba113,91Flores3,5755,39
Zulu - English113,91Flores36,8378,00
English - Zulu113,91Flores21,0678,12

詳細情報をリクエストする

* 必須項目

このフォームを送信することにより、私が受けるサービスの使用と私が提供する個人データには、それぞれ利用規約とプライバシーポリシーが適用されることに同意します。

完了

リクエストは正常に送信されました。

× 
Customize Consent Preferences

We use cookies to help you navigate efficiently and perform certain functions. You will find detailed information about all cookies under each consent category below.

The cookies that are categorized as "Necessary" are stored on your browser as they are essential for enabling the basic functionalities of the site.

We also use third-party cookies that help us analyze how you use this website, store your preferences, and provide the content and advertisements that are relevant to you. These cookies will only be stored in your browser with your prior consent.

You can choose to enable or disable some or all of these cookies but disabling some of them may affect your browsing experience.

Always Active

Necessary cookies are required to enable the basic features of this site, such as providing secure log-in or adjusting your consent preferences. These cookies do not store any personally identifiable data.

No cookies to display.

Always Active

Functional cookies help perform certain functionalities like sharing the content of the website on social media platforms, collecting feedback, and other third-party features.

No cookies to display.

Always Active

Analytical cookies are used to understand how visitors interact with the website. These cookies help provide information on metrics such as the number of visitors, bounce rate, traffic source, etc.

No cookies to display.

Always Active

Performance cookies are used to understand and analyze the key performance indexes of the website which helps in delivering a better user experience for the visitors.

No cookies to display.

Always Active

Advertisement cookies are used to provide visitors with customized advertisements based on the pages you visited previously and to analyze the effectiveness of the ad campaigns.

No cookies to display.