Lingvanexin käännöslaaturaportti

Tämän raportin tavoitteena on esitellä Lingvanex-kielimallien käännösten laatua kahden suosituimman konekäännösarviointimittarin mukaan.

Flores on avoimen lähdekoodin ja julkisesti saatavilla oleva testijoukko, jonka julkaisi Facebook Research ja jolla on suurin kieliparien kattavuus.

Laatumittarien kuvaus

BLEU

BLEU on automaattinen mittari, joka perustuu n-grammiin. Se mittaa konekäännöstulosten n-grammien tarkkuutta viitteeseen verrattuna painotettuna lyhyyden rangaistuksella liian lyhyiden käännösten rankaisemiseksi. Käytämme erityistä BLEU:n toteutusta, nimeltään sacreBLEU. Se tulostaa korpuspisteitä, ei segmenttipisteitä.

Viitteet

  • Papineni, Kishore, S. Roukos, T. Ward and Wei-Jing Zhu. “Bleu: a Method for Automatic Evaluation of Machine Translation.” ACL (2002).
  • Post, Matt. “A Call for Clarity in Reporting BLEU Scores.” WMT (2018).

COMET

COMET (Crosslingual Optimized Metric for Evaluation of Translation) on konekäännösten automaattisen arvioinnin mittari, joka laskee konekäännöstuloksen ja viitekäännöksen samankaltaisuuden käyttämällä tunnus- tai lauseen upotuksia. Toisin kuin muut mittarit, COMET on koulutettu ennustamaan erilaisia ​​inhimillisiä arvioita jälkieditoinnin, suoran arvioinnin tai käännösvirheanalyysin muodossa.

Viitteet

Kieliparit

Huomautus: Kiintolevyllä olevien mallien pienempi koko tarkoittaa pienempää GPU-muistin kulutusta, mikä vähentää käyttöönottokustannuksia. Pienemmällä mallikoolla on parempi suorituskyky käännösajassa. GPU-muistin likimääräinen käyttö lasketaan kiintolevyn mallin kokoina x 1,2

KielipariMallin
Koko, mb
Testata
Data
BLEU
Metrinen
COMET
Metrinen
Afrikaans - English113,91Flores52,3386,43
English - Afrikaans113,91Flores42,5286,92
Albanian - English113,91Lingvanex56,4387,83
English - Albanian113,91Lingvanex57,0489,03
Amharic - English113,91Flores30,4283,78
English - Amharic113,91Flores14,1387,4
Arabic - English184,11Flores45,7987,95
English - Arabic190,63Flores33,4588,28
Armenian - English113,91Flores37,4786,93
English - Armenian113,91Flores22,0989,32
Armenian - Russian190,63Flores20,9785,06
Russian - Armenian190,63Flores15,7187,00
Azerbaijani - English113,91Flores23,7685,88
English - Azerbaijani113,91Flores17,7984,93
Azerbaijani - Russian190,63Flores16,9585,23
Russian - Azerbaijani190,63Flores13,1083,64
Basque - English113,91Flores32,3486,40
English - Basque113,91Flores21,6986,99
Belarusian - English113,91Flores19,7780,89
English - Belarusian113,91Flores15,5485,16
Bengali - English113,91Flores33,9387,86
English - Bengali239,56Flores23,4186,92
Bosnian - English113,91Flores41,6287,65
English - Bosnian113,91Flores35,5790,61
Bulgarian - English184,00Flores44,9788,28
English - Bulgarian184,00Flores45,9091,22
Catalan - English113,91Flores47,5488,55
English - Catalan113,91Flores45,6287,67
Cebuano - English113,91Flores41,1779,24
English - Cebuano113,91Flores35,4671,80
Chichewa - English113,91Flores22,8669,22
English - Chichewa113,91Flores17,4464,71
Chinese (Simplified) - English184,11Flores29,9186,27
English - Chinese (Simplified)184,11Flores42,2987,70
Chinese (Traditional) - English190,63Flores30,2886,51
English - Chinese (Traditional)190,63Flores36,0688,67
Corsican - English113,91Lingvanex59,9083,86
English - Corsican113,91Lingvanex54,2176,34
Croatian - English184,00Flores 10139,8088,08
English - Croatian184,00Flores34,9591,09
Czech - English184,00Flores41,8388,33
English - Czech184,00Flores36,5390,98
Danish - English184,00Flores51,2190,25
English - Danish184,02Flores49,8891,27
Dutch - English184,00Flores34,0487,20
English - Dutch184,00Flores29,7188,10
Esperanto - English113,91Flores41,6087,49
English - Esperanto113,91Flores32,0288,92
Estonian - English113,91Flores41,0789,33
English - Estonian113,91Flores31,4891,64
Filipino - English113,91Flores47,4186,85
English - Filipino113,91Flores37,7584,61
Finnish - English184,00Flores36,8789,76
English - Finnish184,00Flores27,0690,96
French - English190,65Flores48,8289,46
English - French190,65Flores53,2588,40
Frisian - English113,91Lingvanex65,0684,99
English - Frisian113,91Lingvanex54,8281,09
Gaelic-English113,91Flores29,6076,02
English-Gaelic113,91Flores24,7280,92
Galician - English113,91Flores40,5887,52
English - Galician113,91Flores37,7187,03
Georgian - Russian190,63Flores18,6085,80
Russian - Georgian190,63Flores13,8585,77
Georgian - English113,91Flores22,0989,32
English - Georgian113,91Flores16,5687,13
German - English190,65Flores46,7689,34
English - German184,02Flores43,0988,15
Greek - English184,00Flores36,1987,38
English - Greek184,00Flores29,5489,02
Gujarati-English113,91Flores142,0989,68
English - Gujarati113,91Flores28,0088,61
Haitian Creole - English113,91Flores30,0872,24
English - Haitian Creole113,91Flores28,2767,41
Hausa - English113,91Flores33,6478,80
English - Hausa113,91Flores28,7080,76
Hawaiian - English113,91Lingvanex45,1271,92
English - Hawaiian113,91Lingvanex64,2778,56
Hebrew - English184,11Flores46,3188,32
English - Hebrew184,11Flores36,0088,53
Hindi - English113,91Flores38,1888,29
English - Hindi113,91Flores67,6281,99
English - Hmong113,91Lingvanex60,9977,35
Hmong - English184,00Lingvanex45,8775,22
Hungarian - English184,00Flores38,7488,37
English - Hungarian184,00Flores31,4089,97
Icelandic - English113,91Flores37,2785,92
English - Icelandic113,91Flores31,9086,34
Igbo - English113,91Flores26,6068,85
English - Igbo113,91Flores19,0171,94
Indonesian - English184,02Flores46,6589,60
English - Indonesian184,02Flores50,7791,90
Italian - English184,00Flores34,7487,89
English - Italian184,00Flores32,5788,19
Irish - English113,91Flores44,1585,29
English - Irish113,91Flores38,8681,81
Japanese - English190,63Flores31,0588,08
English - Japanese190,63Flores39,6291,56
Javanese - English113,91Flores30,2976,20
English - Javanese113,91Flores28,7986,59
Kannada - English113,91Flores36,0187,38
English - Kannada113,91Flores65,8386,33
Kazakh - English113,91Flores36,2587,61
English - Kazakh113,91Flores26,4890,35
Kazakh - Russian190,63Flores22,7988,12
Russian - Kazakh190,63Flores19,1889,57
Khmer - English113,91Flores33,1985,77
English - Khmer113,91Flores4,7782,37
Kinyarwanda - English113,91Flores32,9873,73
English - Kinyarwanda113,91Flores25,8466,67
Korean - English113,91Flores32,8588,09
English - Korean113,91Flores33,6689,67
Kurdish - English113,91Flores28,8179,90
English - Kurdish113,91Flores12,8580,84
Kyrgyz - English113,91Flores24,2984,79
English - Kyrgyz113,91Flores16,6588,14
Kyrgyz - Russian190,63Flores16,3886,32
Russian - Kyrgyz190,63Flores13,4387,62
Lao - English113,91Flores31,4584,11
English - Lao113,91Flores59,5483,24
Latin - English113,91Lingvanex24,2474,81
English - Latin113,91Lingvanex14,8377,64
Latvian - English113,91Flores38,9587,67
English - Latvian113,91Flores37,8890,49
Lithuanian - English113,91Flores34,9686,24
English - Lithuanian113,91Flores31,2890,11
Luxembourgish - English113,91Flores45,5680,29
English - Luxembourgish113,91Flores27,2857,29
Macedonian - English113,91Flores44,3387,43
English - Macedonian113,91Flores38,1989,28
Malagasy - English113,91Lingvanex38,6083,10
English - Malagasy113,91Lingvanex41,7485,25
Malay - English184,11Flores46,0988,84
English - Malay184,11Flores44,6389,77
Malayalam - English113,91Flores39,2688,28
English - Malayalam113,91Flores23,1988,92
Maltese - English113,91Flores53,1181,83
English - Maltese113,91Flores47,5779,52
Maori - English113,91Flores29,1369,87
English - Maori113,91Flores17,4461,51
Marathi - English113,91Flores37,9487,49
English - Marathi113,91Flores21,4275,73
Mongolian- English113,91Flores31,4085,43
English - Mongolian113,91Flores20,0789,00
Myanmar - English113,91Flores23,1382,43
English - Myanmar113,91Flores55,7287,25
Nepali - English113,91Flores41,6789,94
English - Nepali113,91Flores64,9283,27
Norwegian - English184,00Flores44,2088,78
English - Norwegian184,02Flores35,8590,25
Odia - English113,91Flores32,3486,77
English - Odia113,91Flores21,0981,84
Persian - English184,02Flores39,0487,71
English - Persian184,11Flores27,2487,82
Polish - English184,00Flores30,9485,72
English - Polish184,00Flores24,2489,21
Portuguese (Brazil) - English184,02Flores51,4289,66
English - Portuguese (Brazil)184,02Flores51,6789,40
Portuguese - English184,02Flores50,9389,27
English - Portuguese184,02Flores51,3889,71
Punjabi - English113,91Flores38,1587,74
English - Punjabi113,91Flores28,9184,28
Pushto - English113,91Flores27,0579,68
English - Pushto113,91Flores15,0477,94
Romanian - English184,00Flores45,2289,30
English - Romanian179,58Flores43,6890,29
Russian - English190,65Flores37,8686,63
English - Russian190,65Flores34,6489,37
Russian - Belarusian190,63Lingvanex57,3595,38
Samoan - English113,91Flores27,8168,62
English - Samoan113,91Flores30,0767,20
Serbian - English184,00Flores43,0686,68
English - Serbian184,00Flores36,8788,49
Sesotho - English113,91Flores32,9172,64
English - Sesotho113,91Flores20,3666,18
Shona - English113,91Flores23,0570,04
English - Shona113,91Flores13,7462,88
Sindhi - English113,91Flores31,0979,66
English - Sindhi113,91Flores25,9080,37
Sinhala - English113,91Flores36,7387,92
English - Sinhala113,91Flores21,4287,65
Slovak - English184,11Flores42,3288,33
English - Slovak184,11Flores40,0891,01
Slovenian - English113,91Flores38,2587,76
English - Slovenian113,91Flores35,3990,20
Somali - English113,91Flores29,8977,25
English - Somali113,91Flores14,5680,85
Spanish - English184,02Flores31,4886,92
English - Spanish184,02Flores30,1786,72
Sundanese - English113,91Flores34,0281,42
English - Sundanese113,91Flores20,2379,65
Swahili - English113,91Flores44,9584,91
English - Swahili113,91Flores41,1485,40
Swedish - English184,00Flores51,8990,13
English - Swedish184,00Flores49,3991,24
Tajik - English113,91Flores33,7477,46
English - Tajik113,91Flores24,9776,28
Tajik - Russian190,63Flores23,1579,69
Russian - Tajik190,63Flores19,3875,59
Tamil - English113,91Flores35,1786,48
English - Tamil113,91Flores23,1289,55
Tatar - English113,91Flores30,3779,43
English - Tatar113,91Flores19,0169,26
Telugu-English113,91Flores42,0287,74
English-Telugu113,91Flores28,3087,60
Thai - English184,02Flores31,6387,71
English - Thai113,91Flores63,6888,57
Turkish - English184,00Flores41,8589,63
English - Turkish184,00Flores35,3390,82
Turkmen - English113,91Flores37,1080,90
English - Turkmen113,91Flores22,9167,00
Ukrainian - English184,00Flores41,5486,98
English - Ukrainian184,00Flores34,3089,88
Ukrainian - Russian190,63Flores28,0791,66
Russian - Ukrainian190,63Flores25,6292,16
Urdu - English113,91Flores31,0984,46
English - Urdu113,91Flores24,8081,71
Uyghur - English113,91Flores28,1786,26
English - Uyghur113,91Flores22,6686,21
Uzbek - English113,91Flores37,4287,64
English - Uzbek113,91Flores23,0490,32
Uzbek - Russian190,63Flores22,1185,40
Russian - Uzbek190,63Flores16,8687,37
Vietnamese - English184,00Flores38,6587,53
English - Vietnamese184,11Flores46,2889,16
Welsh - English113,91Flores61,3689,32
English - Welsh113,91Flores57,5588,83
Xhosa - English113,91Flores36,6578,10
English - Xhosa113,91Flores18,9077,05
Yiddish - English113,91Flores41,8976,88
English - Yiddish113,91Flores10,8867,67
Yoruba - English113,91Flores17,5461,56
English - Yoruba113,91Flores3,5755,39
Zulu - English113,91Flores36,8378,00
English - Zulu113,91Flores21,0678,12

Pyydä lisätietoja

* Pakolliset kentät

Lähettämällä tämän lomakkeen hyväksyn, että palveluehdot ja tietosuojakäytäntö säätelevät vastaanottamieni palveluiden käyttöä ja toimittamiani henkilötietoja.

Valmis

Pyyntösi on lähetetty onnistuneesti.

× 
Customize Consent Preferences

We use cookies to help you navigate efficiently and perform certain functions. You will find detailed information about all cookies under each consent category below.

The cookies that are categorized as "Necessary" are stored on your browser as they are essential for enabling the basic functionalities of the site.

We also use third-party cookies that help us analyze how you use this website, store your preferences, and provide the content and advertisements that are relevant to you. These cookies will only be stored in your browser with your prior consent.

You can choose to enable or disable some or all of these cookies but disabling some of them may affect your browsing experience.

Always Active

Necessary cookies are required to enable the basic features of this site, such as providing secure log-in or adjusting your consent preferences. These cookies do not store any personally identifiable data.

No cookies to display.

Always Active

Functional cookies help perform certain functionalities like sharing the content of the website on social media platforms, collecting feedback, and other third-party features.

No cookies to display.

Always Active

Analytical cookies are used to understand how visitors interact with the website. These cookies help provide information on metrics such as the number of visitors, bounce rate, traffic source, etc.

No cookies to display.

Always Active

Performance cookies are used to understand and analyze the key performance indexes of the website which helps in delivering a better user experience for the visitors.

No cookies to display.

Always Active

Advertisement cookies are used to provide visitors with customized advertisements based on the pages you visited previously and to analyze the effectiveness of the ad campaigns.

No cookies to display.