Laporan Kualiti Terjemahan Lingvanex
Matlamat laporan ini adalah untuk mempamerkan kualiti terjemahan model bahasa Lingvanex mengikut dua metrik penilaian terjemahan mesin yang paling popular.
Flores ialah set ujian data sumber terbuka dan tersedia untuk umum yang dikeluarkan oleh Facebook Research dan mempunyai liputan pasangan bahasa terbesar.
Penerangan metrik kualiti
BLEU
BLEU ialah metrik automatik berdasarkan n-gram. Ia mengukur ketepatan n-gram output terjemahan mesin berbanding rujukan, ditimbang dengan penalti ringkas untuk menghukum terjemahan yang terlalu pendek. Kami menggunakan pelaksanaan tertentu BLEU, dipanggil sacreBLEU. Ia mengeluarkan skor korpus, bukan skor segmen.
Rujukan
- Papineni, Kishore, S. Roukos, T. Ward and Wei-Jing Zhu. “Bleu: a Method for Automatic Evaluation of Machine Translation.” ACL (2002).
- Post, Matt. “A Call for Clarity in Reporting BLEU Scores.” WMT (2018).
COMET
COMET (Metrik Dioptimumkan Merentas Bahasa untuk Penilaian Terjemahan) ialah metrik untuk penilaian automatik terjemahan mesin yang mengira persamaan antara output terjemahan mesin dan terjemahan rujukan menggunakan token atau pembenaman ayat. Tidak seperti metrik lain, COMET dilatih untuk meramalkan pelbagai jenis pertimbangan manusia dalam bentuk usaha pasca penyuntingan, penilaian langsung atau analisis ralat terjemahan.
Rujukan
- COMET - https://machinetranslate.org/comet
- COMET: High-quality Machine Translation Evaluation - https://unbabel.github.io/COMET/html/index.html#comet-high-quality-machine-translation-evaluation
Pasangan bahasa
Nota: Saiz model yang lebih rendah pada cakera keras bermakna penggunaan memori GPU yang lebih rendah yang membawa kepada penurunan kos penggunaan. Saiz model yang lebih rendah mempunyai prestasi yang lebih baik dalam masa terjemahan. Anggaran penggunaan memori GPU dikira sebagai saiz model cakera keras x 1.2
Pasangan Bahasa | Model Saiz, mb | Ujian Data | BLEU Metrik | COMET Metrik |
Afrikaans - English | 113,91 | Flores | 52,33 | 86,43 |
English - Afrikaans | 113,91 | Flores | 42,52 | 86,92 |
Albanian - English | 113,91 | Lingvanex | 56,43 | 87,83 |
English - Albanian | 113,91 | Lingvanex | 57,04 | 89,03 |
Amharic - English | 113,91 | Flores | 30,42 | 83,78 |
English - Amharic | 113,91 | Flores | 14,13 | 87,4 |
Arabic - English | 184,11 | Flores | 45,79 | 87,95 |
English - Arabic | 190,63 | Flores | 33,45 | 88,28 |
Armenian - English | 113,91 | Flores | 37,47 | 86,93 |
English - Armenian | 113,91 | Flores | 22,09 | 89,32 |
Armenian - Russian | 190,63 | Flores | 20,97 | 85,06 |
Russian - Armenian | 190,63 | Flores | 15,71 | 87,00 |
Azerbaijani - English | 113,91 | Flores | 23,76 | 85,88 |
English - Azerbaijani | 113,91 | Flores | 17,79 | 84,93 |
Azerbaijani - Russian | 190,63 | Flores | 16,95 | 85,23 |
Russian - Azerbaijani | 190,63 | Flores | 13,10 | 83,64 |
Basque - English | 113,91 | Flores | 32,34 | 86,40 |
English - Basque | 113,91 | Flores | 21,69 | 86,99 |
Belarusian - English | 113,91 | Flores | 19,77 | 80,89 |
English - Belarusian | 113,91 | Flores | 15,54 | 85,16 |
Bengali - English | 113,91 | Flores | 33,93 | 87,86 |
English - Bengali | 239,56 | Flores | 23,41 | 86,92 |
Bosnian - English | 113,91 | Flores | 41,62 | 87,65 |
English - Bosnian | 113,91 | Flores | 35,57 | 90,61 |
Bulgarian - English | 184,00 | Flores | 44,97 | 88,28 |
English - Bulgarian | 184,00 | Flores | 45,90 | 91,22 |
Catalan - English | 113,91 | Flores | 47,54 | 88,55 |
English - Catalan | 113,91 | Flores | 45,62 | 87,67 |
Cebuano - English | 113,91 | Flores | 41,17 | 79,24 |
English - Cebuano | 113,91 | Flores | 35,46 | 71,80 |
Chichewa - English | 113,91 | Flores | 22,86 | 69,22 |
English - Chichewa | 113,91 | Flores | 17,44 | 64,71 |
Chinese (Simplified) - English | 184,11 | Flores | 29,91 | 86,27 |
English - Chinese (Simplified) | 184,11 | Flores | 42,29 | 87,70 |
Chinese (Traditional) - English | 190,63 | Flores | 30,28 | 86,51 |
English - Chinese (Traditional) | 190,63 | Flores | 36,06 | 88,67 |
Corsican - English | 113,91 | Lingvanex | 59,90 | 83,86 |
English - Corsican | 113,91 | Lingvanex | 54,21 | 76,34 |
Croatian - English | 184,00 | Flores 101 | 39,80 | 88,08 |
English - Croatian | 184,00 | Flores | 34,95 | 91,09 |
Czech - English | 184,00 | Flores | 41,83 | 88,33 |
English - Czech | 184,00 | Flores | 36,53 | 90,98 |
Danish - English | 184,00 | Flores | 51,21 | 90,25 |
English - Danish | 184,02 | Flores | 49,88 | 91,27 |
Dutch - English | 184,00 | Flores | 34,04 | 87,20 |
English - Dutch | 184,00 | Flores | 29,71 | 88,10 |
Esperanto - English | 113,91 | Flores | 41,60 | 87,49 |
English - Esperanto | 113,91 | Flores | 32,02 | 88,92 |
Estonian - English | 113,91 | Flores | 41,07 | 89,33 |
English - Estonian | 113,91 | Flores | 31,48 | 91,64 |
Filipino - English | 113,91 | Flores | 47,41 | 86,85 |
English - Filipino | 113,91 | Flores | 37,75 | 84,61 |
Finnish - English | 184,00 | Flores | 36,87 | 89,76 |
English - Finnish | 184,00 | Flores | 27,06 | 90,96 |
French - English | 190,65 | Flores | 48,82 | 89,46 |
English - French | 190,65 | Flores | 53,25 | 88,40 |
Frisian - English | 113,91 | Lingvanex | 65,06 | 84,99 |
English - Frisian | 113,91 | Lingvanex | 54,82 | 81,09 |
Gaelic-English | 113,91 | Flores | 29,60 | 76,02 |
English-Gaelic | 113,91 | Flores | 24,72 | 80,92 |
Galician - English | 113,91 | Flores | 40,58 | 87,52 |
English - Galician | 113,91 | Flores | 37,71 | 87,03 |
Georgian - Russian | 190,63 | Flores | 18,60 | 85,80 |
Russian - Georgian | 190,63 | Flores | 13,85 | 85,77 |
Georgian - English | 113,91 | Flores | 22,09 | 89,32 |
English - Georgian | 113,91 | Flores | 16,56 | 87,13 |
German - English | 190,65 | Flores | 46,76 | 89,34 |
English - German | 184,02 | Flores | 43,09 | 88,15 |
Greek - English | 184,00 | Flores | 36,19 | 87,38 |
English - Greek | 184,00 | Flores | 29,54 | 89,02 |
Gujarati-English | 113,91 | Flores | 142,09 | 89,68 |
English - Gujarati | 113,91 | Flores | 28,00 | 88,61 |
Haitian Creole - English | 113,91 | Flores | 30,08 | 72,24 |
English - Haitian Creole | 113,91 | Flores | 28,27 | 67,41 |
Hausa - English | 113,91 | Flores | 33,64 | 78,80 |
English - Hausa | 113,91 | Flores | 28,70 | 80,76 |
Hawaiian - English | 113,91 | Lingvanex | 45,12 | 71,92 |
English - Hawaiian | 113,91 | Lingvanex | 64,27 | 78,56 |
Hebrew - English | 184,11 | Flores | 46,31 | 88,32 |
English - Hebrew | 184,11 | Flores | 36,00 | 88,53 |
Hindi - English | 113,91 | Flores | 38,18 | 88,29 |
English - Hindi | 113,91 | Flores | 67,62 | 81,99 |
English - Hmong | 113,91 | Lingvanex | 60,99 | 77,35 |
Hmong - English | 184,00 | Lingvanex | 45,87 | 75,22 |
Hungarian - English | 184,00 | Flores | 38,74 | 88,37 |
English - Hungarian | 184,00 | Flores | 31,40 | 89,97 |
Icelandic - English | 113,91 | Flores | 37,27 | 85,92 |
English - Icelandic | 113,91 | Flores | 31,90 | 86,34 |
Igbo - English | 113,91 | Flores | 26,60 | 68,85 |
English - Igbo | 113,91 | Flores | 19,01 | 71,94 |
Indonesian - English | 184,02 | Flores | 46,65 | 89,60 |
English - Indonesian | 184,02 | Flores | 50,77 | 91,90 |
Italian - English | 184,00 | Flores | 34,74 | 87,89 |
English - Italian | 184,00 | Flores | 32,57 | 88,19 |
Irish - English | 113,91 | Flores | 44,15 | 85,29 |
English - Irish | 113,91 | Flores | 38,86 | 81,81 |
Japanese - English | 190,63 | Flores | 31,05 | 88,08 |
English - Japanese | 190,63 | Flores | 39,62 | 91,56 |
Javanese - English | 113,91 | Flores | 30,29 | 76,20 |
English - Javanese | 113,91 | Flores | 28,79 | 86,59 |
Kannada - English | 113,91 | Flores | 36,01 | 87,38 |
English - Kannada | 113,91 | Flores | 65,83 | 86,33 |
Kazakh - English | 113,91 | Flores | 36,25 | 87,61 |
English - Kazakh | 113,91 | Flores | 26,48 | 90,35 |
Kazakh - Russian | 190,63 | Flores | 22,79 | 88,12 |
Russian - Kazakh | 190,63 | Flores | 19,18 | 89,57 |
Khmer - English | 113,91 | Flores | 33,19 | 85,77 |
English - Khmer | 113,91 | Flores | 4,77 | 82,37 |
Kinyarwanda - English | 113,91 | Flores | 32,98 | 73,73 |
English - Kinyarwanda | 113,91 | Flores | 25,84 | 66,67 |
Korean - English | 113,91 | Flores | 32,85 | 88,09 |
English - Korean | 113,91 | Flores | 33,66 | 89,67 |
Kurdish - English | 113,91 | Flores | 28,81 | 79,90 |
English - Kurdish | 113,91 | Flores | 12,85 | 80,84 |
Kyrgyz - English | 113,91 | Flores | 24,29 | 84,79 |
English - Kyrgyz | 113,91 | Flores | 16,65 | 88,14 |
Kyrgyz - Russian | 190,63 | Flores | 16,38 | 86,32 |
Russian - Kyrgyz | 190,63 | Flores | 13,43 | 87,62 |
Lao - English | 113,91 | Flores | 31,45 | 84,11 |
English - Lao | 113,91 | Flores | 59,54 | 83,24 |
Latin - English | 113,91 | Lingvanex | 24,24 | 74,81 |
English - Latin | 113,91 | Lingvanex | 14,83 | 77,64 |
Latvian - English | 113,91 | Flores | 38,95 | 87,67 |
English - Latvian | 113,91 | Flores | 37,88 | 90,49 |
Lithuanian - English | 113,91 | Flores | 34,96 | 86,24 |
English - Lithuanian | 113,91 | Flores | 31,28 | 90,11 |
Luxembourgish - English | 113,91 | Flores | 45,56 | 80,29 |
English - Luxembourgish | 113,91 | Flores | 27,28 | 57,29 |
Macedonian - English | 113,91 | Flores | 44,33 | 87,43 |
English - Macedonian | 113,91 | Flores | 38,19 | 89,28 |
Malagasy - English | 113,91 | Lingvanex | 38,60 | 83,10 |
English - Malagasy | 113,91 | Lingvanex | 41,74 | 85,25 |
Malay - English | 184,11 | Flores | 46,09 | 88,84 |
English - Malay | 184,11 | Flores | 44,63 | 89,77 |
Malayalam - English | 113,91 | Flores | 39,26 | 88,28 |
English - Malayalam | 113,91 | Flores | 23,19 | 88,92 |
Maltese - English | 113,91 | Flores | 53,11 | 81,83 |
English - Maltese | 113,91 | Flores | 47,57 | 79,52 |
Maori - English | 113,91 | Flores | 29,13 | 69,87 |
English - Maori | 113,91 | Flores | 17,44 | 61,51 |
Marathi - English | 113,91 | Flores | 37,94 | 87,49 |
English - Marathi | 113,91 | Flores | 21,42 | 75,73 |
Mongolian- English | 113,91 | Flores | 31,40 | 85,43 |
English - Mongolian | 113,91 | Flores | 20,07 | 89,00 |
Myanmar - English | 113,91 | Flores | 23,13 | 82,43 |
English - Myanmar | 113,91 | Flores | 55,72 | 87,25 |
Nepali - English | 113,91 | Flores | 41,67 | 89,94 |
English - Nepali | 113,91 | Flores | 64,92 | 83,27 |
Norwegian - English | 184,00 | Flores | 44,20 | 88,78 |
English - Norwegian | 184,02 | Flores | 35,85 | 90,25 |
Odia - English | 113,91 | Flores | 32,34 | 86,77 |
English - Odia | 113,91 | Flores | 21,09 | 81,84 |
Persian - English | 184,02 | Flores | 39,04 | 87,71 |
English - Persian | 184,11 | Flores | 27,24 | 87,82 |
Polish - English | 184,00 | Flores | 30,94 | 85,72 |
English - Polish | 184,00 | Flores | 24,24 | 89,21 |
Portuguese (Brazil) - English | 184,02 | Flores | 51,42 | 89,66 |
English - Portuguese (Brazil) | 184,02 | Flores | 51,67 | 89,40 |
Portuguese - English | 184,02 | Flores | 50,93 | 89,27 |
English - Portuguese | 184,02 | Flores | 51,38 | 89,71 |
Punjabi - English | 113,91 | Flores | 38,15 | 87,74 |
English - Punjabi | 113,91 | Flores | 28,91 | 84,28 |
Pushto - English | 113,91 | Flores | 27,05 | 79,68 |
English - Pushto | 113,91 | Flores | 15,04 | 77,94 |
Romanian - English | 184,00 | Flores | 45,22 | 89,30 |
English - Romanian | 179,58 | Flores | 43,68 | 90,29 |
Russian - English | 190,65 | Flores | 37,86 | 86,63 |
English - Russian | 190,65 | Flores | 34,64 | 89,37 |
Russian - Belarusian | 190,63 | Lingvanex | 57,35 | 95,38 |
Samoan - English | 113,91 | Flores | 27,81 | 68,62 |
English - Samoan | 113,91 | Flores | 30,07 | 67,20 |
Serbian - English | 184,00 | Flores | 43,06 | 86,68 |
English - Serbian | 184,00 | Flores | 36,87 | 88,49 |
Sesotho - English | 113,91 | Flores | 32,91 | 72,64 |
English - Sesotho | 113,91 | Flores | 20,36 | 66,18 |
Shona - English | 113,91 | Flores | 23,05 | 70,04 |
English - Shona | 113,91 | Flores | 13,74 | 62,88 |
Sindhi - English | 113,91 | Flores | 31,09 | 79,66 |
English - Sindhi | 113,91 | Flores | 25,90 | 80,37 |
Sinhala - English | 113,91 | Flores | 36,73 | 87,92 |
English - Sinhala | 113,91 | Flores | 21,42 | 87,65 |
Slovak - English | 184,11 | Flores | 42,32 | 88,33 |
English - Slovak | 184,11 | Flores | 40,08 | 91,01 |
Slovenian - English | 113,91 | Flores | 38,25 | 87,76 |
English - Slovenian | 113,91 | Flores | 35,39 | 90,20 |
Somali - English | 113,91 | Flores | 29,89 | 77,25 |
English - Somali | 113,91 | Flores | 14,56 | 80,85 |
Spanish - English | 184,02 | Flores | 31,48 | 86,92 |
English - Spanish | 184,02 | Flores | 30,17 | 86,72 |
Sundanese - English | 113,91 | Flores | 34,02 | 81,42 |
English - Sundanese | 113,91 | Flores | 20,23 | 79,65 |
Swahili - English | 113,91 | Flores | 44,95 | 84,91 |
English - Swahili | 113,91 | Flores | 41,14 | 85,40 |
Swedish - English | 184,00 | Flores | 51,89 | 90,13 |
English - Swedish | 184,00 | Flores | 49,39 | 91,24 |
Tajik - English | 113,91 | Flores | 33,74 | 77,46 |
English - Tajik | 113,91 | Flores | 24,97 | 76,28 |
Tajik - Russian | 190,63 | Flores | 23,15 | 79,69 |
Russian - Tajik | 190,63 | Flores | 19,38 | 75,59 |
Tamil - English | 113,91 | Flores | 35,17 | 86,48 |
English - Tamil | 113,91 | Flores | 23,12 | 89,55 |
Tatar - English | 113,91 | Flores | 30,37 | 79,43 |
English - Tatar | 113,91 | Flores | 19,01 | 69,26 |
Telugu-English | 113,91 | Flores | 42,02 | 87,74 |
English-Telugu | 113,91 | Flores | 28,30 | 87,60 |
Thai - English | 184,02 | Flores | 31,63 | 87,71 |
English - Thai | 113,91 | Flores | 63,68 | 88,57 |
Turkish - English | 184,00 | Flores | 41,85 | 89,63 |
English - Turkish | 184,00 | Flores | 35,33 | 90,82 |
Turkmen - English | 113,91 | Flores | 37,10 | 80,90 |
English - Turkmen | 113,91 | Flores | 22,91 | 67,00 |
Ukrainian - English | 184,00 | Flores | 41,54 | 86,98 |
English - Ukrainian | 184,00 | Flores | 34,30 | 89,88 |
Ukrainian - Russian | 190,63 | Flores | 28,07 | 91,66 |
Russian - Ukrainian | 190,63 | Flores | 25,62 | 92,16 |
Urdu - English | 113,91 | Flores | 31,09 | 84,46 |
English - Urdu | 113,91 | Flores | 24,80 | 81,71 |
Uyghur - English | 113,91 | Flores | 28,17 | 86,26 |
English - Uyghur | 113,91 | Flores | 22,66 | 86,21 |
Uzbek - English | 113,91 | Flores | 37,42 | 87,64 |
English - Uzbek | 113,91 | Flores | 23,04 | 90,32 |
Uzbek - Russian | 190,63 | Flores | 22,11 | 85,40 |
Russian - Uzbek | 190,63 | Flores | 16,86 | 87,37 |
Vietnamese - English | 184,00 | Flores | 38,65 | 87,53 |
English - Vietnamese | 184,11 | Flores | 46,28 | 89,16 |
Welsh - English | 113,91 | Flores | 61,36 | 89,32 |
English - Welsh | 113,91 | Flores | 57,55 | 88,83 |
Xhosa - English | 113,91 | Flores | 36,65 | 78,10 |
English - Xhosa | 113,91 | Flores | 18,90 | 77,05 |
Yiddish - English | 113,91 | Flores | 41,89 | 76,88 |
English - Yiddish | 113,91 | Flores | 10,88 | 67,67 |
Yoruba - English | 113,91 | Flores | 17,54 | 61,56 |
English - Yoruba | 113,91 | Flores | 3,57 | 55,39 |
Zulu - English | 113,91 | Flores | 36,83 | 78,00 |
English - Zulu | 113,91 | Flores | 21,06 | 78,12 |
Minta maklumat lanjut
Selesai
Permintaan anda telah berjaya dihantar.