Lingvanex 翻譯品質報告

February 27, 2024

本報告的目標是根據兩個最受歡迎的機器翻譯評估指標來展示 Lingvanex 語言模型的翻譯品質。

Flores 是一個開源且公開的資料測試集，由 Facebook Research 發布，擁有最大的語言對覆蓋範圍。

品質指標描述

BLEU

BLEU 是一種基於 n 元語法的自動度量。它測量機器翻譯輸出的 n-gram 與參考相比的精度，並透過簡潔性懲罰進行加權，以懲罰過短的翻譯。我們使用 BLEU 的一種特殊實現，稱為 sacreBLEU。它輸出語料庫分數，而不是片段分數。

參考

Papineni, Kishore, S. Roukos, T. Ward and Wei-Jing Zhu. “Bleu: a Method for Automatic Evaluation of Machine Translation.” ACL (2002).
Post, Matt. “A Call for Clarity in Reporting BLEU Scores.” WMT (2018).

COMET

COMET（翻譯評估的跨語言最佳化指標）是一種自動評估機器翻譯的指標，它使用標記或句子嵌入來計算機器翻譯輸出和參考翻譯之間的相似度。與其他指標不同，COMET 經過訓練，以後期編輯工作、直接評估或翻譯錯誤分析的形式預測不同類型的人類判斷。

參考

COMET - https://machinetranslate.org/comet
COMET: High-quality Machine Translation Evaluation - https://unbabel.github.io/COMET/html/index.html#comet-high-quality-machine-translation-evaluation

語言對

注意：硬碟上模型的大小越小意味著 GPU 記憶體消耗越少，從而降低部署成本。模型尺寸越小，翻譯時間的表現越好。 GPU 記憶體的大致使用量計算為硬碟型號大小 x 1.2

語言對	型號的大小，MB	測試數據	BLEU 公制	COMET 公制
Afrikaans - English	113,91	Flores 101	52,33	86,43
English - Afrikaans	113,91	Flores 101	42,52	86,92
Albanian - English	113,91	Lingvanex	55,31	86,68
English - Albanian	113,91	Lingvanex	55,84	87,88
Amharic - English	113,91	Flores 101	30,42	83,78
English - Amharic	113,91	Flores 101	14,13	87,4
Arabic - English	190,63	Flores 101	44,83	87,65
English - Arabic	190,63	Flores 101	33,21	87,81
Armenian - English	113,91	Flores 101	37,47	86,93
English - Armenian	113,91	Flores 101	22,09	89,32
Armenian - Russian	190,63	Flores 200	20,97	85,06
Russian - Armenian	190,63	Flores 200	15,71	87,00
Azerbaijani - English	113,91	Flores 101	23,76	85,88
English - Azerbaijani	113,91	Flores 101	17,79	84,93
Azerbaijani - Russian	190,63	Flores 200	16,95	85,23
Russian - Azerbaijani	190,63	Flores 200	13,10	83,64
Belarusian - English	113,91	Flores 101	19,77	80,89
English - Belarusian	113,91	Flores 101	15,54	85,16
Bengali - English	113,91	Flores 101	33,93	87,86
English - Bengali	113,91	Flores 101	20,98	86,42
Bosnian - English	113,91	Flores 101	41,62	87,65
English - Bosnian	113,91	Flores 101	35,57	90,61
Bulgarian - English	184,00	Flores 101	42,16	87,45
English - Bulgarian	184,00	Flores 101	44,1	89,99
English - Basque	113,91	Flores 101	26,52	83,88
Catalan - English	113,91	Lingvanex	46,31	87,97
English - Catalan	113,91	Flores 101	45,62	87,67
Cebuano - English	113,91	Flores 101	41.17	79,24
English - Cebuano	113,91	Flores 101	35,46	71,8
Chichewa - English	113,91	Flores 101	22,86	69,22
English - Chichewa	113,91	Flores 101	17,44	64,71
Chinese (Simplified) - English	190,63	Flores 101	27,96	84,88
English - Chinese (Simplified)	190,63	Flores 101	38,47	85,47
Chinese (Traditional) - English	190,63	Flores 101	25,11	83,55
English - Chinese (Traditional)	190,63	Lingvanex	30,25	85,67
Corsican - English	113,91	Lingvanex	56,71	83,19
English - Corsican	113,91	Flores 101	54,21	76,34
Croatian - English	184,00	Flores 101	37,1	86,84
English - Croatian	184,00	Flores 101	31,03	88,9
Czech - English	184,00	Flores 101	41,83	88,33
English - Czech	184,00	Flores 101	35,96	90,77
Danish - English	184,00	Flores 101	49,75	89,88
English - Danish	184,00	Flores 101	46,29	89,46
Dutch - English	184,00	Flores 101	34,04	87,2
English - Dutch	184,00	Flores 101	29,71	88,1
Estonian - English	113,91	Flores 101	39,11	88,79
English - Estonian	113,91	Flores 101	30,74	91,03
Finnish - English	184,00	Flores 101	36,87	89,76
English - Finnish	184,00	Flores 101	27,06	90,96
French - English	195,05	Flores 101	47,82	89,17
English - French	195,05	Flores 101	53,25	88,4
Frisian - English	113,91	Lingvanex	63,19	83,68
English - Frisian	113,91	Lingvanex	54,23	80,62
Georgian - Russian	190,63	Flores 200	18,6	85,8
Russian - Georgian	190,63	Flores 200	13,85	85,77
English - Georgian	113,91	Flores 101	16,56	87,13
German - English	190,65	Flores 101	46,54	89,07
English - German	184,02	Flores 101	43,09	88,15
Greek - English	184,02	Flores 101	36,52	87,22
English - Greek	184,02	Flores 101	28,69	88,44
English - Gujarati	113,91	Flores 101	28,00	88,61
Gaelic-English	113,91	Flores 101	25,24	71,42
English-Gaelic	113,91	Flores 101	24,72	80,92
Galician - English	113,91	Flores 101	40,58	87,52
English - Galician	113,91	Flores 101	41,08	87,03
English - Hausa	113,91	Flores 101	28,93	80,76
Hawaiian - English	113,91	Lingvanex	45,12	71,92
English - Hawaiian	113,91	Lingvanex	64,27	78,56
Hebrew - English	113,91	Flores 101	44,92	87,78
English - Hebrew	113,91	Flores 101	35,36	87,8
Hindi - English	113,91	Lingvanex	38,78	87,66
English - Hindi	113,91	Flores 101	38,86	78,84
Hungarian - English	184,00	Flores 101	35,15	87,27
English - Hungarian	184,00	Flores 101	28,48	87,44
Igbo - English	113,91	Flores 101	21,85	64,78
English - Igbo	113,91	Flores 101	19,01	71,94
Indonesian - English	184,00	Flores 101	45,8	89,14
English - Indonesian	184,37	Flores 101	50,51	92,04
Italian - English	184,00	Flores 101	34,74	87,89
English - Italian	184,00	Flores 101	32,57	32,57
Irish - English	113,91	Flores 101	44,15	85,29
English - Irish	113,91	Flores 101	38,86	81,81
Japanese - English	190,63	Flores 101	29,16	86,71
English - Japanese	190,63	Flores 101	35,86	89,8
Khmer - English	113,91	Flores 101	26,12	81,08
English - Khmer	113,91	Flores 101	4,77	82,37
Kinyarwanda - English	113,91	Flores 101	28,36	70,85
English - Kinyarwanda	113,91	Flores 101	25,84	66,67
Korean - English	113,91	Flores 101	29,68	86,22
English - Korean	113,91	Flores 101	32,42	88,94
Kurdish - English	113,91	Flores 101	22,94	74,97
English - Kurdish	113,91	Flores 101	12,85	80,84
Kyrgyz - English	113,91	Flores 101	24,29	84,79
English - Kyrgyz	113,91	Flores 101	16,65	88,14
English - Kannada	113,91	Flores 101	25,33	85,00
Kazakh - Russian	190,63	Flores 200	22,6	87,36
Russian - Kazakh	190,63	Flores 200	19,05	89,42
Kyrgyz - Russian	190,63	Flores 200	16,38	86,32
Russian - Kyrgyz	190,63	Flores 200	13,43	87,62
Lao - English	113,91	Flores 101	31,45	84,11
English - Lao	113,91	Flores 101	59,54	83,24
Latin - English	113,91	Lingvanex	24,24	74,81
English - Latin	113,91	Lingvanex	14,83	77,64
Latvian - English	113,91	Flores 101	36,74	86,88
English - Latvian	113,91	Flores 101	36,05	89,45
Lithuanian - English	113,91	Flores 101	34,16	85,51
English - Lithuanian	113,91	Flores 101	30,84	89,61
Luxembourgish - English	113,91	Flores 101	38,69	74,82
English - Luxembourgish	113,91	Flores 101	27,28	57,29
Malay - English	113,91	Flores 101	42,73	87,6
English - Malay	113,91	Flores 101	44,07	89,38
English - Malayalam	113,91	Flores 101	23,38	88,92
Maltese - English	113,91	Flores 200	53,11	81,83
English - Maltese	113,91	Flores 200	46,23	74,05
Maori - English	113,91	Flores 101	22,93	64,7
English - Maori	113,91	Flores 101	17,44	61,51
Marathi - English	113,91	Flores 101	37,94	87,49
English - Marathi	113,91	Flores 101	21,42	75,73
Mongolian- English	113,91	Lingvanex	31,4	85,43
English - Mongolian	113,91	Flores 101	20,07	89,00
Myanmar - English	113,91	Flores 101	23,13	82,43
English - Myanmar	113,91	Flores 101	55,72	87,25
Malagasy - English	113,91	Lingvanex	38,6	83,1
Norwegian - English	184,00	Flores 101	43,74	88,27
English - Norwegian	184,02	Flores 101	35,03	89,4
Odia - English	113,91	Flores 101	32,34	86,77
English - Odia	113,91	Flores 101	21,09	81,84
Persian - English	190,63	Flores 101	38,6	87,17
English - Persian	190,63	Flores 101	26,76	87,7
Polish - English	184,00	Flores 101	30,94	85,72
English - Polish	184,00	Flores 101	24,24	89,21
Pushto - English	113,91	Flores 101	27,05	79,68
English - Pushto	113,91	Flores 101	15,04	77,94
Portuguese (Brazil) - English	184,00	Lingvanex	65,11	92,95
English - Portuguese (Brazil)	184,02	Lingvanex	62,75	93,84
Portuguese - English	184,02	Flores 101	51,38	89,71
English - Portuguese	184,02	Flores 101	50,93	89,27
Romanian - English	184,00	Flores 101	43,02	88,68
English - Romanian	179,58	Flores 101	41,01	88,84
Russian - English	190,65	Flores 101	37,86	86,63
English - Russian	190,65	Flores 101	34,64	89,37
Russian - Belarusian	190,63	Lingvanex	57,35	25,78
Serbian - English	184,00	Flores 101	43,06	86,68
English - Serbian	184,00	Flores 101	36,87	88,49
Sesotho - English	113,91	Flores 101	32,91	72,64
English - Sesotho	113,91	Flores 101	20,36	66,18
Sinhala - English	113,91	Flores 101	36,73	87,92
English - Sinhala	113,91	Flores 101	21,42	87,65
Slovak - English	113,91	Flores	38,18	86,85
English - Slovak	113,91	Flores	35,75	88,34
Slovenian - English	113,91	Flores 101	34,13	86,55
English - Slovenian	113,91	Flores 101	30,66	87,36
Spanish - English	184,02	Flores 101	29,91	86,35
English - Spanish	184,00	Flores 101	29,43	86,31
Sundanese - English	113,91	Lingvanex	17,17	74,84
English - Sundanese	113,91	Lingvanex	11,58	77,33
Swedish - English	184,00	Flores 101	51,89	90,13
English - Swedish	184,00	Flores 101	49,39	91,24
Tajik - English	113,91	Flores 101	32,61	76,9
English - Tajik	113,91	Flores 101	24,97	76,28
Tajik - Russian	190,63	Flores 200	23,15	79,69
Russian - Tajik	190,63	Flores 200	19,38	75,59
Tamil - English	113,91	Flores 101	35,17	86,48
English - Tamil	113,91	Flores 101	23,12	89,55
Tatar - English	113,91	Flores 101	29,91	79,25
English - Tatar	113,91	Lingvanex	18,69	69,09
Telugu-English	113,91	Flores 101	42,02	87,74
English-Telugu	113,91	Flores 101	28,3	87,6
Thai - English	184,02	Flores 101	31,63	87,71
English - Thai	113,91	Flores 101	63,68	88,57
Turkish - English	184,00	Flores 101	41,85	89,63
English - Turkish	184,00	Flores 101	32,48	88,68
Turkmen - English	113,91	Flores 101	32,68	78,45
English - Turkmen	113,91	Flores 101	22,91	67,00
Ukrainian - English	190,63	Flores 101	41,15	86,92
English - Ukrainian	184,00	Flores 101	30,2	86,99
Ukrainian - Russian	190,63	Flores 200	28,07	91,66
Russian - Ukrainian	190,63	Flores 200	25,62	92,16
Urdu - English	113,91	Flores 101	31,09	84,46
English - Urdu	113,91	Flores 101	24,8	81,71
Uyghur - English	113,91	Flores 101	24,97	83,96
English - Uyghur	113,91	Flores 101	22,66	86,21
Uzbek - Russian	190,63	Flores 200	22,11	85,4
Russian - Uzbek	190,63	Flores 200	16,86	87,37
Vietnamese - English	184,00	Flores 101	38,13	87,3
English - Vietnamese	184,00	Flores 101	43,84	86,95
Xhosa - English	113,91	Flores 101	34,34	76,53
English - Xhosa	113,91	Flores 101	18,9	77,05
Yiddish - English	113,91	Flores 101	41,89	76,88
English - Yiddish	113,91	Flores 101	10,88	67,67
Yoruba - English	113,91	Flores 101	14,06	56,35
English - Yoruba	113,91	Lingvanex	3,57	55,39
Zulu - English	113,91	Flores 101	36,83	78,00
English - Zulu	113,91	Flores 101	21,06	78,12

翻譯者

Lingvanex 翻譯品質報告

品質指標描述

BLEU

參考

COMET

參考

語言對

索取更多信息

翻譯者

Lingvanex 翻譯品質報告

品質指標描述

BLEU

參考

COMET

參考

語言對

索取更多信息

完全的