Speech AI Preferences Test - full_finetune_3B_seed43_temperature=None_top_k=None vs eleven_turbo_v2_no_underscores_temperature=None_top_k=None

Preference test between full_finetune_3B_seed43_temperature=None_top_k=None and eleven_turbo_v2_no_underscores_temperature=None_top_k=None

Overall

pokegabe_eleven_turbo_v2_no_underscores_temperature=None_top_k=None (Model B)

0.09

pokegabe_full_finetune_3B_seed43_temperature=None_top_k=None (Model A)

By Tag

pokegabe_eleven_turbo_v2_no_underscores_temperature=None_top_k=None (Model B)

pokegabe_full_finetune_3B_seed43_temperature=None_top_k=None (Model A)

Answers

2. pokegabe_full_finetune_3B_seed43_temperature=None_top_k=None (Model A) is strongly preferred

-1

-2. pokegabe_eleven_turbo_v2_no_underscores_temperature=None_top_k=None (Model B) is strongly preferred

58 (48.33%)

3 (2.50%)

4 (3.33%)

52 (43.33%)

Tags

All tags

pokegabe_eleven_turbo_v2_no_underscores_temperature=None_top_k=None (Model B)

pokegabe_full_finetune_3B_seed43_temperature=None_top_k=None (Model A)

58 (48.33%)

Deep analysis

2. pokegabe_full_finetune_3B_seed43_temperature=None_top_k=None (Model A) is strongly preferred

-1

-2. pokegabe_eleven_turbo_v2_no_underscores_temperature=None_top_k=None (Model B) is strongly preferred

		Tags	Statistics
00:00 --:-- full_finetune_3B_seed43_temperature=None_top_k=None/media/audio/e06f89cb5f1c5e23859f.wav 00:00 --:-- eleven_turbo_v2_no_underscores_temperature=None_top_k=None/media/audio/b185313ab3317c9c01a1.wav	pokegabe_full_finetune_3B_seed43_temperature=None_top_k=None pokegabe_eleven_turbo_v2_no_underscores_temperature=None_top_k=None	Script: -6006125463382209857 pokegabe_full_finetune_3B_seed43_temperature=None_top_k=None default WER: 0.0 WER_O: 0.0 SIM_O: 0.8282524943351746 SIM_R: 0.8282524943351746 MOS: 4.286427021026611 Script: -6006125463382209857 pokegabe_eleven_turbo_v2_no_underscores_temperature=None_top_k=None default WER: 0.0 WER_O: 0.0 SIM_O: 0.8769372701644897 SIM_R: 0.8769372701644897 MOS: 4.242783069610596	2.00 0.00 0.00
00:00 --:-- full_finetune_3B_seed43_temperature=None_top_k=None/media/audio/970673e8f677aff0e2b3.wav 00:00 --:-- eleven_turbo_v2_no_underscores_temperature=None_top_k=None/media/audio/ac828c22d15599135c55.wav	pokegabe_full_finetune_3B_seed43_temperature=None_top_k=None pokegabe_eleven_turbo_v2_no_underscores_temperature=None_top_k=None	Script: 6408732718787899357 pokegabe_full_finetune_3B_seed43_temperature=None_top_k=None default WER: 0.0 WER_O: 6.666666666666667 SIM_O: 0.8245769739151001 SIM_R: 0.8245769739151001 MOS: 3.8923592567443848 Script: 6408732718787899357 pokegabe_eleven_turbo_v2_no_underscores_temperature=None_top_k=None default WER: 6.666666666666667 WER_O: 13.333333333333334 SIM_O: 0.9021720886230469 SIM_R: 0.9021720886230469 MOS: 4.154809474945068	0.67 5.74 1.33
00:00 --:-- full_finetune_3B_seed43_temperature=None_top_k=None/media/audio/c557b9b5955a54458cc9.wav 00:00 --:-- eleven_turbo_v2_no_underscores_temperature=None_top_k=None/media/audio/1312fc9790a1a7f01499.wav	pokegabe_full_finetune_3B_seed43_temperature=None_top_k=None pokegabe_eleven_turbo_v2_no_underscores_temperature=None_top_k=None	Script: 6617085272261610511 pokegabe_full_finetune_3B_seed43_temperature=None_top_k=None default WER: 0.0 WER_O: 0.0 SIM_O: 0.8088076114654541 SIM_R: 0.8088076114654541 MOS: 3.899986505508423 Script: 6617085272261610511 pokegabe_eleven_turbo_v2_no_underscores_temperature=None_top_k=None default WER: 0.0 WER_O: 0.0 SIM_O: 0.4985923767089844 SIM_R: 0.4985923767089844 MOS: 4.173958778381348	2.00 0.00 0.00
00:00 --:-- full_finetune_3B_seed43_temperature=None_top_k=None/media/audio/e371d7bfb9f4866bb6c3.wav 00:00 --:-- eleven_turbo_v2_no_underscores_temperature=None_top_k=None/media/audio/4c7380bd64684364627b.wav	pokegabe_full_finetune_3B_seed43_temperature=None_top_k=None pokegabe_eleven_turbo_v2_no_underscores_temperature=None_top_k=None	Script: 5430061021594831344 pokegabe_full_finetune_3B_seed43_temperature=None_top_k=None default WER: 0.0 WER_O: 3.571428571428571 SIM_O: 0.8516353368759155 SIM_R: 0.8516353368759155 MOS: 4.147836685180664 Script: 5430061021594831344 pokegabe_eleven_turbo_v2_no_underscores_temperature=None_top_k=None default WER: 0.0 WER_O: 3.571428571428571 SIM_O: 0.8232199549674988 SIM_R: 0.8232199549674988 MOS: 4.279565334320068	0.67 5.74 1.33
00:00 --:-- full_finetune_3B_seed43_temperature=None_top_k=None/media/audio/2450bc16ae75863cfff3.wav 00:00 --:-- eleven_turbo_v2_no_underscores_temperature=None_top_k=None/media/audio/698d8700ccb7e266ba10.wav	pokegabe_full_finetune_3B_seed43_temperature=None_top_k=None pokegabe_eleven_turbo_v2_no_underscores_temperature=None_top_k=None	Script: 6550140456215039481 pokegabe_full_finetune_3B_seed43_temperature=None_top_k=None default WER: 0.0 WER_O: 33.33333333333333 SIM_O: 0.44665098190307617 SIM_R: 0.44665098190307617 MOS: 3.6182737350463867 Script: 6550140456215039481 pokegabe_eleven_turbo_v2_no_underscores_temperature=None_top_k=None default WER: 0.0 WER_O: 33.33333333333333 SIM_O: 0.5343602895736694 SIM_R: 0.5343602895736694 MOS: 4.037581443786621	-0.67 5.74 1.33
00:00 --:-- full_finetune_3B_seed43_temperature=None_top_k=None/media/audio/bc87d50e7ba4687a417b.wav 00:00 --:-- eleven_turbo_v2_no_underscores_temperature=None_top_k=None/media/audio/0c51e9b6d648fa029695.wav	pokegabe_full_finetune_3B_seed43_temperature=None_top_k=None pokegabe_eleven_turbo_v2_no_underscores_temperature=None_top_k=None	Script: 1148811978304038921 pokegabe_full_finetune_3B_seed43_temperature=None_top_k=None default WER: 0.0 WER_O: 0.0 SIM_O: 0.881961464881897 SIM_R: 0.881961464881897 MOS: 4.096604824066162 Script: 1148811978304038921 pokegabe_eleven_turbo_v2_no_underscores_temperature=None_top_k=None default WER: 0.0 WER_O: 0.0 SIM_O: 0.8439435362815857 SIM_R: 0.8439435362815857 MOS: 4.40484094619751	0.67 5.74 1.33
00:00 --:-- full_finetune_3B_seed43_temperature=None_top_k=None/media/audio/3da7f5c4488c92c62769.wav 00:00 --:-- eleven_turbo_v2_no_underscores_temperature=None_top_k=None/media/audio/0a7dcdabc841b75b5126.wav	pokegabe_full_finetune_3B_seed43_temperature=None_top_k=None pokegabe_eleven_turbo_v2_no_underscores_temperature=None_top_k=None	Script: 4253468055971498463 pokegabe_full_finetune_3B_seed43_temperature=None_top_k=None default WER: 0.0 WER_O: 0.0 SIM_O: 0.904725193977356 SIM_R: 0.904725193977356 MOS: 4.081268310546875 Script: 4253468055971498463 pokegabe_eleven_turbo_v2_no_underscores_temperature=None_top_k=None default WER: 0.0 WER_O: 0.0 SIM_O: 0.7346110343933105 SIM_R: 0.7346110343933105 MOS: 4.282021522521973	0.67 5.74 1.33
00:00 --:-- full_finetune_3B_seed43_temperature=None_top_k=None/media/audio/5dae249f36b72d8ed5f5.wav 00:00 --:-- eleven_turbo_v2_no_underscores_temperature=None_top_k=None/media/audio/03b22bd6282cb7e109d9.wav	pokegabe_full_finetune_3B_seed43_temperature=None_top_k=None pokegabe_eleven_turbo_v2_no_underscores_temperature=None_top_k=None	Script: -1480359582308764161 pokegabe_full_finetune_3B_seed43_temperature=None_top_k=None default WER: 0.0 WER_O: 7.6923076923076925 SIM_O: 0.836482048034668 SIM_R: 0.836482048034668 MOS: 4.128872871398926 Script: -1480359582308764161 pokegabe_eleven_turbo_v2_no_underscores_temperature=None_top_k=None default WER: 0.0 WER_O: 7.6923076923076925 SIM_O: 0.7148769497871399 SIM_R: 0.7148769497871399 MOS: 4.325758934020996	0.67 5.74 1.33

Overall

pokegabe_eleven_turbo_v2_no_underscores_temperature=None_top_k=None (Model B)

0.09

pokegabe_full_finetune_3B_seed43_temperature=None_top_k=None (Model A)

By Tag

pokegabe_eleven_turbo_v2_no_underscores_temperature=None_top_k=None (Model B)

pokegabe_full_finetune_3B_seed43_temperature=None_top_k=None (Model A)

Answers

2. pokegabe_full_finetune_3B_seed43_temperature=None_top_k=None (Model A) is strongly preferred

-1

-2. pokegabe_eleven_turbo_v2_no_underscores_temperature=None_top_k=None (Model B) is strongly preferred

58 (48.33%)

3 (2.50%)

4 (3.33%)

52 (43.33%)

Tags

All tags

pokegabe_eleven_turbo_v2_no_underscores_temperature=None_top_k=None (Model B)

pokegabe_full_finetune_3B_seed43_temperature=None_top_k=None (Model A)

58 (48.33%)

Deep analysis

2. pokegabe_full_finetune_3B_seed43_temperature=None_top_k=None (Model A) is strongly preferred

-1

-2. pokegabe_eleven_turbo_v2_no_underscores_temperature=None_top_k=None (Model B) is strongly preferred

		Tags	Statistics
00:00 --:-- full_finetune_3B_seed43_temperature=None_top_k=None/media/audio/e06f89cb5f1c5e23859f.wav 00:00 --:-- eleven_turbo_v2_no_underscores_temperature=None_top_k=None/media/audio/b185313ab3317c9c01a1.wav	pokegabe_full_finetune_3B_seed43_temperature=None_top_k=None pokegabe_eleven_turbo_v2_no_underscores_temperature=None_top_k=None	Script: -6006125463382209857 pokegabe_full_finetune_3B_seed43_temperature=None_top_k=None default WER: 0.0 WER_O: 0.0 SIM_O: 0.8282524943351746 SIM_R: 0.8282524943351746 MOS: 4.286427021026611 Script: -6006125463382209857 pokegabe_eleven_turbo_v2_no_underscores_temperature=None_top_k=None default WER: 0.0 WER_O: 0.0 SIM_O: 0.8769372701644897 SIM_R: 0.8769372701644897 MOS: 4.242783069610596	2.00 0.00 0.00
00:00 --:-- full_finetune_3B_seed43_temperature=None_top_k=None/media/audio/970673e8f677aff0e2b3.wav 00:00 --:-- eleven_turbo_v2_no_underscores_temperature=None_top_k=None/media/audio/ac828c22d15599135c55.wav	pokegabe_full_finetune_3B_seed43_temperature=None_top_k=None pokegabe_eleven_turbo_v2_no_underscores_temperature=None_top_k=None	Script: 6408732718787899357 pokegabe_full_finetune_3B_seed43_temperature=None_top_k=None default WER: 0.0 WER_O: 6.666666666666667 SIM_O: 0.8245769739151001 SIM_R: 0.8245769739151001 MOS: 3.8923592567443848 Script: 6408732718787899357 pokegabe_eleven_turbo_v2_no_underscores_temperature=None_top_k=None default WER: 6.666666666666667 WER_O: 13.333333333333334 SIM_O: 0.9021720886230469 SIM_R: 0.9021720886230469 MOS: 4.154809474945068	0.67 5.74 1.33
00:00 --:-- full_finetune_3B_seed43_temperature=None_top_k=None/media/audio/c557b9b5955a54458cc9.wav 00:00 --:-- eleven_turbo_v2_no_underscores_temperature=None_top_k=None/media/audio/1312fc9790a1a7f01499.wav	pokegabe_full_finetune_3B_seed43_temperature=None_top_k=None pokegabe_eleven_turbo_v2_no_underscores_temperature=None_top_k=None	Script: 6617085272261610511 pokegabe_full_finetune_3B_seed43_temperature=None_top_k=None default WER: 0.0 WER_O: 0.0 SIM_O: 0.8088076114654541 SIM_R: 0.8088076114654541 MOS: 3.899986505508423 Script: 6617085272261610511 pokegabe_eleven_turbo_v2_no_underscores_temperature=None_top_k=None default WER: 0.0 WER_O: 0.0 SIM_O: 0.4985923767089844 SIM_R: 0.4985923767089844 MOS: 4.173958778381348	2.00 0.00 0.00
00:00 --:-- full_finetune_3B_seed43_temperature=None_top_k=None/media/audio/e371d7bfb9f4866bb6c3.wav 00:00 --:-- eleven_turbo_v2_no_underscores_temperature=None_top_k=None/media/audio/4c7380bd64684364627b.wav	pokegabe_full_finetune_3B_seed43_temperature=None_top_k=None pokegabe_eleven_turbo_v2_no_underscores_temperature=None_top_k=None	Script: 5430061021594831344 pokegabe_full_finetune_3B_seed43_temperature=None_top_k=None default WER: 0.0 WER_O: 3.571428571428571 SIM_O: 0.8516353368759155 SIM_R: 0.8516353368759155 MOS: 4.147836685180664 Script: 5430061021594831344 pokegabe_eleven_turbo_v2_no_underscores_temperature=None_top_k=None default WER: 0.0 WER_O: 3.571428571428571 SIM_O: 0.8232199549674988 SIM_R: 0.8232199549674988 MOS: 4.279565334320068	0.67 5.74 1.33
00:00 --:-- full_finetune_3B_seed43_temperature=None_top_k=None/media/audio/2450bc16ae75863cfff3.wav 00:00 --:-- eleven_turbo_v2_no_underscores_temperature=None_top_k=None/media/audio/698d8700ccb7e266ba10.wav	pokegabe_full_finetune_3B_seed43_temperature=None_top_k=None pokegabe_eleven_turbo_v2_no_underscores_temperature=None_top_k=None	Script: 6550140456215039481 pokegabe_full_finetune_3B_seed43_temperature=None_top_k=None default WER: 0.0 WER_O: 33.33333333333333 SIM_O: 0.44665098190307617 SIM_R: 0.44665098190307617 MOS: 3.6182737350463867 Script: 6550140456215039481 pokegabe_eleven_turbo_v2_no_underscores_temperature=None_top_k=None default WER: 0.0 WER_O: 33.33333333333333 SIM_O: 0.5343602895736694 SIM_R: 0.5343602895736694 MOS: 4.037581443786621	-0.67 5.74 1.33
00:00 --:-- full_finetune_3B_seed43_temperature=None_top_k=None/media/audio/bc87d50e7ba4687a417b.wav 00:00 --:-- eleven_turbo_v2_no_underscores_temperature=None_top_k=None/media/audio/0c51e9b6d648fa029695.wav	pokegabe_full_finetune_3B_seed43_temperature=None_top_k=None pokegabe_eleven_turbo_v2_no_underscores_temperature=None_top_k=None	Script: 1148811978304038921 pokegabe_full_finetune_3B_seed43_temperature=None_top_k=None default WER: 0.0 WER_O: 0.0 SIM_O: 0.881961464881897 SIM_R: 0.881961464881897 MOS: 4.096604824066162 Script: 1148811978304038921 pokegabe_eleven_turbo_v2_no_underscores_temperature=None_top_k=None default WER: 0.0 WER_O: 0.0 SIM_O: 0.8439435362815857 SIM_R: 0.8439435362815857 MOS: 4.40484094619751	0.67 5.74 1.33
00:00 --:-- full_finetune_3B_seed43_temperature=None_top_k=None/media/audio/3da7f5c4488c92c62769.wav 00:00 --:-- eleven_turbo_v2_no_underscores_temperature=None_top_k=None/media/audio/0a7dcdabc841b75b5126.wav	pokegabe_full_finetune_3B_seed43_temperature=None_top_k=None pokegabe_eleven_turbo_v2_no_underscores_temperature=None_top_k=None	Script: 4253468055971498463 pokegabe_full_finetune_3B_seed43_temperature=None_top_k=None default WER: 0.0 WER_O: 0.0 SIM_O: 0.904725193977356 SIM_R: 0.904725193977356 MOS: 4.081268310546875 Script: 4253468055971498463 pokegabe_eleven_turbo_v2_no_underscores_temperature=None_top_k=None default WER: 0.0 WER_O: 0.0 SIM_O: 0.7346110343933105 SIM_R: 0.7346110343933105 MOS: 4.282021522521973	0.67 5.74 1.33
00:00 --:-- full_finetune_3B_seed43_temperature=None_top_k=None/media/audio/5dae249f36b72d8ed5f5.wav 00:00 --:-- eleven_turbo_v2_no_underscores_temperature=None_top_k=None/media/audio/03b22bd6282cb7e109d9.wav	pokegabe_full_finetune_3B_seed43_temperature=None_top_k=None pokegabe_eleven_turbo_v2_no_underscores_temperature=None_top_k=None	Script: -1480359582308764161 pokegabe_full_finetune_3B_seed43_temperature=None_top_k=None default WER: 0.0 WER_O: 7.6923076923076925 SIM_O: 0.836482048034668 SIM_R: 0.836482048034668 MOS: 4.128872871398926 Script: -1480359582308764161 pokegabe_eleven_turbo_v2_no_underscores_temperature=None_top_k=None default WER: 0.0 WER_O: 7.6923076923076925 SIM_O: 0.7148769497871399 SIM_R: 0.7148769497871399 MOS: 4.325758934020996	0.67 5.74 1.33