CALLAII® model card · Callmenta
CALLAII® by CALLMENTA

CALLAII ASR v4 technical sheet

Speech recognition · A speech recognition model that separates call recordings into speaker channels and converts them to text. Analysis, coaching, and quality scoring operate on this transcription.

CALLAII ASR v4 Live Released September 2026

Splits call recordings into speaker channels and converts them to text. This version was trained on audio mimicking telephone line conditions across ten languages; error rates in call center conversations dropped significantly.

What improved in this version

  • Turkish word error rate in call center scenarios dropped from 25.70% to 6.32%
  • Error rates in Arabic, German, French, English, and Italian fell to one-quarter and one-fifth respectively; per-language results shown in the chart
  • Improved in read Turkish speech as well: 4.72% → 4.47% (800 recordings)
  • Trained on telephone line, noisy environment, and studio versions of the same text
  • Previous version's data was also included in training; new data was added without degrading learned knowledge

What it does

  • Transcribes customer and agent separately in two-channel recordings
  • Detects each channel's language independently
  • Produces timestamped, speaker-labeled transcripts
  • Leaves silent sections blank; trained to avoid generating fabricated sentences

Limitations and responsible use

  • Measurements were performed on test sets; reference-based error rate on real phone calls has not yet been measured
  • Error increases with overlapping speech and very low-quality recordings
  • Real-time transcription latency was not measured

Training and measurements

19,369Step
1Epoch
0.0981Final validation loss
9.41%Final word error rate
2.82%Final character error rate
Training data2,479,180 recordings, 3,654 hours of audio
New data share26.5%; the remainder is from the previous version
LanguagesTurkish, Arabic, German, French, Kazakh, Italian, English, Bashkir, Uzbek, Tatar
Audio typestelephone line simulation, noisy environment, studio
Validation set11,859 recordings
Steps19,369 · 1 epoch
Training dateSeptember 2026
Training loss
0 5 10 15 5,000 10,000 15,000 Step Training loss Training loss: 175 · 10.81 Training loss: 350 · 8.66 Training loss: 525 · 7.60 Training loss: 700 · 7.07 Training loss: 875 · 6.61 Training loss: 1,050 · 6.46 Training loss: 1,225 · 6.17 Training loss: 1,400 · 6.18 Training loss: 1,575 · 5.93 Training loss: 1,750 · 5.77 Training loss: 1,925 · 5.65 Training loss: 2,100 · 5.73 Training loss: 2,275 · 5.54 Training loss: 2,450 · 5.41 Training loss: 2,625 · 5.55 Training loss: 2,800 · 5.33 Training loss: 2,975 · 5.34 Training loss: 3,150 · 5.20 Training loss: 3,325 · 5.12 Training loss: 3,500 · 5.12 Training loss: 3,675 · 5.02 Training loss: 3,850 · 4.99 Training loss: 4,025 · 4.94 Training loss: 4,200 · 4.99 Training loss: 4,375 · 4.91 Training loss: 4,550 · 4.74 Training loss: 4,725 · 4.77 Training loss: 4,900 · 4.85 Training loss: 5,075 · 4.82 Training loss: 5,250 · 4.71 Training loss: 5,425 · 4.61 Training loss: 5,600 · 4.58 Training loss: 5,775 · 4.61 Training loss: 5,950 · 4.55 Training loss: 6,125 · 4.63 Training loss: 6,300 · 4.61 Training loss: 6,475 · 4.57 Training loss: 6,650 · 4.54 Training loss: 6,825 · 4.53 Training loss: 7,000 · 4.50 Training loss: 7,175 · 4.50 Training loss: 7,350 · 4.46 Training loss: 7,525 · 4.53 Training loss: 7,700 · 4.44 Training loss: 7,875 · 4.41 Training loss: 8,050 · 4.39 Training loss: 8,225 · 4.45 Training loss: 8,400 · 4.41 Training loss: 8,575 · 4.40 Training loss: 8,750 · 4.43 Training loss: 8,925 · 4.44 Training loss: 9,100 · 4.28 Training loss: 9,275 · 4.34 Training loss: 9,450 · 4.24 Training loss: 9,625 · 4.20 Training loss: 9,800 · 4.34 Training loss: 9,975 · 4.30 Training loss: 10,150 · 4.28 Training loss: 10,325 · 4.29 Training loss: 10,500 · 4.22 Training loss: 10,675 · 4.23 Training loss: 10,850 · 4.21 Training loss: 11,025 · 4.26 Training loss: 11,200 · 4.28 Training loss: 11,375 · 4.23 Training loss: 11,550 · 4.18 Training loss: 11,725 · 4.15 Training loss: 11,900 · 4.15 Training loss: 12,075 · 4.10 Training loss: 12,250 · 4.16 Training loss: 12,425 · 4.28 Training loss: 12,600 · 4.22 Training loss: 12,775 · 4.27 Training loss: 12,950 · 4.21 Training loss: 13,125 · 4.19 Training loss: 13,300 · 4.16 Training loss: 13,475 · 4.10 Training loss: 13,650 · 4.15 Training loss: 13,825 · 4.20 Training loss: 14,000 · 4.12 Training loss: 14,175 · 4.15 Training loss: 14,350 · 4.16 Training loss: 14,525 · 4.14 Training loss: 14,700 · 4.13 Training loss: 14,875 · 4.12 Training loss: 15,050 · 4.17 Training loss: 15,225 · 3.98 Training loss: 15,400 · 4.18 Training loss: 15,575 · 4.07 Training loss: 15,750 · 4.17 Training loss: 15,925 · 4.08 Training loss: 16,100 · 4.06 Training loss: 16,275 · 4.12 Training loss: 16,450 · 4.15 Training loss: 16,625 · 4.07 Training loss: 16,800 · 4.00 Training loss: 16,975 · 4.13 Training loss: 17,150 · 4.04 Training loss: 17,325 · 4.11 Training loss: 17,500 · 4.15 Training loss: 17,675 · 4.05 Training loss: 17,850 · 4.04 Training loss: 18,025 · 4.13 Training loss: 18,200 · 4.06 Training loss: 18,375 · 4.10 Training loss: 18,550 · 4.20 Training loss: 18,725 · 3.94 Training loss: 18,900 · 4.06 Training loss: 19,075 · 4.09 Training loss: 19,250 · 4.07 Training loss: 19,350 · 4.02

Each point is the average of 7 consecutive recordings.

Show values as a table
Training loss
17510.8073
1,0506.4649
1,9255.6508
2,8005.3350
3,6755.0206
4,5504.7431
5,4254.6104
6,3004.6143
7,1754.5038
8,0504.3882
8,9254.4358
9,8004.3362
10,6754.2276
11,5504.1843
12,4254.2788
13,3004.1626
14,1754.1483
15,0504.1727
15,9254.0753
16,8004.0031
17,6754.0504
18,5504.2040
19,3504.0197
Validation loss
0.00 0.05 0.10 0.15 0.20 5,000 10,000 15,000 Step Validation loss Validation loss: 1,000 · 0.19 Validation loss: 1,000 · 0.19 Validation loss: 2,000 · 0.16 Validation loss: 2,000 · 0.16 Validation loss: 3,000 · 0.14 Validation loss: 3,000 · 0.14 Validation loss: 4,000 · 0.13 Validation loss: 4,000 · 0.13 Validation loss: 5,000 · 0.13 Validation loss: 5,000 · 0.13 Validation loss: 6,000 · 0.12 Validation loss: 6,000 · 0.12 Validation loss: 7,000 · 0.11 Validation loss: 7,000 · 0.11 Validation loss: 8,000 · 0.11 Validation loss: 8,000 · 0.11 Validation loss: 9,000 · 0.11 Validation loss: 9,000 · 0.11 Validation loss: 10,000 · 0.10 Validation loss: 10,000 · 0.10 Validation loss: 11,000 · 0.10 Validation loss: 11,000 · 0.10 Validation loss: 12,000 · 0.10 Validation loss: 12,000 · 0.10 Validation loss: 13,000 · 0.10 Validation loss: 13,000 · 0.10 Validation loss: 14,000 · 0.10 Validation loss: 14,000 · 0.10 Validation loss: 15,000 · 0.10 Validation loss: 15,000 · 0.10 Validation loss: 16,000 · 0.10 Validation loss: 16,000 · 0.10 Validation loss: 17,000 · 0.10 Validation loss: 17,000 · 0.10 Validation loss: 18,000 · 0.10 Validation loss: 18,000 · 0.10 Validation loss: 19,000 · 0.10 Validation loss: 19,000 · 0.10 Validation loss: 19,369 · 0.10 Validation loss: 19,369 · 0.10

Measured on examples never seen in training; a falling curve shows the model learns rather than memorises.

Show values as a table
Validation loss
1,0000.1911
2,0000.1611
3,0000.1446
4,0000.1345
5,0000.1259
6,0000.1187
7,0000.1135
8,0000.1097
9,0000.1068
10,0000.1050
11,0000.1025
12,0000.1021
13,0000.1010
14,0000.0997
15,0000.0994
16,0000.0985
17,0000.0985
18,0000.0984
19,0000.0980
19,3690.0981
Error rate on validation (%)
Word error rate (WER) Character error rate (CER)
0% 5% 10% 15% 20% 5,000 10,000 15,000 Step Word error rate (WER) Character error rate (CER) Word error rate (WER): 1,000 · 15.03% Word error rate (WER): 1,000 · 15.03% Word error rate (WER): 2,000 · 13.22% Word error rate (WER): 2,000 · 13.22% Word error rate (WER): 3,000 · 12.06% Word error rate (WER): 3,000 · 12.06% Word error rate (WER): 4,000 · 10.78% Word error rate (WER): 4,000 · 10.78% Word error rate (WER): 5,000 · 10.51% Word error rate (WER): 5,000 · 10.51% Word error rate (WER): 6,000 · 10.01% Word error rate (WER): 6,000 · 10.01% Word error rate (WER): 7,000 · 9.60% Word error rate (WER): 7,000 · 9.60% Word error rate (WER): 8,000 · 9.72% Word error rate (WER): 8,000 · 9.72% Word error rate (WER): 9,000 · 9.52% Word error rate (WER): 9,000 · 9.52% Word error rate (WER): 10,000 · 9.43% Word error rate (WER): 10,000 · 9.43% Word error rate (WER): 11,000 · 9.36% Word error rate (WER): 11,000 · 9.36% Word error rate (WER): 12,000 · 9.35% Word error rate (WER): 12,000 · 9.35% Word error rate (WER): 13,000 · 9.38% Word error rate (WER): 13,000 · 9.38% Word error rate (WER): 14,000 · 9.52% Word error rate (WER): 14,000 · 9.52% Word error rate (WER): 15,000 · 9.23% Word error rate (WER): 15,000 · 9.23% Word error rate (WER): 16,000 · 9.36% Word error rate (WER): 16,000 · 9.36% Word error rate (WER): 17,000 · 9.35% Word error rate (WER): 17,000 · 9.35% Word error rate (WER): 18,000 · 9.36% Word error rate (WER): 18,000 · 9.36% Word error rate (WER): 19,000 · 9.24% Word error rate (WER): 19,000 · 9.24% Word error rate (WER): 19,369 · 9.41% Word error rate (WER): 19,369 · 9.41% Character error rate (CER): 1,000 · 4.24% Character error rate (CER): 1,000 · 4.24% Character error rate (CER): 2,000 · 3.85% Character error rate (CER): 2,000 · 3.85% Character error rate (CER): 3,000 · 3.47% Character error rate (CER): 3,000 · 3.47% Character error rate (CER): 4,000 · 3.13% Character error rate (CER): 4,000 · 3.13% Character error rate (CER): 5,000 · 3.01% Character error rate (CER): 5,000 · 3.01% Character error rate (CER): 6,000 · 2.94% Character error rate (CER): 6,000 · 2.94% Character error rate (CER): 7,000 · 2.87% Character error rate (CER): 7,000 · 2.87% Character error rate (CER): 8,000 · 2.89% Character error rate (CER): 8,000 · 2.89% Character error rate (CER): 9,000 · 2.89% Character error rate (CER): 9,000 · 2.89% Character error rate (CER): 10,000 · 2.85% Character error rate (CER): 10,000 · 2.85% Character error rate (CER): 11,000 · 2.88% Character error rate (CER): 11,000 · 2.88% Character error rate (CER): 12,000 · 2.81% Character error rate (CER): 12,000 · 2.81% Character error rate (CER): 13,000 · 2.92% Character error rate (CER): 13,000 · 2.92% Character error rate (CER): 14,000 · 2.93% Character error rate (CER): 14,000 · 2.93% Character error rate (CER): 15,000 · 2.80% Character error rate (CER): 15,000 · 2.80% Character error rate (CER): 16,000 · 2.83% Character error rate (CER): 16,000 · 2.83% Character error rate (CER): 17,000 · 2.77% Character error rate (CER): 17,000 · 2.77% Character error rate (CER): 18,000 · 2.80% Character error rate (CER): 18,000 · 2.80% Character error rate (CER): 19,000 · 2.72% Character error rate (CER): 19,000 · 2.72% Character error rate (CER): 19,369 · 2.82% Character error rate (CER): 19,369 · 2.82%

Lower is better. Measured on the validation set throughout training.

Show values as a table
Word error rate (WER)
1,00015.03
2,00013.22
3,00012.06
4,00010.78
5,00010.51
6,00010.01
7,0009.60
8,0009.72
9,0009.52
10,0009.43
11,0009.36
12,0009.35
13,0009.38
14,0009.52
15,0009.23
16,0009.36
17,0009.35
18,0009.36
19,0009.24
19,3699.41
Character error rate (CER)
1,0004.24
2,0003.85
3,0003.47
4,0003.13
5,0003.01
6,0002.94
7,0002.87
8,0002.89
9,0002.89
10,0002.85
11,0002.88
12,0002.81
13,0002.92
14,0002.93
15,0002.80
16,0002.83
17,0002.77
18,0002.80
19,0002.72
19,3692.82
Call center evaluation · Per-language word error rate (%)
v3 v4
0% 20% 40% 60% 80% Arabic German English French Italian Turkish Arabic · v3: 67.6% 67.6% Arabic · v4: 13.9% 13.9% German · v3: 54.6% 54.6% German · v4: 15.4% 15.4% English · v3: 59.9% 59.9% English · v4: 12.6% 12.6% French · v3: 62.4% 62.4% French · v4: 15.6% 15.6% Italian · v3: 38.9% 38.9% Italian · v4: 8.4% 8.4% Turkish · v3: 25.7% 25.7% Turkish · v4: 6.3% 6.3%

Lower is better. 300 samples per language.

Show values as a table
v3v4
Arabic67.5913.86
German54.5815.37
English59.9212.58
French62.3615.57
Italian38.888.42
Turkish25.706.32
Call center evaluation · Per-language character error rate (%)
v3 v4
0% 20% 40% 60% Arabic German English French Italian Turkish Arabic · v3: 48.6% 48.6% Arabic · v4: 5.8% 5.8% German · v3: 46.0% 46.0% German · v4: 7.2% 7.2% English · v3: 44.4% 44.4% English · v4: 6.2% 6.2% French · v3: 47.9% 47.9% French · v4: 7.3% 7.3% Italian · v3: 26.9% 26.9% Italian · v4: 3.5% 3.5% Turkish · v3: 10.5% 10.5% Turkish · v4: 2.0% 2.0%

Lower is better. 300 samples per language.

Show values as a table
v3v4
Arabic48.625.76
German46.007.23
English44.416.23
French47.927.34
Italian26.923.45
Turkish10.471.96
Read Turkish speech · Per-language word error rate (%)
v3 v4
0% 2% 4% 6% Turkish Turkish · v3: 4.7% 4.7% Turkish · v4: 4.5% 4.5%

Lower is better. 800 samples per language.

Show values as a table
v3v4
Turkish4.724.47
Read Turkish speech · Character error rate by language (%)
v3 v4
0.0% 0.5% 1.0% 1.5% Turkish Turkish · v3: 1.1% 1.1% Turkish · v4: 1.0% 1.0%

Lower is better. 800 samples per language.

Show values as a table
v3v4
Turkish1.110.95

Comparison with open models

CALLAII ASR was measured on the same Turkish audio clips using the same methodology as widely used open speech recognition models.

October 8, 2026Measurement date
1,000Number of clips
3Compared model
  • In word error rate, CALLAII ASR recorded the lowest value in all four tests.
  • In character error rate, the difference between read speech and phone line is 0.04 points; the models are at the same level.
  • The training portion of the volunteer recordings is included in CALLAII ASR training; the portion used in testing is not.
  • The phone line condition increases error across all models. This is the focus of the next version.
  • Reference-based measurement on real phone conversations has not yet been performed.
Word error rate comparison (%)
Qwen3-ASR 1.7B Whisper large-v3 CALLAII ASR
0% 10% 20% 30% 40% Read speech Telephone line Volunteer recordin… Call scenarios Read speech · Qwen3-ASR 1.7B: 9.8% 9.8% Read speech · Whisper large-v3: 7.0% 7.0% Read speech · CALLAII ASR: 6.7% 6.7% Telephone line · Qwen3-ASR 1.7B: 11.3% 11.3% Telephone line · Whisper large-v3: 8.3% 8.3% Telephone line · CALLAII ASR: 8.0% 8.0% Volunteer recordings · Qwen3-ASR 1.7B: 15.8% 15.8% Volunteer recordings · Whisper large-v3: 13.1% 13.1% Volunteer recordings · CALLAII ASR: 4.1% 4.1% Call scenarios · Qwen3-ASR 1.7B: 30.7% 30.7% Call scenarios · Whisper large-v3: 22.3% 22.3% Call scenarios · CALLAII ASR: 4.5% 4.5%

Lower is better. Same 1,000 Turkish clips and same measurement method. Read speech and telephone line: FLEURS; telephone line uses the same clips under 8 kHz telephone conditions. Volunteer recordings: Common Voice. Call scenarios: sentences prepared from call center dialogues; CALLAII ASR was trained on this domain's data. Measurement: October 8, 2026.

Show values as a table
Qwen3-ASR 1.7BWhisper large-v3CALLAII ASR
Read speech9.757.046.67
Telephone line11.338.328.03
Volunteer recordings15.8013.144.10
Call scenarios30.7422.334.45
Character error rate comparison (%)
Qwen3-ASR 1.7B Whisper large-v3 CALLAII ASR
0% 5% 10% 15% Read speech Phone line Volunteer recordin… Call scenarios Read speech · Qwen3-ASR 1.7B: 2.4% 2.4% Read speech · Whisper large-v3: 1.6% 1.6% Read speech · CALLAII ASR: 1.6% 1.6% Phone line · Qwen3-ASR 1.7B: 3.0% 3.0% Phone line · Whisper large-v3: 1.9% 1.9% Phone line · CALLAII ASR: 2.0% 2.0% Volunteer recordings · Qwen3-ASR 1.7B: 4.4% 4.4% Volunteer recordings · Whisper large-v3: 4.5% 4.5% Volunteer recordings · CALLAII ASR: 1.0% 1.0% Call scenarios · Qwen3-ASR 1.7B: 12.0% 12.0% Call scenarios · Whisper large-v3: 8.0% 8.0% Call scenarios · CALLAII ASR: 1.0% 1.0%

Lower is better. Same 1,000 Turkish clips and same measurement method. Read speech and telephone line: FLEURS; telephone line uses the same clips under 8 kHz telephone conditions. Volunteer recordings: Common Voice. Call scenarios: sentences prepared from call center dialogues; CALLAII ASR was trained on this domain's data. Measurement: October 8, 2026.

Show values as a table
Qwen3-ASR 1.7BWhisper large-v3CALLAII ASR
Read speech2.431.561.60
Phone line3.041.941.98
Volunteer recordings4.444.530.98
Call scenarios12.027.971.00

Technical sheet

Versionv4
ReleaseSeptember 8, 2026
InputAudio recording, single or dual channel
OutputSpeaker-labeled, timestamped text
Language detectionAutomatic per channel
Runs onServers managed by Callmenta

Customer data is never used to train the models. This sheet was generated on 23.11.2026. See callmenta.com for the current version. Versions and improvements