CALLAII® model card · Callmenta
CALLAII® by CALLMENTA

CALLAII ASR v3 technical sheet

Speech recognition · A speech recognition model that separates call recordings into speaker channels and converts them into text. Analysis, coaching, and quality scoring operate on this transcription.

CALLAII ASR v3 Previous version Released August 2026

Version with extensive multilingual training. Phone line simulation and silence detection were added in this version.

What improved in this version

  • Turkish word error rate dropped from 8.87% to 7.11% on the open benchmark set
  • Arabic dropped from 7.96% to 7.82%
  • Learned to leave silent sections blank; hallucinated sentences decreased
  • No degradation observed at the end of long calls (571-second real call)
  • Automatic language detection per channel has begun

Limitations and responsible use

  • The open benchmark set consists of read news sentences; it does not represent absolute success in phone calls
  • Error rates were high in languages other than Turkish in call center scenarios; resolved in v4

Training and measurements

11.540Step
1Epoch
0,1315Final validation loss
%10,49Final word error rate
%3,01Final character error rate
Training data1,477,006 records, 2,190.9 hours of audio
Silence samples25.7 hours
Audio typesphone band simulation (8 kHz), cross-mixing, dialogue segmentation
Steps11,540 · 1 epoch
Training dateAugust 2026
Training loss
0 1 2 3 2,500 5,000 7,500 10,000 Step Training loss Training loss: 100 · 2.63 Training loss: 200 · 1.90 Training loss: 300 · 1.80 Training loss: 400 · 1.82 Training loss: 500 · 1.72 Training loss: 600 · 1.67 Training loss: 700 · 1.67 Training loss: 800 · 1.68 Training loss: 900 · 1.67 Training loss: 1,000 · 1.66 Training loss: 1,100 · 1.60 Training loss: 1,200 · 1.57 Training loss: 1,300 · 1.54 Training loss: 1,400 · 1.56 Training loss: 1,500 · 1.56 Training loss: 1,600 · 1.56 Training loss: 1,700 · 1.53 Training loss: 1,800 · 1.50 Training loss: 1,900 · 1.52 Training loss: 2,000 · 1.51 Training loss: 2,100 · 1.52 Training loss: 2,200 · 1.47 Training loss: 2,300 · 1.48 Training loss: 2,400 · 1.45 Training loss: 2,500 · 1.44 Training loss: 2,600 · 1.43 Training loss: 2,700 · 1.45 Training loss: 2,800 · 1.44 Training loss: 2,900 · 1.44 Training loss: 3,000 · 1.42 Training loss: 3,100 · 1.41 Training loss: 3,200 · 1.46 Training loss: 3,300 · 1.37 Training loss: 3,400 · 1.41 Training loss: 3,500 · 1.46 Training loss: 3,600 · 1.35 Training loss: 3,700 · 1.37 Training loss: 3,800 · 1.35 Training loss: 3,900 · 1.39 Training loss: 4,000 · 1.40 Training loss: 4,100 · 1.38 Training loss: 4,200 · 1.38 Training loss: 4,300 · 1.36 Training loss: 4,400 · 1.38 Training loss: 4,500 · 1.34 Training loss: 4,600 · 1.35 Training loss: 4,700 · 1.36 Training loss: 4,800 · 1.31 Training loss: 4,900 · 1.30 Training loss: 5,000 · 1.34 Training loss: 5,100 · 1.33 Training loss: 5,200 · 1.31 Training loss: 5,300 · 1.30 Training loss: 5,400 · 1.31 Training loss: 5,500 · 1.33 Training loss: 5,600 · 1.33 Training loss: 5,700 · 1.27 Training loss: 5,800 · 1.32 Training loss: 5,900 · 1.33 Training loss: 6,000 · 1.31 Training loss: 6,100 · 1.30 Training loss: 6,200 · 1.27 Training loss: 6,300 · 1.26 Training loss: 6,400 · 1.33 Training loss: 6,500 · 1.29 Training loss: 6,600 · 1.29 Training loss: 6,700 · 1.31 Training loss: 6,800 · 1.22 Training loss: 6,900 · 1.25 Training loss: 7,000 · 1.31 Training loss: 7,100 · 1.27 Training loss: 7,200 · 1.23 Training loss: 7,300 · 1.26 Training loss: 7,400 · 1.27 Training loss: 7,500 · 1.25 Training loss: 7,600 · 1.26 Training loss: 7,700 · 1.24 Training loss: 7,800 · 1.25 Training loss: 7,900 · 1.24 Training loss: 8,000 · 1.24 Training loss: 8,100 · 1.26 Training loss: 8,200 · 1.23 Training loss: 8,300 · 1.24 Training loss: 8,400 · 1.24 Training loss: 8,500 · 1.23 Training loss: 8,600 · 1.27 Training loss: 8,700 · 1.23 Training loss: 8,800 · 1.22 Training loss: 8,900 · 1.20 Training loss: 9,000 · 1.22 Training loss: 9,100 · 1.22 Training loss: 9,200 · 1.22 Training loss: 9,300 · 1.24 Training loss: 9,400 · 1.23 Training loss: 9,500 · 1.21 Training loss: 9,600 · 1.23 Training loss: 9,700 · 1.18 Training loss: 9,800 · 1.23 Training loss: 9,900 · 1.21 Training loss: 10,000 · 1.18 Training loss: 10,100 · 1.22 Training loss: 10,200 · 1.21 Training loss: 10,300 · 1.26 Training loss: 10,400 · 1.20 Training loss: 10,500 · 1.20 Training loss: 10,600 · 1.21 Training loss: 10,700 · 1.21 Training loss: 10,800 · 1.23 Training loss: 10,900 · 1.17 Training loss: 11,000 · 1.19 Training loss: 11,100 · 1.18 Training loss: 11,200 · 1.18 Training loss: 11,300 · 1.20 Training loss: 11,400 · 1.20 Training loss: 11,500 · 1.17

Each point is the average of two consecutive records.

Show values as a table
Training loss
1002.6340
6001.6663
1,1001.6001
1,6001.5598
2,1001.5177
2,6001.4338
3,1001.4080
3,6001.3531
4,1001.3821
4,6001.3530
5,1001.3306
5,6001.3333
6,1001.3021
6,6001.2923
7,1001.2736
7,6001.2613
8,1001.2606
8,6001.2679
9,1001.2166
9,6001.2256
10,1001.2165
10,6001.2068
11,1001.1835
11,5001.1665
Validation loss
0.13 0.14 0.15 0.16 0.17 2,000 4,000 6,000 8,000 10,000 Step Validation loss Validation loss: 2,000 · 0.16 Validation loss: 2,000 · 0.16 Validation loss: 4,000 · 0.15 Validation loss: 4,000 · 0.15 Validation loss: 6,000 · 0.15 Validation loss: 6,000 · 0.15 Validation loss: 8,000 · 0.14 Validation loss: 8,000 · 0.14 Validation loss: 10,000 · 0.13 Validation loss: 10,000 · 0.13 Validation loss: 11,540 · 0.13 Validation loss: 11,540 · 0.13

Measured on examples never seen in training; a falling curve shows the model learns rather than memorises.

Show values as a table
Validation loss
2,0000.1633
4,0000.1528
6,0000.1456
8,0000.1383
10,0000.1340
11,5400.1316
Error rate on validation (%)
Word error rate (WER) Character error rate (CER)
0% 5% 10% 15% 20% 2,000 4,000 6,000 8,000 10,000 Step Word error rate (WER) Character error rate (CER) Word error rate (WER): 2,000 · 12.60% Word error rate (WER): 2,000 · 12.60% Word error rate (WER): 4,000 · 11.94% Word error rate (WER): 4,000 · 11.94% Word error rate (WER): 6,000 · 19.10% Word error rate (WER): 6,000 · 19.10% Word error rate (WER): 8,000 · 10.71% Word error rate (WER): 8,000 · 10.71% Word error rate (WER): 10,000 · 18.35% Word error rate (WER): 10,000 · 18.35% Word error rate (WER): 11,540 · 10.49% Word error rate (WER): 11,540 · 10.49% Character error rate (CER): 2,000 · 3.64% Character error rate (CER): 2,000 · 3.64% Character error rate (CER): 4,000 · 3.33% Character error rate (CER): 4,000 · 3.33% Character error rate (CER): 6,000 · 10.35% Character error rate (CER): 6,000 · 10.35% Character error rate (CER): 8,000 · 2.94% Character error rate (CER): 8,000 · 2.94% Character error rate (CER): 10,000 · 10.79% Character error rate (CER): 10,000 · 10.79% Character error rate (CER): 11,540 · 3.01% Character error rate (CER): 11,540 · 3.01%

Lower is better. Measured on the validation set throughout training.

Show values as a table
Word error rate (WER)
2,00012.60
4,00011.94
6,00019.10
8,00010.71
10,00018.35
11,54010.49
Character error rate (CER)
2,0003.64
4,0003.33
6,00010.35
8,0002.94
10,00010.79
11,5403.01
Open benchmark set (FLEURS) · Word error rate (%)
v2 v3
0.0% 2.5% 5.0% 7.5% 10.0% Turkish Arabic Turkish · v2: 8.9% 8.9% Turkish · v3: 7.1% 7.1% Arabic · v2: 8.0% 8.0% Arabic · v3: 7.8% 7.8%

Lower is better. 150 records per language, read news sentences. Valid for ranking; does not represent absolute success in phone calls.

Show values as a table
v2v3
Turkish8.877.11
Arabic7.967.82
Open benchmark set (FLEURS) · Character error rate (%)
v2 v3
0% 1% 2% 3% Turkish Arabic Turkish · v2: 2.8% 2.8% Turkish · v3: 1.7% 1.7% Arabic · v2: 2.7% 2.7% Arabic · v3: 2.5% 2.5%

Lower is better. 150 records per language.

Show values as a table
v2v3
Turkish2.831.67
Arabic2.652.46
Internal test set · Language-based word error rate (%)
0% 5% 10% 15% 20% Arabic German English French Italian Turkish Arabic · v3: 19.7% 19.7% German · v3: 6.8% 6.8% English · v3: 19.1% 19.1% French · v3: 12.7% 12.7% Italian · v3: 8.1% 8.1% Turkish · v3: 10.5% 10.5%

Lower is better. 200 samples per language.

Show values as a table
v3
Arabic19.72
German6.80
English19.10
French12.74
Italian8.11
Turkish10.49
Internal test set · Language-based character error rate (%)
0% 5% 10% 15% Arabic German English French Italian Turkish Arabic · v3: 7.4% 7.4% German · v3: 2.0% 2.0% English · v3: 11.9% 11.9% French · v3: 3.8% 3.8% Italian · v3: 2.2% 2.2% Turkish · v3: 3.0% 3.0%

Lower is better. 200 samples per language.

Show values as a table
v3
Arabic7.41
German1.95
English11.89
French3.75
Italian2.22
Turkish3.01

Technical sheet

Versionv3
ReleaseAugust 25, 2026
StatusSuperseded by v4

Customer data is never used to train the models. This sheet was generated on 27.09.2026. See callmenta.com for the current version. Versions and improvements