Skip to content
Sign In Start free trial
CALLAII models

CALLAII® Versions and improvements

Every version of the CALLAII models that power the AI in Callmenta: what it does, what improved in this version and where its limits are.

Customer data is never used to train the models.

CALLAII® by CALLMENTA

CALLAII

Language model

The language model powering the AI surfaces in the Callmenta panel: call analysis, coaching, live support, CRM, and CALLAII Legal.

CALLAII v7 Live Released September 2026

CALLAII v7 is live. The previous live version, v6, was put through the same exam under live settings. On legal questions, it checks procedure almost every time: competent and authorized court, conditions of the lawsuit, and deadlines. It works alongside an additional audit that catches and extracts information the lawyer did not mention in petitions.

What improved in this version

  • Procedure check in legal mode: performed in 69 out of 70 questions; live v6 did not perform it in any of the same exam.
  • Procedure check points in legal mode rose from 8 to 57 across 70 questions.
  • Procedure check in general mode with thinking off rose from 0 to 23.
  • Answers are more concise: answers cut off at the limit in general mode dropped from 40 to 11.
  • Text extraction from document images and image description were fully preserved after training.

Limitations and responsible use

  • Legal knowledge does not come from the model's memory; laws, regulations, and rulings are fetched from the current library every time.
  • The text it generates is a draft; petitions, contracts, and official documents must be approved by an authorized person.
  • Numbers and dates must be verified.
  • Quality in languages other than Turkish has not been measured in as much detail as Turkish.

Training and measurements

0,629 → 0,513Validation loss
69 out of 70 questionsProcedure check · legal
0 out of 70 questionsLive v6 on the same exam
8 → 57Procedure points · legal
Training data45,847 samples
Validation set200 samples never seen during training
Steps2,859 · 1 epoch · 16 samples per step
Training context4,096 tokens
Validation loss0.629 → 0.513
Exam70 procedure questions; same questions as live v6, live settings
Training dateSeptember 2026
Training and validation loss
Training loss Validation loss
0.4 0.5 0.6 0.7 0.8 0.9 0.2 0.4 0.6 0.8 1.0 Epoch Training loss Validation loss Training loss: 0.0 · 0.84 Training loss: 0.0 · 0.76 Training loss: 0.0 · 0.68 Training loss: 0.0 · 0.67 Training loss: 0.0 · 0.64 Training loss: 0.1 · 0.65 Training loss: 0.1 · 0.63 Training loss: 0.1 · 0.61 Training loss: 0.1 · 0.60 Training loss: 0.1 · 0.61 Training loss: 0.1 · 0.60 Training loss: 0.1 · 0.59 Training loss: 0.1 · 0.60 Training loss: 0.1 · 0.58 Training loss: 0.1 · 0.61 Training loss: 0.1 · 0.58 Training loss: 0.1 · 0.58 Training loss: 0.2 · 0.59 Training loss: 0.2 · 0.57 Training loss: 0.2 · 0.56 Training loss: 0.2 · 0.56 Training loss: 0.2 · 0.55 Training loss: 0.2 · 0.55 Training loss: 0.2 · 0.56 Training loss: 0.2 · 0.55 Training loss: 0.2 · 0.56 Training loss: 0.2 · 0.55 Training loss: 0.2 · 0.57 Training loss: 0.3 · 0.56 Training loss: 0.3 · 0.55 Training loss: 0.3 · 0.55 Training loss: 0.3 · 0.56 Training loss: 0.3 · 0.54 Training loss: 0.3 · 0.54 Training loss: 0.3 · 0.55 Training loss: 0.3 · 0.54 Training loss: 0.3 · 0.56 Training loss: 0.3 · 0.57 Training loss: 0.3 · 0.56 Training loss: 0.3 · 0.56 Training loss: 0.4 · 0.55 Training loss: 0.4 · 0.54 Training loss: 0.4 · 0.53 Training loss: 0.4 · 0.52 Training loss: 0.4 · 0.54 Training loss: 0.4 · 0.52 Training loss: 0.4 · 0.55 Training loss: 0.4 · 0.54 Training loss: 0.4 · 0.51 Training loss: 0.4 · 0.52 Training loss: 0.4 · 0.54 Training loss: 0.5 · 0.55 Training loss: 0.5 · 0.51 Training loss: 0.5 · 0.54 Training loss: 0.5 · 0.52 Training loss: 0.5 · 0.54 Training loss: 0.5 · 0.53 Training loss: 0.5 · 0.54 Training loss: 0.5 · 0.54 Training loss: 0.5 · 0.53 Training loss: 0.5 · 0.53 Training loss: 0.5 · 0.52 Training loss: 0.6 · 0.52 Training loss: 0.6 · 0.54 Training loss: 0.6 · 0.52 Training loss: 0.6 · 0.53 Training loss: 0.6 · 0.53 Training loss: 0.6 · 0.51 Training loss: 0.6 · 0.52 Training loss: 0.6 · 0.52 Training loss: 0.6 · 0.51 Training loss: 0.6 · 0.50 Training loss: 0.6 · 0.50 Training loss: 0.6 · 0.53 Training loss: 0.7 · 0.50 Training loss: 0.7 · 0.52 Training loss: 0.7 · 0.51 Training loss: 0.7 · 0.53 Training loss: 0.7 · 0.52 Training loss: 0.7 · 0.53 Training loss: 0.7 · 0.51 Training loss: 0.7 · 0.51 Training loss: 0.7 · 0.53 Training loss: 0.7 · 0.52 Training loss: 0.7 · 0.50 Training loss: 0.8 · 0.53 Training loss: 0.8 · 0.51 Training loss: 0.8 · 0.50 Training loss: 0.8 · 0.52 Training loss: 0.8 · 0.51 Training loss: 0.8 · 0.53 Training loss: 0.8 · 0.49 Training loss: 0.8 · 0.50 Training loss: 0.8 · 0.51 Training loss: 0.8 · 0.50 Training loss: 0.8 · 0.50 Training loss: 0.8 · 0.50 Training loss: 0.9 · 0.49 Training loss: 0.9 · 0.49 Training loss: 0.9 · 0.51 Training loss: 0.9 · 0.51 Training loss: 0.9 · 0.53 Training loss: 0.9 · 0.50 Training loss: 0.9 · 0.49 Training loss: 0.9 · 0.51 Training loss: 0.9 · 0.51 Training loss: 0.9 · 0.53 Training loss: 0.9 · 0.51 Training loss: 1.0 · 0.50 Training loss: 1.0 · 0.49 Training loss: 1.0 · 0.50 Training loss: 1.0 · 0.50 Training loss: 1.0 · 0.50 Training loss: 1.0 · 0.50 Training loss: 1.0 · 0.49 Validation loss: 0.1 · 0.63 Validation loss: 0.1 · 0.63 Validation loss: 0.1 · 0.60 Validation loss: 0.1 · 0.60 Validation loss: 0.2 · 0.58 Validation loss: 0.2 · 0.58 Validation loss: 0.3 · 0.56 Validation loss: 0.3 · 0.56 Validation loss: 0.3 · 0.55 Validation loss: 0.3 · 0.55 Validation loss: 0.4 · 0.54 Validation loss: 0.4 · 0.54 Validation loss: 0.5 · 0.54 Validation loss: 0.5 · 0.54 Validation loss: 0.6 · 0.53 Validation loss: 0.6 · 0.53 Validation loss: 0.6 · 0.52 Validation loss: 0.6 · 0.52 Validation loss: 0.7 · 0.52 Validation loss: 0.7 · 0.52 Validation loss: 0.8 · 0.52 Validation loss: 0.8 · 0.52 Validation loss: 0.8 · 0.51 Validation loss: 0.8 · 0.51 Validation loss: 0.9 · 0.51 Validation loss: 0.9 · 0.51 Validation loss: 1.0 · 0.51 Validation loss: 1.0 · 0.51 Validation loss: 1.0 · 0.51 Validation loss: 1.0 · 0.51

Each point is the average of 5 consecutive records.

Show values as a table
Training loss
0.010.8421
0.050.6483
0.100.5975
0.140.5828
0.180.5622
0.230.5602
0.270.5509
0.310.5387
0.360.5504
0.400.5220
0.450.5358
0.490.5406
0.530.5296
0.580.5333
0.620.5053
0.660.5165
0.710.5090
0.750.5308
0.800.5334
0.840.5043
0.880.5118
0.930.5051
0.970.5022
1.000.4946
Validation loss
0.070.6293
0.140.5990
0.210.5781
0.280.5642
0.350.5544
0.420.5433
0.490.5368
0.560.5298
0.630.5238
0.700.5194
0.770.5163
0.840.5139
0.910.5129
0.980.5127
10.5126
Live v6 vs v7: procedure exam
Live v6 v7
0% 25% 50% 75% 100% Procedure chec… Procedure poin… Procedure chec… Procedure poin… Procedure check · legal · Live v6: 0.0% 0.0% Procedure check · legal · v7: 98.6% 98.6% Procedure points · legal · Live v6: 11.4% 11.4% Procedure points · legal · v7: 81.4% 81.4% Procedure check · general · Live v6: 0.0% 0.0% Procedure check · general · v7: 32.9% 32.9% Procedure points · general · Live v6: 11.4% 11.4% Procedure points · general · v7: 45.7% 45.7%

Same 70 legal questions, live settings. Thinking on in legal mode, off in general mode.

Show values as a table
Live v6v7
Procedure check · legal0.0098.60
Procedure points · legal11.4081.40
Procedure check · general0.0032.90
Procedure points · general11.4045.70

Technical sheet

Versionv7
ReleaseSeptember 2026
InputText and image
OutputText
LanguageTurkish-focused; responds in the same language for other languages
Open technical sheet

Previous versions

CALLAII v6 · Previous version
CALLAII v6 Previous version Released September 2026

CALLAII v6 powers all AI surfaces within the Callmenta panel. It was trained on data generated from scratch for this purpose; every sample passed rule-based checks and a judge evaluation.

What improved in this version

  • Petition and contract style was learned from real documents of the law firm we work with
  • Contract drafting and evidence review were added
  • Multi-turn conversation and tool use were strengthened
  • Training was regularly evaluated on 740 unseen samples; validation error decreased at every measurement, and the model did not memorize

What it does

  • Call analysis: summary, action items, missed opportunities, and criteria scorecard from call transcripts
  • Coaching: evidence-based feedback for agent performance
  • Live support: answers visitor questions and escalates to a human when needed
  • CRM and reporting: customer card summary, metric interpretation, and commentary
  • CALLAII Legal: drafting petitions and contracts, document analysis, and legal/regulatory/case-law queries
  • KVKK masking: extracting personal data from text
  • Tool use: search, record creation, and document retrieval within the panel
  • Visual reading: understanding documents and screenshots

Limitations and responsible use

  • Legal knowledge does not come from the model's memory; laws, regulations, and rulings are retrieved from the current library each time
  • Generated text is a draft; petitions, contracts, and official correspondence must be approved by an authorized person
  • Numbers and dates must be verified; the model was trained to state missing information rather than fabricate it
  • Very long documents are processed in segments
  • Quality in languages other than Turkish has not been measured with the same granularity as Turkish

Training and measurements

3.101Step
2Epoch
0,4729Final validation loss
Training data37,102 samples, 118 million tokens
Quality filtering37,600 of 47,872 generated samples were accepted
Validation set740 samples never seen during training
Steps3,101 · 2 epochs · 96,000 tokens per step
Training context12,288 tokens
Validation loss0.7435 → 0.4729
Training dateSeptember 2026
Training and validation loss
Training loss Validation loss
0.0 0.2 0.4 0.6 0.8 0 1,000 2,000 3,000 Step Training loss Validation loss Training loss: 0 · 0.78 Training loss: 0 · 0.78 Training loss: 2,700 · 0.40 Training loss: 2,700 · 0.40 Training loss: 3,101 · 0.38 Training loss: 3,101 · 0.38 Validation loss: 0 · 0.74 Validation loss: 0 · 0.74 Validation loss: 2,700 · 0.47 Validation loss: 2,700 · 0.47 Validation loss: 3,101 · 0.47 Validation loss: 3,101 · 0.47

Only measured points; records of intermediate steps were not retained.

Show values as a table
Training loss
00.7800
2,7000.3951
3,1010.3763
Validation loss
00.7435
2,7000.4730
3,1010.4729
Training data filtering (example)
0 20,000 40,000 60,000 Generated Accepted Training Validation Generated · Samples: 47,872.0 47,872.0 Accepted · Samples: 37,600.0 37,600.0 Training · Samples: 37,102.0 37,102.0 Validation · Samples: 740.0 740.0

Every generated sample passed rule-based checks and judge evaluation; those that failed were discarded.

Show values as a table
Samples
Generated47,872.00
Accepted37,600.00
Training37,102.00
Validation740.00

Technical sheet

Versionv6
ReleaseSeptember 2026
InputText and images
OutputText
LanguageTurkish-focused; other languages answered in the same language
Customer datanot used for training
Hostingservers managed by Callmenta; your data is not sent to a third-party AI provider
Open technical sheet
CALLAII v5 · Previous version
CALLAII v5 Previous version Released September 2026

CALLAII's first comprehensive release.

What improved in this version

  • Call center, coaching, and live support surfaces unified in a single model.

Training and measurements

411Step
1Epoch
0,4867Final validation loss
Training data45,000 samples, 15 categories, 9 languages
Steps411 · 1 epoch
Tokens processed26.7 million
Final validation loss0.4867
Validation token accuracy85.4%
Training and validation loss
Training loss Validation loss
0 1 2 3 100 200 300 400 Step Training loss Validation loss Training loss: 10 · 2.73 Training loss: 20 · 2.13 Training loss: 30 · 1.25 Training loss: 40 · 0.94 Training loss: 50 · 0.84 Training loss: 60 · 0.77 Training loss: 70 · 0.70 Training loss: 80 · 0.64 Training loss: 90 · 0.61 Training loss: 100 · 0.61 Training loss: 110 · 0.58 Training loss: 120 · 0.58 Training loss: 130 · 0.57 Training loss: 140 · 0.55 Training loss: 150 · 0.56 Training loss: 160 · 0.55 Training loss: 170 · 0.54 Training loss: 180 · 0.56 Training loss: 190 · 0.56 Training loss: 200 · 0.52 Training loss: 210 · 0.54 Training loss: 220 · 0.51 Training loss: 230 · 0.54 Training loss: 240 · 0.52 Training loss: 250 · 0.51 Training loss: 260 · 0.50 Training loss: 270 · 0.50 Training loss: 280 · 0.52 Training loss: 290 · 0.51 Training loss: 300 · 0.49 Training loss: 310 · 0.50 Training loss: 320 · 0.51 Training loss: 330 · 0.50 Training loss: 340 · 0.48 Training loss: 350 · 0.47 Training loss: 360 · 0.51 Training loss: 370 · 0.49 Training loss: 380 · 0.50 Training loss: 390 · 0.50 Training loss: 400 · 0.51 Training loss: 410 · 0.50 Validation loss: 40 · 0.88 Validation loss: 40 · 0.88 Validation loss: 80 · 0.62 Validation loss: 80 · 0.62 Validation loss: 120 · 0.57 Validation loss: 120 · 0.57 Validation loss: 160 · 0.54 Validation loss: 160 · 0.54 Validation loss: 200 · 0.52 Validation loss: 200 · 0.52 Validation loss: 240 · 0.51 Validation loss: 240 · 0.51 Validation loss: 280 · 0.50 Validation loss: 280 · 0.50 Validation loss: 320 · 0.49 Validation loss: 320 · 0.49 Validation loss: 360 · 0.49 Validation loss: 360 · 0.49 Validation loss: 400 · 0.49 Validation loss: 400 · 0.49 Validation loss: 411 · 0.49 Validation loss: 411 · 0.49
Show values as a table
Training loss
102.7335
301.2499
500.8402
700.7024
900.6135
1100.5847
1300.5689
1500.5584
1700.5385
1900.5599
2100.5412
2300.5371
2500.5122
2700.5038
2900.5125
3100.5011
3300.4956
3500.4728
3700.4874
3900.4968
4100.4969
Validation loss
400.8849
800.6208
1200.5651
1600.5402
2000.5221
2400.5106
2800.5010
3200.4927
3600.4880
4000.4867
4110.4867
Token accuracy (%)
Training Validation
50% 60% 70% 80% 90% 100 200 300 400 Step Training Validation Training: 10 · 57.20% Training: 20 · 62.39% Training: 30 · 70.07% Training: 40 · 75.85% Training: 50 · 78.02% Training: 60 · 79.40% Training: 70 · 80.74% Training: 80 · 82.12% Training: 90 · 82.59% Training: 100 · 82.66% Training: 110 · 83.02% Training: 120 · 83.18% Training: 130 · 83.40% Training: 140 · 83.70% Training: 150 · 83.64% Training: 160 · 83.54% Training: 170 · 84.01% Training: 180 · 83.55% Training: 190 · 83.63% Training: 200 · 84.33% Training: 210 · 83.78% Training: 220 · 84.54% Training: 230 · 83.92% Training: 240 · 84.32% Training: 250 · 84.41% Training: 260 · 84.74% Training: 270 · 84.64% Training: 280 · 84.24% Training: 290 · 84.49% Training: 300 · 85.05% Training: 310 · 84.78% Training: 320 · 84.73% Training: 330 · 84.86% Training: 340 · 85.15% Training: 350 · 85.33% Training: 360 · 84.60% Training: 370 · 85.09% Training: 380 · 84.89% Training: 390 · 84.94% Training: 400 · 84.42% Training: 410 · 84.81% Validation: 40 · 77.40% Validation: 40 · 77.40% Validation: 80 · 82.75% Validation: 80 · 82.75% Validation: 120 · 83.79% Validation: 120 · 83.79% Validation: 160 · 84.25% Validation: 160 · 84.25% Validation: 200 · 84.61% Validation: 200 · 84.61% Validation: 240 · 84.85% Validation: 240 · 84.85% Validation: 280 · 85.04% Validation: 280 · 85.04% Validation: 320 · 85.23% Validation: 320 · 85.23% Validation: 360 · 85.32% Validation: 360 · 85.32% Validation: 400 · 85.37% Validation: 400 · 85.37% Validation: 411 · 85.37% Validation: 411 · 85.37%
Show values as a table
Training
1057.20
3070.07
5078.02
7080.74
9082.59
11083.02
13083.40
15083.64
17084.01
19083.63
21083.78
23083.92
25084.41
27084.64
29084.49
31084.78
33084.86
35085.33
37085.09
39084.94
41084.81
Validation
4077.40
8082.75
12083.79
16084.25
20084.61
24084.85
28085.04
32085.23
36085.32
40085.37
41185.37

Technical sheet

Versionv5
ReleaseSeptember 2026
StatusSuperseded by v6
Open technical sheet
CALLAII® by CALLMENTA

CALLAII Edge

Language model

The CALLAII language model prepared to run on the organization's own server.

CALLAII Edge v1 In training

CALLAII Edge is currently in training. The goal is to consolidate CALLAII's legal and call center capabilities into a model that runs on the organization's own server. The graph below is updated regularly during training.

What improved in this version

  • Goal: Identify the competent and authorized court, litigation prerequisites, and deadlines for a legal question
  • Goal: Introduce itself as CALLAII under all circumstances
  • Results will be published here after training is complete, using the same exam taken before training

Training and measurements

0,49Epoch
0,6186Final validation loss
StatusIn training
Training data45,847 samples
Validation set200 samples never seen during training
Training context4,096 tokens
Training dateSeptember 2026
Training and validation loss
Training loss Validation loss
0 2 4 6 0.1 0.2 0.3 0.4 Epoch Training loss Validation loss Training loss: 0.0 · 5.09 Training loss: 0.0 · 2.96 Training loss: 0.0 · 1.32 Training loss: 0.0 · 1.02 Training loss: 0.0 · 0.95 Training loss: 0.0 · 0.88 Training loss: 0.0 · 0.85 Training loss: 0.0 · 0.83 Training loss: 0.0 · 0.79 Training loss: 0.1 · 0.76 Training loss: 0.1 · 0.78 Training loss: 0.1 · 0.78 Training loss: 0.1 · 0.79 Training loss: 0.1 · 0.73 Training loss: 0.1 · 0.71 Training loss: 0.1 · 0.76 Training loss: 0.1 · 0.72 Training loss: 0.1 · 0.73 Training loss: 0.1 · 0.77 Training loss: 0.1 · 0.75 Training loss: 0.1 · 0.71 Training loss: 0.1 · 0.71 Training loss: 0.1 · 0.71 Training loss: 0.1 · 0.70 Training loss: 0.1 · 0.67 Training loss: 0.1 · 0.70 Training loss: 0.1 · 0.68 Training loss: 0.1 · 0.70 Training loss: 0.2 · 0.68 Training loss: 0.2 · 0.67 Training loss: 0.2 · 0.70 Training loss: 0.2 · 0.67 Training loss: 0.2 · 0.63 Training loss: 0.2 · 0.67 Training loss: 0.2 · 0.69 Training loss: 0.2 · 0.67 Training loss: 0.2 · 0.66 Training loss: 0.2 · 0.69 Training loss: 0.2 · 0.65 Training loss: 0.2 · 0.66 Training loss: 0.2 · 0.66 Training loss: 0.2 · 0.62 Training loss: 0.2 · 0.68 Training loss: 0.2 · 0.68 Training loss: 0.2 · 0.66 Training loss: 0.2 · 0.66 Training loss: 0.2 · 0.67 Training loss: 0.3 · 0.64 Training loss: 0.3 · 0.66 Training loss: 0.3 · 0.66 Training loss: 0.3 · 0.66 Training loss: 0.3 · 0.65 Training loss: 0.3 · 0.65 Training loss: 0.3 · 0.67 Training loss: 0.3 · 0.62 Training loss: 0.3 · 0.64 Training loss: 0.3 · 0.66 Training loss: 0.3 · 0.66 Training loss: 0.3 · 0.65 Training loss: 0.3 · 0.65 Training loss: 0.3 · 0.64 Training loss: 0.3 · 0.64 Training loss: 0.3 · 0.63 Training loss: 0.3 · 0.60 Training loss: 0.3 · 0.63 Training loss: 0.3 · 0.64 Training loss: 0.4 · 0.65 Training loss: 0.4 · 0.64 Training loss: 0.4 · 0.63 Training loss: 0.4 · 0.61 Training loss: 0.4 · 0.68 Training loss: 0.4 · 0.65 Training loss: 0.4 · 0.60 Training loss: 0.4 · 0.61 Training loss: 0.4 · 0.62 Training loss: 0.4 · 0.60 Training loss: 0.4 · 0.64 Training loss: 0.4 · 0.66 Training loss: 0.4 · 0.63 Training loss: 0.4 · 0.61 Training loss: 0.4 · 0.63 Training loss: 0.4 · 0.59 Training loss: 0.4 · 0.63 Training loss: 0.4 · 0.65 Training loss: 0.4 · 0.62 Training loss: 0.5 · 0.64 Training loss: 0.5 · 0.63 Training loss: 0.5 · 0.62 Training loss: 0.5 · 0.62 Training loss: 0.5 · 0.62 Training loss: 0.5 · 0.61 Training loss: 0.5 · 0.60 Training loss: 0.5 · 0.62 Training loss: 0.5 · 0.60 Validation loss: 0.1 · 0.74 Validation loss: 0.1 · 0.74 Validation loss: 0.3 · 0.63 Validation loss: 0.3 · 0.63 Validation loss: 0.4 · 0.62 Validation loss: 0.4 · 0.62

Each point is the average of 3 consecutive records.

Show values as a table
Training loss
0.015.0917
0.030.9474
0.050.7875
0.070.7862
0.090.7166
0.110.7118
0.130.6670
0.150.6756
0.170.6324
0.190.6646
0.220.6551
0.240.6583
0.260.6638
0.280.6530
0.300.6599
0.320.6420
0.340.6337
0.360.6260
0.380.6009
0.400.6391
0.420.6311
0.450.6200
0.470.6200
0.490.6187
0.490.6022
Validation loss
0.070.7438
0.350.6291
0.420.6186
Open technical sheet
CALLAII® by CALLMENTA

CALLAII Edge Small Business

Language model

A compact CALLAII language model prepared to run on small businesses' own hardware.

CALLAII Edge Small Business v1 Live Released September 2026

CALLAII Edge Small Business is designed for businesses that want a compact CALLAII running on their own server. It automatically checks procedural requirements in legal questions and reads document images.

What improved in this version

  • First release
  • Procedural check in legal questions: performed in 67 out of 70 exam questions, none before training
  • Procedural check points improved from 9 to 57 across 70 questions
  • Text extraction and image description from document images were fully preserved after training

What it does

  • Legal: petition and contract drafting, procedural check on legal questions
  • Procedural check: competent and authorized court, conditions of action, and deadlines
  • Call center: call summary, customer response, and quality evaluation
  • Visual reading: understanding text and content in documents and screenshots

Limitations and responsible use

  • Generated text is a draft; petitions, contracts, and official documents must be approved by an authorized person
  • Must be used together with a legislation library for current laws, regulations, and rulings; the model's memory is not up to date
  • Audio recordings are transcribed with CALLAII ASR; this model is not used for speech recognition
  • Quality in languages other than Turkish has not been measured in as much detail as Turkish

Training and measurements

1Epoch
0,810 → 0,605Validation loss
In 67 out of 70 questionsProcedural check
Training data45,847 samples
Validation set200 samples never seen during training
Steps2,861 · 1 epoch · 16 samples per step
Training context4,096 tokens
Validation loss0.810 → 0.605
Exam70 procedural questions, same questions before and after training
Training dateSeptember 2026
Training and validation loss
Training loss Validation loss
0 1 2 3 0.2 0.4 0.6 0.8 1.0 Epoch Training loss Validation loss Training loss: 0.0 · 2.09 Training loss: 0.0 · 0.99 Training loss: 0.0 · 0.88 Training loss: 0.0 · 0.81 Training loss: 0.0 · 0.83 Training loss: 0.1 · 0.82 Training loss: 0.1 · 0.78 Training loss: 0.1 · 0.81 Training loss: 0.1 · 0.76 Training loss: 0.1 · 0.78 Training loss: 0.1 · 0.77 Training loss: 0.1 · 0.81 Training loss: 0.1 · 0.74 Training loss: 0.1 · 0.76 Training loss: 0.1 · 0.72 Training loss: 0.1 · 0.73 Training loss: 0.1 · 0.74 Training loss: 0.2 · 0.70 Training loss: 0.2 · 0.70 Training loss: 0.2 · 0.68 Training loss: 0.2 · 0.70 Training loss: 0.2 · 0.69 Training loss: 0.2 · 0.72 Training loss: 0.2 · 0.68 Training loss: 0.2 · 0.68 Training loss: 0.2 · 0.68 Training loss: 0.2 · 0.67 Training loss: 0.2 · 0.68 Training loss: 0.3 · 0.68 Training loss: 0.3 · 0.69 Training loss: 0.3 · 0.66 Training loss: 0.3 · 0.69 Training loss: 0.3 · 0.65 Training loss: 0.3 · 0.65 Training loss: 0.3 · 0.68 Training loss: 0.3 · 0.67 Training loss: 0.3 · 0.66 Training loss: 0.3 · 0.64 Training loss: 0.3 · 0.66 Training loss: 0.3 · 0.66 Training loss: 0.4 · 0.65 Training loss: 0.4 · 0.63 Training loss: 0.4 · 0.71 Training loss: 0.4 · 0.65 Training loss: 0.4 · 0.65 Training loss: 0.4 · 0.63 Training loss: 0.4 · 0.65 Training loss: 0.4 · 0.63 Training loss: 0.4 · 0.62 Training loss: 0.4 · 0.67 Training loss: 0.4 · 0.63 Training loss: 0.5 · 0.64 Training loss: 0.5 · 0.65 Training loss: 0.5 · 0.63 Training loss: 0.5 · 0.62 Training loss: 0.5 · 0.64 Training loss: 0.5 · 0.66 Training loss: 0.5 · 0.63 Training loss: 0.5 · 0.63 Training loss: 0.5 · 0.63 Training loss: 0.5 · 0.62 Training loss: 0.5 · 0.64 Training loss: 0.6 · 0.60 Training loss: 0.6 · 0.65 Training loss: 0.6 · 0.63 Training loss: 0.6 · 0.62 Training loss: 0.6 · 0.61 Training loss: 0.6 · 0.63 Training loss: 0.6 · 0.64 Training loss: 0.6 · 0.62 Training loss: 0.6 · 0.64 Training loss: 0.6 · 0.63 Training loss: 0.6 · 0.63 Training loss: 0.6 · 0.64 Training loss: 0.7 · 0.62 Training loss: 0.7 · 0.60 Training loss: 0.7 · 0.62 Training loss: 0.7 · 0.60 Training loss: 0.7 · 0.61 Training loss: 0.7 · 0.62 Training loss: 0.7 · 0.61 Training loss: 0.7 · 0.61 Training loss: 0.7 · 0.61 Training loss: 0.7 · 0.61 Training loss: 0.7 · 0.63 Training loss: 0.8 · 0.60 Training loss: 0.8 · 0.59 Training loss: 0.8 · 0.60 Training loss: 0.8 · 0.59 Training loss: 0.8 · 0.62 Training loss: 0.8 · 0.63 Training loss: 0.8 · 0.59 Training loss: 0.8 · 0.62 Training loss: 0.8 · 0.59 Training loss: 0.8 · 0.63 Training loss: 0.8 · 0.60 Training loss: 0.8 · 0.58 Training loss: 0.9 · 0.59 Training loss: 0.9 · 0.59 Training loss: 0.9 · 0.58 Training loss: 0.9 · 0.60 Training loss: 0.9 · 0.59 Training loss: 0.9 · 0.63 Training loss: 0.9 · 0.61 Training loss: 0.9 · 0.60 Training loss: 0.9 · 0.60 Training loss: 0.9 · 0.67 Training loss: 0.9 · 0.58 Training loss: 1.0 · 0.59 Training loss: 1.0 · 0.58 Training loss: 1.0 · 0.63 Training loss: 1.0 · 0.60 Training loss: 1.0 · 0.59 Training loss: 1.0 · 0.61 Training loss: 1.0 · 0.59 Validation loss: 0.1 · 0.78 Validation loss: 0.1 · 0.78 Validation loss: 0.2 · 0.70 Validation loss: 0.2 · 0.70 Validation loss: 0.3 · 0.68 Validation loss: 0.3 · 0.68 Validation loss: 0.3 · 0.66 Validation loss: 0.3 · 0.66 Validation loss: 0.4 · 0.65 Validation loss: 0.4 · 0.65 Validation loss: 0.5 · 0.65 Validation loss: 0.5 · 0.65 Validation loss: 0.6 · 0.64 Validation loss: 0.6 · 0.64 Validation loss: 0.6 · 0.64 Validation loss: 0.6 · 0.64 Validation loss: 0.7 · 0.62 Validation loss: 0.7 · 0.62 Validation loss: 0.8 · 0.61 Validation loss: 0.8 · 0.61 Validation loss: 0.8 · 0.61 Validation loss: 0.8 · 0.61 Validation loss: 0.9 · 0.61 Validation loss: 0.9 · 0.61 Validation loss: 1.0 · 0.60 Validation loss: 1.0 · 0.60 Validation loss: 1.0 · 0.61 Validation loss: 1.0 · 0.61

Each point is the average of 5 consecutive records.

Show values as a table
Training loss
0.012.0882
0.050.8227
0.100.7656
0.140.7346
0.180.6991
0.230.6836
0.270.6616
0.310.6667
0.360.6545
0.400.6291
0.450.6276
0.490.6401
0.530.6182
0.580.6186
0.620.6388
0.660.5980
0.710.6144
0.750.6025
0.800.6273
0.840.6025
0.880.6050
0.930.5961
0.970.6281
1.000.5878
Validation loss
0.140.7797
0.210.7025
0.280.6809
0.350.6608
0.420.6528
0.490.6549
0.560.6427
0.630.6439
0.700.6169
0.770.6117
0.840.6079
0.910.6055
0.980.6049
10.6050
Pre- and post-training exam
Before training After training
0% 25% 50% 75% 100% Court and juri… Procedural poi… Court and jurisdiction · Before training: 0.0% 0.0% Court and jurisdiction · After training: 95.7% 95.7% Procedural points · Before training: 12.9% 12.9% Procedural points · After training: 81.4% 81.4%

70 procedural questions, same questions before and after training.

Show values as a table
Before trainingAfter training
Court and jurisdiction0.0095.70
Procedural points12.9081.40

Technical sheet

Versionv1
ReleaseSeptember 2026
InputText and image
OutputText
LanguageTurkish-focused; responds in the same language for other languages
Runs onthe business's own server
Open technical sheet
CALLAII® by CALLMENTA

CALLAII ASR

Speech recognition

A speech recognition model that separates call recordings into speaker channels and converts them into text. Analysis, coaching, and quality scoring operate on this transcription.

CALLAII ASR v4 Live Released September 2026

Separates call recordings into speaker channels and converts them to text. This version was trained on ten languages and phone-line-simulated audio; error rates in call center conversations dropped significantly.

What improved in this version

  • Turkish word error rate in call center scenarios dropped from 25.70% to 6.32%
  • Error rates in Arabic, German, French, English, and Italian fell to one-quarter and one-fifth; per-language results shown in the chart
  • Improved on read Turkish speech as well: 4.72% → 4.47% (800 recordings)
  • Trained on phone-line, noisy-environment, and studio versions of the same text
  • Previous version's data included in training; new data added without degrading learned knowledge

What it does

  • Transcribes customer and agent separately in two-channel recordings
  • Detects each channel's language independently
  • Produces timestamped, speaker-labeled transcripts
  • Leaves silent sections blank; trained not to generate hallucinated sentences

Limitations and responsible use

  • Measurements were made on test sets; reference-based error rate on real phone calls has not yet been measured
  • Error increases with overlapping speech and very low-quality recordings
  • Real-time transcription latency was not measured

Training and measurements

19.369Step
1Epoch
0,0981Final validation loss
%9,41Final word error rate
%2,82Final character error rate
Training data2,479,180 recordings, 3,654 hours of audio
New data share26.5%; remainder from previous version
LanguagesTurkish, Arabic, German, French, Kazakh, Italian, English, Bashkir, Uzbek, Tatar
Audio typesphone-line simulation, noisy environment, studio
Validation set11,859 recordings
Steps19,369 · 1 epoch
Training dateSeptember 2026
Training loss
0 5 10 15 5,000 10,000 15,000 Step Training loss Training loss: 175 · 10.81 Training loss: 350 · 8.66 Training loss: 525 · 7.60 Training loss: 700 · 7.07 Training loss: 875 · 6.61 Training loss: 1,050 · 6.46 Training loss: 1,225 · 6.17 Training loss: 1,400 · 6.18 Training loss: 1,575 · 5.93 Training loss: 1,750 · 5.77 Training loss: 1,925 · 5.65 Training loss: 2,100 · 5.73 Training loss: 2,275 · 5.54 Training loss: 2,450 · 5.41 Training loss: 2,625 · 5.55 Training loss: 2,800 · 5.33 Training loss: 2,975 · 5.34 Training loss: 3,150 · 5.20 Training loss: 3,325 · 5.12 Training loss: 3,500 · 5.12 Training loss: 3,675 · 5.02 Training loss: 3,850 · 4.99 Training loss: 4,025 · 4.94 Training loss: 4,200 · 4.99 Training loss: 4,375 · 4.91 Training loss: 4,550 · 4.74 Training loss: 4,725 · 4.77 Training loss: 4,900 · 4.85 Training loss: 5,075 · 4.82 Training loss: 5,250 · 4.71 Training loss: 5,425 · 4.61 Training loss: 5,600 · 4.58 Training loss: 5,775 · 4.61 Training loss: 5,950 · 4.55 Training loss: 6,125 · 4.63 Training loss: 6,300 · 4.61 Training loss: 6,475 · 4.57 Training loss: 6,650 · 4.54 Training loss: 6,825 · 4.53 Training loss: 7,000 · 4.50 Training loss: 7,175 · 4.50 Training loss: 7,350 · 4.46 Training loss: 7,525 · 4.53 Training loss: 7,700 · 4.44 Training loss: 7,875 · 4.41 Training loss: 8,050 · 4.39 Training loss: 8,225 · 4.45 Training loss: 8,400 · 4.41 Training loss: 8,575 · 4.40 Training loss: 8,750 · 4.43 Training loss: 8,925 · 4.44 Training loss: 9,100 · 4.28 Training loss: 9,275 · 4.34 Training loss: 9,450 · 4.24 Training loss: 9,625 · 4.20 Training loss: 9,800 · 4.34 Training loss: 9,975 · 4.30 Training loss: 10,150 · 4.28 Training loss: 10,325 · 4.29 Training loss: 10,500 · 4.22 Training loss: 10,675 · 4.23 Training loss: 10,850 · 4.21 Training loss: 11,025 · 4.26 Training loss: 11,200 · 4.28 Training loss: 11,375 · 4.23 Training loss: 11,550 · 4.18 Training loss: 11,725 · 4.15 Training loss: 11,900 · 4.15 Training loss: 12,075 · 4.10 Training loss: 12,250 · 4.16 Training loss: 12,425 · 4.28 Training loss: 12,600 · 4.22 Training loss: 12,775 · 4.27 Training loss: 12,950 · 4.21 Training loss: 13,125 · 4.19 Training loss: 13,300 · 4.16 Training loss: 13,475 · 4.10 Training loss: 13,650 · 4.15 Training loss: 13,825 · 4.20 Training loss: 14,000 · 4.12 Training loss: 14,175 · 4.15 Training loss: 14,350 · 4.16 Training loss: 14,525 · 4.14 Training loss: 14,700 · 4.13 Training loss: 14,875 · 4.12 Training loss: 15,050 · 4.17 Training loss: 15,225 · 3.98 Training loss: 15,400 · 4.18 Training loss: 15,575 · 4.07 Training loss: 15,750 · 4.17 Training loss: 15,925 · 4.08 Training loss: 16,100 · 4.06 Training loss: 16,275 · 4.12 Training loss: 16,450 · 4.15 Training loss: 16,625 · 4.07 Training loss: 16,800 · 4.00 Training loss: 16,975 · 4.13 Training loss: 17,150 · 4.04 Training loss: 17,325 · 4.11 Training loss: 17,500 · 4.15 Training loss: 17,675 · 4.05 Training loss: 17,850 · 4.04 Training loss: 18,025 · 4.13 Training loss: 18,200 · 4.06 Training loss: 18,375 · 4.10 Training loss: 18,550 · 4.20 Training loss: 18,725 · 3.94 Training loss: 18,900 · 4.06 Training loss: 19,075 · 4.09 Training loss: 19,250 · 4.07 Training loss: 19,350 · 4.02

Each point is the average of 7 consecutive recordings.

Show values as a table
Training loss
17510.8073
1,0506.4649
1,9255.6508
2,8005.3350
3,6755.0206
4,5504.7431
5,4254.6104
6,3004.6143
7,1754.5038
8,0504.3882
8,9254.4358
9,8004.3362
10,6754.2276
11,5504.1843
12,4254.2788
13,3004.1626
14,1754.1483
15,0504.1727
15,9254.0753
16,8004.0031
17,6754.0504
18,5504.2040
19,3504.0197
Validation loss
0.00 0.05 0.10 0.15 0.20 5,000 10,000 15,000 Step Validation loss Validation loss: 1,000 · 0.19 Validation loss: 1,000 · 0.19 Validation loss: 2,000 · 0.16 Validation loss: 2,000 · 0.16 Validation loss: 3,000 · 0.14 Validation loss: 3,000 · 0.14 Validation loss: 4,000 · 0.13 Validation loss: 4,000 · 0.13 Validation loss: 5,000 · 0.13 Validation loss: 5,000 · 0.13 Validation loss: 6,000 · 0.12 Validation loss: 6,000 · 0.12 Validation loss: 7,000 · 0.11 Validation loss: 7,000 · 0.11 Validation loss: 8,000 · 0.11 Validation loss: 8,000 · 0.11 Validation loss: 9,000 · 0.11 Validation loss: 9,000 · 0.11 Validation loss: 10,000 · 0.10 Validation loss: 10,000 · 0.10 Validation loss: 11,000 · 0.10 Validation loss: 11,000 · 0.10 Validation loss: 12,000 · 0.10 Validation loss: 12,000 · 0.10 Validation loss: 13,000 · 0.10 Validation loss: 13,000 · 0.10 Validation loss: 14,000 · 0.10 Validation loss: 14,000 · 0.10 Validation loss: 15,000 · 0.10 Validation loss: 15,000 · 0.10 Validation loss: 16,000 · 0.10 Validation loss: 16,000 · 0.10 Validation loss: 17,000 · 0.10 Validation loss: 17,000 · 0.10 Validation loss: 18,000 · 0.10 Validation loss: 18,000 · 0.10 Validation loss: 19,000 · 0.10 Validation loss: 19,000 · 0.10 Validation loss: 19,369 · 0.10 Validation loss: 19,369 · 0.10

Measured on examples never seen in training; a falling curve shows the model learns rather than memorises.

Show values as a table
Validation loss
1,0000.1911
2,0000.1611
3,0000.1446
4,0000.1345
5,0000.1259
6,0000.1187
7,0000.1135
8,0000.1097
9,0000.1068
10,0000.1050
11,0000.1025
12,0000.1021
13,0000.1010
14,0000.0997
15,0000.0994
16,0000.0985
17,0000.0985
18,0000.0984
19,0000.0980
19,3690.0981
Error rate on validation (%)
Word error rate (WER) Character error rate (CER)
0% 5% 10% 15% 20% 5,000 10,000 15,000 Step Word error rate (WER) Character error rate (CER) Word error rate (WER): 1,000 · 15.03% Word error rate (WER): 1,000 · 15.03% Word error rate (WER): 2,000 · 13.22% Word error rate (WER): 2,000 · 13.22% Word error rate (WER): 3,000 · 12.06% Word error rate (WER): 3,000 · 12.06% Word error rate (WER): 4,000 · 10.78% Word error rate (WER): 4,000 · 10.78% Word error rate (WER): 5,000 · 10.51% Word error rate (WER): 5,000 · 10.51% Word error rate (WER): 6,000 · 10.01% Word error rate (WER): 6,000 · 10.01% Word error rate (WER): 7,000 · 9.60% Word error rate (WER): 7,000 · 9.60% Word error rate (WER): 8,000 · 9.72% Word error rate (WER): 8,000 · 9.72% Word error rate (WER): 9,000 · 9.52% Word error rate (WER): 9,000 · 9.52% Word error rate (WER): 10,000 · 9.43% Word error rate (WER): 10,000 · 9.43% Word error rate (WER): 11,000 · 9.36% Word error rate (WER): 11,000 · 9.36% Word error rate (WER): 12,000 · 9.35% Word error rate (WER): 12,000 · 9.35% Word error rate (WER): 13,000 · 9.38% Word error rate (WER): 13,000 · 9.38% Word error rate (WER): 14,000 · 9.52% Word error rate (WER): 14,000 · 9.52% Word error rate (WER): 15,000 · 9.23% Word error rate (WER): 15,000 · 9.23% Word error rate (WER): 16,000 · 9.36% Word error rate (WER): 16,000 · 9.36% Word error rate (WER): 17,000 · 9.35% Word error rate (WER): 17,000 · 9.35% Word error rate (WER): 18,000 · 9.36% Word error rate (WER): 18,000 · 9.36% Word error rate (WER): 19,000 · 9.24% Word error rate (WER): 19,000 · 9.24% Word error rate (WER): 19,369 · 9.41% Word error rate (WER): 19,369 · 9.41% Character error rate (CER): 1,000 · 4.24% Character error rate (CER): 1,000 · 4.24% Character error rate (CER): 2,000 · 3.85% Character error rate (CER): 2,000 · 3.85% Character error rate (CER): 3,000 · 3.47% Character error rate (CER): 3,000 · 3.47% Character error rate (CER): 4,000 · 3.13% Character error rate (CER): 4,000 · 3.13% Character error rate (CER): 5,000 · 3.01% Character error rate (CER): 5,000 · 3.01% Character error rate (CER): 6,000 · 2.94% Character error rate (CER): 6,000 · 2.94% Character error rate (CER): 7,000 · 2.87% Character error rate (CER): 7,000 · 2.87% Character error rate (CER): 8,000 · 2.89% Character error rate (CER): 8,000 · 2.89% Character error rate (CER): 9,000 · 2.89% Character error rate (CER): 9,000 · 2.89% Character error rate (CER): 10,000 · 2.85% Character error rate (CER): 10,000 · 2.85% Character error rate (CER): 11,000 · 2.88% Character error rate (CER): 11,000 · 2.88% Character error rate (CER): 12,000 · 2.81% Character error rate (CER): 12,000 · 2.81% Character error rate (CER): 13,000 · 2.92% Character error rate (CER): 13,000 · 2.92% Character error rate (CER): 14,000 · 2.93% Character error rate (CER): 14,000 · 2.93% Character error rate (CER): 15,000 · 2.80% Character error rate (CER): 15,000 · 2.80% Character error rate (CER): 16,000 · 2.83% Character error rate (CER): 16,000 · 2.83% Character error rate (CER): 17,000 · 2.77% Character error rate (CER): 17,000 · 2.77% Character error rate (CER): 18,000 · 2.80% Character error rate (CER): 18,000 · 2.80% Character error rate (CER): 19,000 · 2.72% Character error rate (CER): 19,000 · 2.72% Character error rate (CER): 19,369 · 2.82% Character error rate (CER): 19,369 · 2.82%

Lower is better. Measured on the validation set throughout training.

Show values as a table
Word error rate (WER)
1,00015.03
2,00013.22
3,00012.06
4,00010.78
5,00010.51
6,00010.01
7,0009.60
8,0009.72
9,0009.52
10,0009.43
11,0009.36
12,0009.35
13,0009.38
14,0009.52
15,0009.23
16,0009.36
17,0009.35
18,0009.36
19,0009.24
19,3699.41
Character error rate (CER)
1,0004.24
2,0003.85
3,0003.47
4,0003.13
5,0003.01
6,0002.94
7,0002.87
8,0002.89
9,0002.89
10,0002.85
11,0002.88
12,0002.81
13,0002.92
14,0002.93
15,0002.80
16,0002.83
17,0002.77
18,0002.80
19,0002.72
19,3692.82
Call center evaluation · Per-language word error rate (%)
v3 v4
0% 20% 40% 60% 80% Arabic German English French Italian Turkish Arabic · v3: 67.6% 67.6% Arabic · v4: 13.9% 13.9% German · v3: 54.6% 54.6% German · v4: 15.4% 15.4% English · v3: 59.9% 59.9% English · v4: 12.6% 12.6% French · v3: 62.4% 62.4% French · v4: 15.6% 15.6% Italian · v3: 38.9% 38.9% Italian · v4: 8.4% 8.4% Turkish · v3: 25.7% 25.7% Turkish · v4: 6.3% 6.3%

Lower is better. 300 samples per language.

Show values as a table
v3v4
Arabic67.5913.86
German54.5815.37
English59.9212.58
French62.3615.57
Italian38.888.42
Turkish25.706.32
Call center evaluation · Per-language character error rate (%)
v3 v4
0% 20% 40% 60% Arabic German English French Italian Turkish Arabic · v3: 48.6% 48.6% Arabic · v4: 5.8% 5.8% German · v3: 46.0% 46.0% German · v4: 7.2% 7.2% English · v3: 44.4% 44.4% English · v4: 6.2% 6.2% French · v3: 47.9% 47.9% French · v4: 7.3% 7.3% Italian · v3: 26.9% 26.9% Italian · v4: 3.5% 3.5% Turkish · v3: 10.5% 10.5% Turkish · v4: 2.0% 2.0%

Lower is better. 300 samples per language.

Show values as a table
v3v4
Arabic48.625.76
German46.007.23
English44.416.23
French47.927.34
Italian26.923.45
Turkish10.471.96
Read Turkish speech · Per-language word error rate (%)
v3 v4
0% 2% 4% 6% Turkish Turkish · v3: 4.7% 4.7% Turkish · v4: 4.5% 4.5%

Lower is better. 800 samples per language.

Show values as a table
v3v4
Turkish4.724.47
Read Turkish speech · Per-language character error rate (%)
v3 v4
0.0% 0.5% 1.0% 1.5% Turkish Turkish · v3: 1.1% 1.1% Turkish · v4: 1.0% 1.0%

Lower is better. 800 samples per language.

Show values as a table
v3v4
Turkish1.110.95

Technical sheet

Versionv4
Release8 September 2026
InputAudio recording, single or dual channel
OutputSpeaker-labeled, timestamped text
Language detectionAutomatic per channel
Runs onServers managed by Callmenta
Open technical sheet

Previous versions

CALLAII ASR v3 · Previous version
CALLAII ASR v3 Previous version Released August 2026

Version with extensive multilingual training. Phone line simulation and silence detection were added in this version.

What improved in this version

  • Turkish word error rate dropped from 8.87% to 7.11% on the open benchmark set
  • Arabic dropped from 7.96% to 7.82%
  • Learned to leave silent sections blank; hallucinated sentences decreased
  • No degradation observed at the end of long calls (571-second real call)
  • Automatic language detection per channel has begun

Limitations and responsible use

  • The open benchmark set consists of read news sentences; it does not represent absolute success in phone calls
  • Error rates were high in languages other than Turkish in call center scenarios; resolved in v4

Training and measurements

11.540Step
1Epoch
0,1315Final validation loss
%10,49Final word error rate
%3,01Final character error rate
Training data1,477,006 records, 2,190.9 hours of audio
Silence samples25.7 hours
Audio typesphone band simulation (8 kHz), cross-mixing, dialogue segmentation
Steps11,540 · 1 epoch
Training dateAugust 2026
Training loss
0 1 2 3 2,500 5,000 7,500 10,000 Step Training loss Training loss: 100 · 2.63 Training loss: 200 · 1.90 Training loss: 300 · 1.80 Training loss: 400 · 1.82 Training loss: 500 · 1.72 Training loss: 600 · 1.67 Training loss: 700 · 1.67 Training loss: 800 · 1.68 Training loss: 900 · 1.67 Training loss: 1,000 · 1.66 Training loss: 1,100 · 1.60 Training loss: 1,200 · 1.57 Training loss: 1,300 · 1.54 Training loss: 1,400 · 1.56 Training loss: 1,500 · 1.56 Training loss: 1,600 · 1.56 Training loss: 1,700 · 1.53 Training loss: 1,800 · 1.50 Training loss: 1,900 · 1.52 Training loss: 2,000 · 1.51 Training loss: 2,100 · 1.52 Training loss: 2,200 · 1.47 Training loss: 2,300 · 1.48 Training loss: 2,400 · 1.45 Training loss: 2,500 · 1.44 Training loss: 2,600 · 1.43 Training loss: 2,700 · 1.45 Training loss: 2,800 · 1.44 Training loss: 2,900 · 1.44 Training loss: 3,000 · 1.42 Training loss: 3,100 · 1.41 Training loss: 3,200 · 1.46 Training loss: 3,300 · 1.37 Training loss: 3,400 · 1.41 Training loss: 3,500 · 1.46 Training loss: 3,600 · 1.35 Training loss: 3,700 · 1.37 Training loss: 3,800 · 1.35 Training loss: 3,900 · 1.39 Training loss: 4,000 · 1.40 Training loss: 4,100 · 1.38 Training loss: 4,200 · 1.38 Training loss: 4,300 · 1.36 Training loss: 4,400 · 1.38 Training loss: 4,500 · 1.34 Training loss: 4,600 · 1.35 Training loss: 4,700 · 1.36 Training loss: 4,800 · 1.31 Training loss: 4,900 · 1.30 Training loss: 5,000 · 1.34 Training loss: 5,100 · 1.33 Training loss: 5,200 · 1.31 Training loss: 5,300 · 1.30 Training loss: 5,400 · 1.31 Training loss: 5,500 · 1.33 Training loss: 5,600 · 1.33 Training loss: 5,700 · 1.27 Training loss: 5,800 · 1.32 Training loss: 5,900 · 1.33 Training loss: 6,000 · 1.31 Training loss: 6,100 · 1.30 Training loss: 6,200 · 1.27 Training loss: 6,300 · 1.26 Training loss: 6,400 · 1.33 Training loss: 6,500 · 1.29 Training loss: 6,600 · 1.29 Training loss: 6,700 · 1.31 Training loss: 6,800 · 1.22 Training loss: 6,900 · 1.25 Training loss: 7,000 · 1.31 Training loss: 7,100 · 1.27 Training loss: 7,200 · 1.23 Training loss: 7,300 · 1.26 Training loss: 7,400 · 1.27 Training loss: 7,500 · 1.25 Training loss: 7,600 · 1.26 Training loss: 7,700 · 1.24 Training loss: 7,800 · 1.25 Training loss: 7,900 · 1.24 Training loss: 8,000 · 1.24 Training loss: 8,100 · 1.26 Training loss: 8,200 · 1.23 Training loss: 8,300 · 1.24 Training loss: 8,400 · 1.24 Training loss: 8,500 · 1.23 Training loss: 8,600 · 1.27 Training loss: 8,700 · 1.23 Training loss: 8,800 · 1.22 Training loss: 8,900 · 1.20 Training loss: 9,000 · 1.22 Training loss: 9,100 · 1.22 Training loss: 9,200 · 1.22 Training loss: 9,300 · 1.24 Training loss: 9,400 · 1.23 Training loss: 9,500 · 1.21 Training loss: 9,600 · 1.23 Training loss: 9,700 · 1.18 Training loss: 9,800 · 1.23 Training loss: 9,900 · 1.21 Training loss: 10,000 · 1.18 Training loss: 10,100 · 1.22 Training loss: 10,200 · 1.21 Training loss: 10,300 · 1.26 Training loss: 10,400 · 1.20 Training loss: 10,500 · 1.20 Training loss: 10,600 · 1.21 Training loss: 10,700 · 1.21 Training loss: 10,800 · 1.23 Training loss: 10,900 · 1.17 Training loss: 11,000 · 1.19 Training loss: 11,100 · 1.18 Training loss: 11,200 · 1.18 Training loss: 11,300 · 1.20 Training loss: 11,400 · 1.20 Training loss: 11,500 · 1.17

Each point is the average of two consecutive records.

Show values as a table
Training loss
1002.6340
6001.6663
1,1001.6001
1,6001.5598
2,1001.5177
2,6001.4338
3,1001.4080
3,6001.3531
4,1001.3821
4,6001.3530
5,1001.3306
5,6001.3333
6,1001.3021
6,6001.2923
7,1001.2736
7,6001.2613
8,1001.2606
8,6001.2679
9,1001.2166
9,6001.2256
10,1001.2165
10,6001.2068
11,1001.1835
11,5001.1665
Validation loss
0.13 0.14 0.15 0.16 0.17 2,000 4,000 6,000 8,000 10,000 Step Validation loss Validation loss: 2,000 · 0.16 Validation loss: 2,000 · 0.16 Validation loss: 4,000 · 0.15 Validation loss: 4,000 · 0.15 Validation loss: 6,000 · 0.15 Validation loss: 6,000 · 0.15 Validation loss: 8,000 · 0.14 Validation loss: 8,000 · 0.14 Validation loss: 10,000 · 0.13 Validation loss: 10,000 · 0.13 Validation loss: 11,540 · 0.13 Validation loss: 11,540 · 0.13

Measured on examples never seen in training; a falling curve shows the model learns rather than memorises.

Show values as a table
Validation loss
2,0000.1633
4,0000.1528
6,0000.1456
8,0000.1383
10,0000.1340
11,5400.1316
Error rate on validation (%)
Word error rate (WER) Character error rate (CER)
0% 5% 10% 15% 20% 2,000 4,000 6,000 8,000 10,000 Step Word error rate (WER) Character error rate (CER) Word error rate (WER): 2,000 · 12.60% Word error rate (WER): 2,000 · 12.60% Word error rate (WER): 4,000 · 11.94% Word error rate (WER): 4,000 · 11.94% Word error rate (WER): 6,000 · 19.10% Word error rate (WER): 6,000 · 19.10% Word error rate (WER): 8,000 · 10.71% Word error rate (WER): 8,000 · 10.71% Word error rate (WER): 10,000 · 18.35% Word error rate (WER): 10,000 · 18.35% Word error rate (WER): 11,540 · 10.49% Word error rate (WER): 11,540 · 10.49% Character error rate (CER): 2,000 · 3.64% Character error rate (CER): 2,000 · 3.64% Character error rate (CER): 4,000 · 3.33% Character error rate (CER): 4,000 · 3.33% Character error rate (CER): 6,000 · 10.35% Character error rate (CER): 6,000 · 10.35% Character error rate (CER): 8,000 · 2.94% Character error rate (CER): 8,000 · 2.94% Character error rate (CER): 10,000 · 10.79% Character error rate (CER): 10,000 · 10.79% Character error rate (CER): 11,540 · 3.01% Character error rate (CER): 11,540 · 3.01%

Lower is better. Measured on the validation set throughout training.

Show values as a table
Word error rate (WER)
2,00012.60
4,00011.94
6,00019.10
8,00010.71
10,00018.35
11,54010.49
Character error rate (CER)
2,0003.64
4,0003.33
6,00010.35
8,0002.94
10,00010.79
11,5403.01
Open benchmark set (FLEURS) · Word error rate (%)
v2 v3
0.0% 2.5% 5.0% 7.5% 10.0% Turkish Arabic Turkish · v2: 8.9% 8.9% Turkish · v3: 7.1% 7.1% Arabic · v2: 8.0% 8.0% Arabic · v3: 7.8% 7.8%

Lower is better. 150 records per language, read news sentences. Valid for ranking; does not represent absolute success in phone calls.

Show values as a table
v2v3
Turkish8.877.11
Arabic7.967.82
Open benchmark set (FLEURS) · Character error rate (%)
v2 v3
0% 1% 2% 3% Turkish Arabic Turkish · v2: 2.8% 2.8% Turkish · v3: 1.7% 1.7% Arabic · v2: 2.7% 2.7% Arabic · v3: 2.5% 2.5%

Lower is better. 150 records per language.

Show values as a table
v2v3
Turkish2.831.67
Arabic2.652.46
Internal test set · Language-based word error rate (%)
0% 5% 10% 15% 20% Arabic German English French Italian Turkish Arabic · v3: 19.7% 19.7% German · v3: 6.8% 6.8% English · v3: 19.1% 19.1% French · v3: 12.7% 12.7% Italian · v3: 8.1% 8.1% Turkish · v3: 10.5% 10.5%

Lower is better. 200 samples per language.

Show values as a table
v3
Arabic19.72
German6.80
English19.10
French12.74
Italian8.11
Turkish10.49
Internal test set · Language-based character error rate (%)
0% 5% 10% 15% Arabic German English French Italian Turkish Arabic · v3: 7.4% 7.4% German · v3: 2.0% 2.0% English · v3: 11.9% 11.9% French · v3: 3.8% 3.8% Italian · v3: 2.2% 2.2% Turkish · v3: 3.0% 3.0%

Lower is better. 200 samples per language.

Show values as a table
v3
Arabic7.41
German1.95
English11.89
French3.75
Italian2.22
Turkish3.01

Technical sheet

Versionv3
ReleaseAugust 25, 2026
StatusSuperseded by v4
Open technical sheet
CALLAII ASR v2 · Previous version
CALLAII ASR v2 Previous version Released June 2026

Second version adapted to call center Turkish. The error rate on real recordings dropped to half of the first version.

What improved in this version

  • Word error rate on 10 real recordings dropped from %14.25 to %6.51
  • Character error rate dropped from %8.68 to %4.57
  • Domain prompt added for call center expressions (announcements and institution names)

Limitations and responsible use

  • Could enter a repetition loop at the end of long calls; fixed in v3
  • Output was corrupted when the language label was incorrectly assigned in Arabic recordings; fixed in v3 with automatic language detection

Training and measurements

Turkish validationword error rate %7.02, character error rate %2.15
Error rate (%) on 10 real recordings
v1 v2
0% 5% 10% 15% Word error rat… Character erro… Word error rate (WER) · v1: 14.3% 14.3% Word error rate (WER) · v2: 6.5% 6.5% Character error rate (CER) · v1: 8.7% 8.7% Character error rate (CER) · v2: 4.6% 4.6%

Lower is better. The same 10 real call recordings were fed to both versions.

Show values as a table
v1v2
Word error rate (WER)14.256.51
Character error rate (CER)8.684.57

Technical sheet

Versionv2
ReleaseJune 2026
StatusReplaced by v3
Open technical sheet
CALLAII ASR v1 · Previous version
CALLAII ASR v1 Previous version Released April 2026

First CALLAII ASR version: The initial model that converts Turkish call recordings into text.

Technical sheet

Versionv1
ReleaseApril 2026
StatusReplaced by v2
Open technical sheet

Product release notes