Skip to content
Callmenta
CALLAII models

CALLAII® Versions and improvements

Every version of the CALLAII models that power the AI in Callmenta: what it does, what improved in this version and where its limits are.

Customer data is never used to train the models.

CALLAII® by CALLMENTA

CALLAII

Language model

The language model powering the AI surfaces in the Callmenta panel: call analysis, coaching, live support, CRM, and CALLAII Legal.

CALLAII v8 Live Released October 2026

CALLAII's eighth version and new primary model. It scores calls based on the company's own criteria and writes summaries using only what was discussed. It prepares legal texts strictly adhering to library sources and procedure. Training was completed on October 15, 2026. CALLAII v8 passed the same exams as v7, results are below.

What improved in this version

  • In call analysis, it identifies the request type with 81.8% accuracy across 40 real calls, up from 22.7% in v7
  • It produces valid output in all 534 analyses
  • Even with thinking mode off, it starts legal answers with a procedure check: 61 out of 70 questions, compared to 22 in v7
  • Critical citation and quote errors in legal documents dropped from 8 to 4 across 70 documents
  • Scored 90 on map questions and 95 on shape counting
  • Route plan evaluation on energy screens improved from 20 to 93.3

Limitations and responsible use

  • Accuracy in resolution status remains limited: 76 correct out of 123 questions in targeted calls. This area should remain as a suggestion on screen.
  • The quality of legal documents was not scored by a judge in this round. Every document must undergo lawyer review.
  • Legal knowledge does not come from the model's memory, sources are fetched from the current library each time.
  • Numbers and dates must be verified.

Training and measurements

3,842Step
1Epoch
0.2929Final validation loss
Training data61,461 samples, 8,000 of which are visual
Validation set186 samples
Step3,842 · 1 epoch · 16 samples per step
Training context32,768 tokens
Training dateOctober 2026
Data for each screen was generated by the live system's own prompts. The model is trained on the prompts it sees
Petitions, formal notices, and legal chats draw sources from the library. Procedure checks rely solely on the provided articles
Call scoring was taught using examples where the missing behavior was known in advance. In real calls, the median of three separate evaluations was used, and classification fields were corrected based on human labels
Not sharing internal rules, speaking in the company's own name in every language, and not inventing unknown products were additionally taught
Sales, cross-selling, and competency scenarios were verified against known truths
Real calls reserved for exams were excluded from training
Training and validation loss
Training loss Validation loss
0.0 0.2 0.4 0.6 0.8 1,000 2,000 3,000 Step Training loss Validation loss Training loss: 35 · 0.70 Training loss: 70 · 0.66 Training loss: 105 · 0.58 Training loss: 140 · 0.55 Training loss: 175 · 0.58 Training loss: 210 · 0.55 Training loss: 245 · 0.58 Training loss: 280 · 0.56 Training loss: 315 · 0.55 Training loss: 350 · 0.55 Training loss: 385 · 0.53 Training loss: 420 · 0.52 Training loss: 455 · 0.52 Training loss: 490 · 0.52 Training loss: 525 · 0.52 Training loss: 560 · 0.49 Training loss: 595 · 0.53 Training loss: 630 · 0.51 Training loss: 665 · 0.52 Training loss: 700 · 0.51 Training loss: 735 · 0.52 Training loss: 770 · 0.50 Training loss: 805 · 0.52 Training loss: 840 · 0.50 Training loss: 875 · 0.49 Training loss: 910 · 0.52 Training loss: 945 · 0.49 Training loss: 980 · 0.50 Training loss: 1,015 · 0.51 Training loss: 1,050 · 0.48 Training loss: 1,085 · 0.50 Training loss: 1,120 · 0.48 Training loss: 1,155 · 0.49 Training loss: 1,190 · 0.49 Training loss: 1,225 · 0.48 Training loss: 1,260 · 0.49 Training loss: 1,295 · 0.49 Training loss: 1,330 · 0.49 Training loss: 1,365 · 0.48 Training loss: 1,400 · 0.46 Training loss: 1,435 · 0.49 Training loss: 1,470 · 0.50 Training loss: 1,505 · 0.48 Training loss: 1,540 · 0.49 Training loss: 1,575 · 0.50 Training loss: 1,610 · 0.48 Training loss: 1,645 · 0.47 Training loss: 1,680 · 0.47 Training loss: 1,715 · 0.46 Training loss: 1,750 · 0.48 Training loss: 1,785 · 0.50 Training loss: 1,820 · 0.50 Training loss: 1,855 · 0.49 Training loss: 1,890 · 0.48 Training loss: 1,925 · 0.45 Training loss: 1,960 · 0.45 Training loss: 1,995 · 0.45 Training loss: 2,030 · 0.46 Training loss: 2,065 · 0.44 Training loss: 2,100 · 0.46 Training loss: 2,135 · 0.47 Training loss: 2,170 · 0.45 Training loss: 2,205 · 0.46 Training loss: 2,240 · 0.46 Training loss: 2,275 · 0.48 Training loss: 2,310 · 0.48 Training loss: 2,345 · 0.46 Training loss: 2,380 · 0.48 Training loss: 2,415 · 0.47 Training loss: 2,450 · 0.46 Training loss: 2,485 · 0.48 Training loss: 2,520 · 0.46 Training loss: 2,555 · 0.45 Training loss: 2,590 · 0.45 Training loss: 2,625 · 0.44 Training loss: 2,660 · 0.46 Training loss: 2,695 · 0.46 Training loss: 2,730 · 0.45 Training loss: 2,765 · 0.44 Training loss: 2,800 · 0.43 Training loss: 2,835 · 0.46 Training loss: 2,870 · 0.44 Training loss: 2,905 · 0.43 Training loss: 2,940 · 0.46 Training loss: 2,975 · 0.43 Training loss: 3,010 · 0.45 Training loss: 3,045 · 0.44 Training loss: 3,080 · 0.43 Training loss: 3,115 · 0.43 Training loss: 3,150 · 0.44 Training loss: 3,185 · 0.44 Training loss: 3,220 · 0.45 Training loss: 3,255 · 0.45 Training loss: 3,290 · 0.44 Training loss: 3,325 · 0.47 Training loss: 3,360 · 0.44 Training loss: 3,395 · 0.46 Training loss: 3,430 · 0.45 Training loss: 3,465 · 0.41 Training loss: 3,500 · 0.43 Training loss: 3,535 · 0.43 Training loss: 3,570 · 0.46 Training loss: 3,605 · 0.46 Training loss: 3,640 · 0.45 Training loss: 3,675 · 0.45 Training loss: 3,710 · 0.46 Training loss: 3,745 · 0.44 Training loss: 3,780 · 0.42 Training loss: 3,815 · 0.46 Training loss: 3,840 · 0.42 Validation loss: 100 · 0.46 Validation loss: 200 · 0.42 Validation loss: 300 · 0.39 Validation loss: 400 · 0.38 Validation loss: 500 · 0.38 Validation loss: 600 · 0.38 Validation loss: 700 · 0.37 Validation loss: 800 · 0.36 Validation loss: 900 · 0.36 Validation loss: 1,000 · 0.35 Validation loss: 1,100 · 0.34 Validation loss: 1,200 · 0.34 Validation loss: 1,300 · 0.34 Validation loss: 1,400 · 0.34 Validation loss: 1,500 · 0.33 Validation loss: 1,600 · 0.33 Validation loss: 1,700 · 0.33 Validation loss: 1,800 · 0.32 Validation loss: 1,900 · 0.32 Validation loss: 2,000 · 0.32 Validation loss: 2,100 · 0.32 Validation loss: 2,200 · 0.32 Validation loss: 2,300 · 0.31 Validation loss: 2,400 · 0.31 Validation loss: 2,500 · 0.31 Validation loss: 2,600 · 0.30 Validation loss: 2,700 · 0.30 Validation loss: 2,800 · 0.30 Validation loss: 2,900 · 0.30 Validation loss: 3,000 · 0.30 Validation loss: 3,100 · 0.30 Validation loss: 3,200 · 0.30 Validation loss: 3,300 · 0.29 Validation loss: 3,400 · 0.29 Validation loss: 3,500 · 0.29 Validation loss: 3,600 · 0.29 Validation loss: 3,700 · 0.29 Validation loss: 3,800 · 0.29 Validation loss: 3,842 · 0.29

Each point is the average of 7 consecutive records.

Show values as a table
Training loss
350.7007
2100.5514
3850.5294
5600.4948
7350.5226
9100.5209
1,0850.4995
1,2600.4866
1,4350.4916
1,6100.4829
1,7850.4963
1,9600.4516
2,1350.4679
2,3100.4778
2,4850.4750
2,6600.4592
2,8350.4596
3,0100.4480
3,1850.4427
3,3600.4400
3,5350.4327
3,7100.4643
3,8400.4225
Validation loss
1000.4600
3000.3948
5000.3773
7000.3684
9000.3574
1,1000.3421
1,3000.3386
1,5000.3330
1,7000.3276
1,9000.3223
2,1000.3152
2,3000.3126
2,5000.3068
2,7000.3018
2,9000.2982
3,1000.2957
3,3000.2947
3,5000.2932
3,7000.2930
3,8420.2929

Comparison with CALLAII v7

v8 and the previous live main model v7 entered the same exams under the same conditions. The measurement was conducted on the same exam server on October 15, 2026. Call analysis and thinking mode procedure exam were run three times, others once. Calls and prompts were identical for both models. Verification losses were measured with different sets, so they are not comparable between versions.

45,847 → 61,461 samplesTraining data
4,096 → 32,768 tokensTraining context
3 out of 534 → 0 out of 534Invalid analysis output
22.7% → 81.8%Request type (40 real calls)
73 → 76Resolution status (138 targeted calls, 123 questions)
22 → 61Starting with procedure check, without thinking (70 questions)
8 → 4Critical citation and quote errors in law (70 documents, live setting)
92.1 → 96.2 pointsAddress parsing
  • Invalid output in call analysis dropped from 3 to 0 out of 534 analyses.
  • Request type increased from 22.7% to 81.8% on 40 real calls with pre-determined correct answers. The biggest jump was in this area.
  • Resolution status increased from 54.5% to 68.2% on the same calls. Scorable call separation increased from 84.2% to 94.2%, and risk level from 77.3% to 81.8%.
  • Request type increased from 70.4% to 97.2% on 138 targeted calls. Scorable call separation increased from 64.7% to 78.4%, and risk level from 66.7% to 74.2%.
  • Resolution status on targeted calls barely moved. It increased from 73 to 76 on 123 questions. This area remains the weakest.
  • In legal procedure audit, both v7 and v8 started the answer with a procedure check on all 70 questions in thinking mode. In without thinking mode, this increased from 22 to 61.
  • Citation and quote errors deemed critical by the audit dropped from 8 to 4 on 70 documents with live setting. In without thinking run, it dropped from 8 to 3. Cut documents dropped from 1 to 0. The sample is small, and document quality was not separately scored in this round.
  • Score on map questions increased from 60 to 90, and shape counting from 80 to 95.
  • Route plan evaluation on energy screens increased from 20 to 93.3, and field task preparation from 53.3 to 90. Manager route remained the same at 93.3.
Call analysis: 40 real calls with pre-determined correct answers (%)
CALLAII v7 CALLAII v8
0% 25% 50% 75% 100% Scorable Risk level Resolution status Request type Scorable · CALLAII v7: 84.2% 84.2% Scorable · CALLAII v8: 94.2% 94.2% Risk level · CALLAII v7: 77.3% 77.3% Risk level · CALLAII v8: 81.8% 81.8% Resolution status · CALLAII v7: 54.5% 54.5% Resolution status · CALLAII v8: 68.2% 68.2% Request type · CALLAII v7: 22.7% 22.7% Request type · CALLAII v8: 81.8% 81.8%

Same calls and same prompt, sum of 3 runs. Analyses not producing valid output were counted as incorrect.

Show values as a table
CALLAII v7CALLAII v8
Scorable84.2094.20
Risk level77.3081.80
Resolution status54.5068.20
Request type22.7081.80
Call analysis: 138 targeted exam calls (%)
CALLAII v7 CALLAII v8
0% 25% 50% 75% 100% Scorable Risk level Resolution status Request type Scorable · CALLAII v7: 64.7% 64.7% Scorable · CALLAII v8: 78.4% 78.4% Risk level · CALLAII v7: 66.7% 66.7% Risk level · CALLAII v8: 74.2% 74.2% Resolution status · CALLAII v7: 59.3% 59.3% Resolution status · CALLAII v8: 61.8% 61.8% Request type · CALLAII v7: 70.4% 70.4% Request type · CALLAII v8: 97.2% 97.2%

Same calls and same prompt, sum of 3 runs. Analyses not producing valid output were counted as incorrect.

Show values as a table
CALLAII v7CALLAII v8
Scorable64.7078.40
Risk level66.7074.20
Resolution status59.3061.80
Request type70.4097.20
Legal procedure audit: 70 questions (%)
CALLAII v7 CALLAII v8
0% 25% 50% 75% 100% Start with procedu… Procedure points,… Start with procedu… Start with procedure check, thinking mode · CALLAII v7: 100.0% 100.0% Start with procedure check, thinking mode · CALLAII v8: 100.0% 100.0% Procedure points, thinking mode · CALLAII v7: 85.7% 85.7% Procedure points, thinking mode · CALLAII v8: 88.6% 88.6% Start with procedure check, without thinking · CALLAII v7: 31.4% 31.4% Start with procedure check, without thinking · CALLAII v8: 87.1% 87.1%

In thinking mode, the median of 3 runs. Without thinking, one run.

Show values as a table
CALLAII v7CALLAII v8
Start with procedure check, thinking mode100.00100.00
Procedure points, thinking mode85.7088.60
Start with procedure check, without thinking31.4087.10
Legal documents: critical citation and quote errors in 70 documents
CALLAII v7 CALLAII v8
0 belge 2 belge 4 belge 6 belge 8 belge Critical error, li… Critical error, wi… Truncated document… Critical error, live setting · CALLAII v7: 8.0 belge 8.0 belge Critical error, live setting · CALLAII v8: 4.0 belge 4.0 belge Critical error, without thinking · CALLAII v7: 8.0 belge 8.0 belge Critical error, without thinking · CALLAII v8: 3.0 belge 3.0 belge Truncated document, live setting · CALLAII v7: 1.0 belge 1.0 belge Truncated document, live setting · CALLAII v8: 0.0 belge 0.0 belge

Lower is better. Compares audit decision number, law article, and quote with the actual text. Document quality was not additionally scored in this round.

Show values as a table
CALLAII v7CALLAII v8
Critical error, live setting8.004.00
Critical error, without thinking8.003.00
Truncated document, live setting1.000.00
Image and map (%)
CALLAII v7 CALLAII v8
0% 25% 50% 75% 100% Map Other view Shape counting Map · CALLAII v7: 60.0% 60.0% Map · CALLAII v8: 90.0% 90.0% Other view · CALLAII v7: 82.3% 82.3% Other view · CALLAII v8: 100.0% 100.0% Shape counting · CALLAII v7: 80.0% 80.0% Shape counting · CALLAII v8: 95.0% 95.0%

Definitive answer questions, one run. Map 100 questions, other image 18-point question, shape counting 60 questions.

Show values as a table
CALLAII v7CALLAII v8
Map60.0090.00
Other view82.30100.00
Shape counting80.0095.00
Energy distribution screens and address parsing (score)
CALLAII v7 CALLAII v8
0 puan 25 puan 50 puan 75 puan 100 puan Route plan eva… Field assistan… Field task pre… Agent assistan… Manager route Address parsin… Route plan evaluation · CALLAII v7: 20.0 puan 20.0 puan Route plan evaluation · CALLAII v8: 93.3 puan 93.3 puan Field assistant · CALLAII v7: 96.7 puan 96.7 puan Field assistant · CALLAII v8: 100.0 puan 100.0 puan Field task preparation · CALLAII v7: 53.3 puan 53.3 puan Field task preparation · CALLAII v8: 90.0 puan 90.0 puan Agent assistant · CALLAII v7: 80.0 puan 80.0 puan Agent assistant · CALLAII v8: 96.7 puan 96.7 puan Manager route · CALLAII v7: 93.3 puan 93.3 puan Manager route · CALLAII v8: 93.3 puan 93.3 puan Address parsing · CALLAII v7: 92.1 puan 92.1 puan Address parsing · CALLAII v8: 96.2 puan 96.2 puan

30 tasks per screen, 38 addresses, single run.

Show values as a table
CALLAII v7CALLAII v8
Route plan evaluation20.0093.30
Field assistant96.70100.00
Field task preparation53.3090.00
Agent assistant80.0096.70
Manager route93.3093.30
Address parsing92.1096.20
Open technical sheet

Previous versions

CALLAII v7 · Previous version
CALLAII v7 Previous version Released September 2026

CALLAII v7 is live. The previous live version, v6, was put through the same exam under live settings. On legal questions, it checks procedure almost every time: competent and authorized court, conditions of the lawsuit, and deadlines. It works alongside an additional audit that catches and extracts information the lawyer did not mention in petitions.

What improved in this version

  • Procedure check in legal mode: performed in 69 out of 70 questions; live v6 did not perform it in any of the same exam.
  • Procedure check points in legal mode rose from 8 to 57 across 70 questions.
  • Procedure check in general mode with thinking off rose from 0 to 23.
  • Answers are more concise: answers cut off at the limit in general mode dropped from 40 to 11.
  • Text extraction from document images and image description were fully preserved after training.

Limitations and responsible use

  • Legal knowledge does not come from the model's memory; laws, regulations, and rulings are fetched from the current library every time.
  • The text it generates is a draft; petitions, contracts, and official documents must be approved by an authorized person.
  • Numbers and dates must be verified.
  • Quality in languages other than Turkish has not been measured in as much detail as Turkish.

Training and measurements

0.629 → 0.513Validation loss
69 out of 70 questionsProcedure check · legal
0 out of 70 questionsLive v6 on the same exam
8 → 57Procedure points · legal
Training data45,847 samples
Validation set200 samples never seen during training
Steps2,859 · 1 epoch · 16 samples per step
Training context4,096 tokens
Validation loss0.629 → 0.513
Exam70 procedure questions; same questions as live v6, live settings
Training dateSeptember 2026
Training and validation loss
Training loss Validation loss
0.4 0.5 0.6 0.7 0.8 0.9 0.2 0.4 0.6 0.8 1.0 Epoch Training loss Validation loss Training loss: 0.0 · 0.84 Training loss: 0.0 · 0.76 Training loss: 0.0 · 0.68 Training loss: 0.0 · 0.67 Training loss: 0.0 · 0.64 Training loss: 0.1 · 0.65 Training loss: 0.1 · 0.63 Training loss: 0.1 · 0.61 Training loss: 0.1 · 0.60 Training loss: 0.1 · 0.61 Training loss: 0.1 · 0.60 Training loss: 0.1 · 0.59 Training loss: 0.1 · 0.60 Training loss: 0.1 · 0.58 Training loss: 0.1 · 0.61 Training loss: 0.1 · 0.58 Training loss: 0.1 · 0.58 Training loss: 0.2 · 0.59 Training loss: 0.2 · 0.57 Training loss: 0.2 · 0.56 Training loss: 0.2 · 0.56 Training loss: 0.2 · 0.55 Training loss: 0.2 · 0.55 Training loss: 0.2 · 0.56 Training loss: 0.2 · 0.55 Training loss: 0.2 · 0.56 Training loss: 0.2 · 0.55 Training loss: 0.2 · 0.57 Training loss: 0.3 · 0.56 Training loss: 0.3 · 0.55 Training loss: 0.3 · 0.55 Training loss: 0.3 · 0.56 Training loss: 0.3 · 0.54 Training loss: 0.3 · 0.54 Training loss: 0.3 · 0.55 Training loss: 0.3 · 0.54 Training loss: 0.3 · 0.56 Training loss: 0.3 · 0.57 Training loss: 0.3 · 0.56 Training loss: 0.3 · 0.56 Training loss: 0.4 · 0.55 Training loss: 0.4 · 0.54 Training loss: 0.4 · 0.53 Training loss: 0.4 · 0.52 Training loss: 0.4 · 0.54 Training loss: 0.4 · 0.52 Training loss: 0.4 · 0.55 Training loss: 0.4 · 0.54 Training loss: 0.4 · 0.51 Training loss: 0.4 · 0.52 Training loss: 0.4 · 0.54 Training loss: 0.5 · 0.55 Training loss: 0.5 · 0.51 Training loss: 0.5 · 0.54 Training loss: 0.5 · 0.52 Training loss: 0.5 · 0.54 Training loss: 0.5 · 0.53 Training loss: 0.5 · 0.54 Training loss: 0.5 · 0.54 Training loss: 0.5 · 0.53 Training loss: 0.5 · 0.53 Training loss: 0.5 · 0.52 Training loss: 0.6 · 0.52 Training loss: 0.6 · 0.54 Training loss: 0.6 · 0.52 Training loss: 0.6 · 0.53 Training loss: 0.6 · 0.53 Training loss: 0.6 · 0.51 Training loss: 0.6 · 0.52 Training loss: 0.6 · 0.52 Training loss: 0.6 · 0.51 Training loss: 0.6 · 0.50 Training loss: 0.6 · 0.50 Training loss: 0.6 · 0.53 Training loss: 0.7 · 0.50 Training loss: 0.7 · 0.52 Training loss: 0.7 · 0.51 Training loss: 0.7 · 0.53 Training loss: 0.7 · 0.52 Training loss: 0.7 · 0.53 Training loss: 0.7 · 0.51 Training loss: 0.7 · 0.51 Training loss: 0.7 · 0.53 Training loss: 0.7 · 0.52 Training loss: 0.7 · 0.50 Training loss: 0.8 · 0.53 Training loss: 0.8 · 0.51 Training loss: 0.8 · 0.50 Training loss: 0.8 · 0.52 Training loss: 0.8 · 0.51 Training loss: 0.8 · 0.53 Training loss: 0.8 · 0.49 Training loss: 0.8 · 0.50 Training loss: 0.8 · 0.51 Training loss: 0.8 · 0.50 Training loss: 0.8 · 0.50 Training loss: 0.8 · 0.50 Training loss: 0.9 · 0.49 Training loss: 0.9 · 0.49 Training loss: 0.9 · 0.51 Training loss: 0.9 · 0.51 Training loss: 0.9 · 0.53 Training loss: 0.9 · 0.50 Training loss: 0.9 · 0.49 Training loss: 0.9 · 0.51 Training loss: 0.9 · 0.51 Training loss: 0.9 · 0.53 Training loss: 0.9 · 0.51 Training loss: 1.0 · 0.50 Training loss: 1.0 · 0.49 Training loss: 1.0 · 0.50 Training loss: 1.0 · 0.50 Training loss: 1.0 · 0.50 Training loss: 1.0 · 0.50 Training loss: 1.0 · 0.49 Validation loss: 0.1 · 0.63 Validation loss: 0.1 · 0.63 Validation loss: 0.1 · 0.60 Validation loss: 0.1 · 0.60 Validation loss: 0.2 · 0.58 Validation loss: 0.2 · 0.58 Validation loss: 0.3 · 0.56 Validation loss: 0.3 · 0.56 Validation loss: 0.3 · 0.55 Validation loss: 0.3 · 0.55 Validation loss: 0.4 · 0.54 Validation loss: 0.4 · 0.54 Validation loss: 0.5 · 0.54 Validation loss: 0.5 · 0.54 Validation loss: 0.6 · 0.53 Validation loss: 0.6 · 0.53 Validation loss: 0.6 · 0.52 Validation loss: 0.6 · 0.52 Validation loss: 0.7 · 0.52 Validation loss: 0.7 · 0.52 Validation loss: 0.8 · 0.52 Validation loss: 0.8 · 0.52 Validation loss: 0.8 · 0.51 Validation loss: 0.8 · 0.51 Validation loss: 0.9 · 0.51 Validation loss: 0.9 · 0.51 Validation loss: 1.0 · 0.51 Validation loss: 1.0 · 0.51 Validation loss: 1.0 · 0.51 Validation loss: 1.0 · 0.51

Each point is the average of 5 consecutive records.

Show values as a table
Training loss
0.010.8421
0.050.6483
0.100.5975
0.140.5828
0.180.5622
0.230.5602
0.270.5509
0.310.5387
0.360.5504
0.400.5220
0.450.5358
0.490.5406
0.530.5296
0.580.5333
0.620.5053
0.660.5165
0.710.5090
0.750.5308
0.800.5334
0.840.5043
0.880.5118
0.930.5051
0.970.5022
1.000.4946
Validation loss
0.070.6293
0.140.5990
0.210.5781
0.280.5642
0.350.5544
0.420.5433
0.490.5368
0.560.5298
0.630.5238
0.700.5194
0.770.5163
0.840.5139
0.910.5129
0.980.5127
10.5126
Live v6 vs v7: procedure exam
Live v6 v7
0% 25% 50% 75% 100% Procedure check ·… Procedure points ·… Procedure check ·… Procedure points ·… Procedure check · legal · Live v6: 0.0% 0.0% Procedure check · legal · v7: 98.6% 98.6% Procedure points · legal · Live v6: 11.4% 11.4% Procedure points · legal · v7: 81.4% 81.4% Procedure check · general · Live v6: 0.0% 0.0% Procedure check · general · v7: 32.9% 32.9% Procedure points · general · Live v6: 11.4% 11.4% Procedure points · general · v7: 45.7% 45.7%

Same 70 legal questions, live settings. Thinking on in legal mode, off in general mode.

Show values as a table
Live v6v7
Procedure check · legal0.0098.60
Procedure points · legal11.4081.40
Procedure check · general0.0032.90
Procedure points · general11.4045.70

Technical sheet

Versionv7
ReleaseSeptember 2026
InputText and image
OutputText
LanguageTurkish-focused; responds in the same language for other languages
Open technical sheet
CALLAII v6 · Previous version
CALLAII v6 Previous version Released September 2026

CALLAII v6 powers all AI surfaces within the Callmenta panel. It was trained on data generated from scratch for this purpose; every sample passed rule-based checks and a judge evaluation.

What improved in this version

  • Petition and contract style was learned from real documents of the law firm we work with
  • Contract drafting and evidence review were added
  • Multi-turn conversation and tool use were strengthened
  • Training was regularly evaluated on 740 unseen samples; validation error decreased at every measurement, and the model did not memorize

What it does

  • Call analysis: summary, action items, missed opportunities, and criteria scorecard from call transcripts
  • Coaching: evidence-based feedback for agent performance
  • Live support: answers visitor questions and escalates to a human when needed
  • CRM and reporting: customer card summary, metric interpretation, and commentary
  • CALLAII Legal: drafting petitions and contracts, document analysis, and legal/regulatory/case-law queries
  • KVKK masking: extracting personal data from text
  • Tool use: search, record creation, and document retrieval within the panel
  • Visual reading: understanding documents and screenshots

Limitations and responsible use

  • Legal knowledge does not come from the model's memory; laws, regulations, and rulings are retrieved from the current library each time
  • Generated text is a draft; petitions, contracts, and official correspondence must be approved by an authorized person
  • Numbers and dates must be verified; the model was trained to state missing information rather than fabricate it
  • Very long documents are processed in segments
  • Quality in languages other than Turkish has not been measured with the same granularity as Turkish

Training and measurements

3,101Step
2Epoch
0.4729Final validation loss
Training data37,102 samples, 118 million tokens
Quality filtering37,600 of 47,872 generated samples were accepted
Validation set740 samples never seen during training
Steps3,101 · 2 epochs · 96,000 tokens per step
Training context12,288 tokens
Validation loss0.7435 → 0.4729
Training dateSeptember 2026
Training and validation loss
Training loss Validation loss
0.0 0.2 0.4 0.6 0.8 0 1,000 2,000 3,000 Step Training loss Validation loss Training loss: 0 · 0.78 Training loss: 0 · 0.78 Training loss: 2,700 · 0.40 Training loss: 2,700 · 0.40 Training loss: 3,101 · 0.38 Training loss: 3,101 · 0.38 Validation loss: 0 · 0.74 Validation loss: 0 · 0.74 Validation loss: 2,700 · 0.47 Validation loss: 2,700 · 0.47 Validation loss: 3,101 · 0.47 Validation loss: 3,101 · 0.47

Only measured points; records of intermediate steps were not retained.

Show values as a table
Training loss
00.7800
2,7000.3951
3,1010.3763
Validation loss
00.7435
2,7000.4730
3,1010.4729
Training data filtering (example)
0 20,000 40,000 60,000 Generated Accepted Training Validation Generated · Samples: 47,872.0 47,872.0 Accepted · Samples: 37,600.0 37,600.0 Training · Samples: 37,102.0 37,102.0 Validation · Samples: 740.0 740.0

Every generated sample passed rule-based checks and judge evaluation; those that failed were discarded.

Show values as a table
Samples
Generated47,872.00
Accepted37,600.00
Training37,102.00
Validation740.00

Technical sheet

Versionv6
ReleaseSeptember 2026
InputText and images
OutputText
LanguageTurkish-focused; other languages answered in the same language
Customer datanot used for training
Hostingservers managed by Callmenta; your data is not sent to a third-party AI provider
Open technical sheet
CALLAII v5 · Previous version
CALLAII v5 Previous version Released September 2026

CALLAII's first comprehensive release.

What improved in this version

  • Call center, coaching, and live support surfaces unified in a single model.

Training and measurements

411Step
1Epoch
0.4867Final validation loss
Training data45,000 samples, 15 categories, 9 languages
Steps411 · 1 epoch
Tokens processed26.7 million
Final validation loss0.4867
Validation token accuracy85.4%
Training and validation loss
Training loss Validation loss
0 1 2 3 100 200 300 400 Step Training loss Validation loss Training loss: 10 · 2.73 Training loss: 20 · 2.13 Training loss: 30 · 1.25 Training loss: 40 · 0.94 Training loss: 50 · 0.84 Training loss: 60 · 0.77 Training loss: 70 · 0.70 Training loss: 80 · 0.64 Training loss: 90 · 0.61 Training loss: 100 · 0.61 Training loss: 110 · 0.58 Training loss: 120 · 0.58 Training loss: 130 · 0.57 Training loss: 140 · 0.55 Training loss: 150 · 0.56 Training loss: 160 · 0.55 Training loss: 170 · 0.54 Training loss: 180 · 0.56 Training loss: 190 · 0.56 Training loss: 200 · 0.52 Training loss: 210 · 0.54 Training loss: 220 · 0.51 Training loss: 230 · 0.54 Training loss: 240 · 0.52 Training loss: 250 · 0.51 Training loss: 260 · 0.50 Training loss: 270 · 0.50 Training loss: 280 · 0.52 Training loss: 290 · 0.51 Training loss: 300 · 0.49 Training loss: 310 · 0.50 Training loss: 320 · 0.51 Training loss: 330 · 0.50 Training loss: 340 · 0.48 Training loss: 350 · 0.47 Training loss: 360 · 0.51 Training loss: 370 · 0.49 Training loss: 380 · 0.50 Training loss: 390 · 0.50 Training loss: 400 · 0.51 Training loss: 410 · 0.50 Validation loss: 40 · 0.88 Validation loss: 40 · 0.88 Validation loss: 80 · 0.62 Validation loss: 80 · 0.62 Validation loss: 120 · 0.57 Validation loss: 120 · 0.57 Validation loss: 160 · 0.54 Validation loss: 160 · 0.54 Validation loss: 200 · 0.52 Validation loss: 200 · 0.52 Validation loss: 240 · 0.51 Validation loss: 240 · 0.51 Validation loss: 280 · 0.50 Validation loss: 280 · 0.50 Validation loss: 320 · 0.49 Validation loss: 320 · 0.49 Validation loss: 360 · 0.49 Validation loss: 360 · 0.49 Validation loss: 400 · 0.49 Validation loss: 400 · 0.49 Validation loss: 411 · 0.49 Validation loss: 411 · 0.49
Show values as a table
Training loss
102.7335
301.2499
500.8402
700.7024
900.6135
1100.5847
1300.5689
1500.5584
1700.5385
1900.5599
2100.5412
2300.5371
2500.5122
2700.5038
2900.5125
3100.5011
3300.4956
3500.4728
3700.4874
3900.4968
4100.4969
Validation loss
400.8849
800.6208
1200.5651
1600.5402
2000.5221
2400.5106
2800.5010
3200.4927
3600.4880
4000.4867
4110.4867
Token accuracy (%)
Training Validation
50% 60% 70% 80% 90% 100 200 300 400 Step Training Validation Training: 10 · 57.20% Training: 20 · 62.39% Training: 30 · 70.07% Training: 40 · 75.85% Training: 50 · 78.02% Training: 60 · 79.40% Training: 70 · 80.74% Training: 80 · 82.12% Training: 90 · 82.59% Training: 100 · 82.66% Training: 110 · 83.02% Training: 120 · 83.18% Training: 130 · 83.40% Training: 140 · 83.70% Training: 150 · 83.64% Training: 160 · 83.54% Training: 170 · 84.01% Training: 180 · 83.55% Training: 190 · 83.63% Training: 200 · 84.33% Training: 210 · 83.78% Training: 220 · 84.54% Training: 230 · 83.92% Training: 240 · 84.32% Training: 250 · 84.41% Training: 260 · 84.74% Training: 270 · 84.64% Training: 280 · 84.24% Training: 290 · 84.49% Training: 300 · 85.05% Training: 310 · 84.78% Training: 320 · 84.73% Training: 330 · 84.86% Training: 340 · 85.15% Training: 350 · 85.33% Training: 360 · 84.60% Training: 370 · 85.09% Training: 380 · 84.89% Training: 390 · 84.94% Training: 400 · 84.42% Training: 410 · 84.81% Validation: 40 · 77.40% Validation: 40 · 77.40% Validation: 80 · 82.75% Validation: 80 · 82.75% Validation: 120 · 83.79% Validation: 120 · 83.79% Validation: 160 · 84.25% Validation: 160 · 84.25% Validation: 200 · 84.61% Validation: 200 · 84.61% Validation: 240 · 84.85% Validation: 240 · 84.85% Validation: 280 · 85.04% Validation: 280 · 85.04% Validation: 320 · 85.23% Validation: 320 · 85.23% Validation: 360 · 85.32% Validation: 360 · 85.32% Validation: 400 · 85.37% Validation: 400 · 85.37% Validation: 411 · 85.37% Validation: 411 · 85.37%
Show values as a table
Training
1057.20
3070.07
5078.02
7080.74
9082.59
11083.02
13083.40
15083.64
17084.01
19083.63
21083.78
23083.92
25084.41
27084.64
29084.49
31084.78
33084.86
35085.33
37085.09
39084.94
41084.81
Validation
4077.40
8082.75
12083.79
16084.25
20084.61
24084.85
28085.04
32085.23
36085.32
40085.37
41185.37

Technical sheet

Versionv5
ReleaseSeptember 2026
StatusSuperseded by v6
Open technical sheet
CALLAII® by CALLMENTA

CALLAII Edge

Language model

The CALLAII language model prepared to run on the institution's own server.

CALLAII Edge v2 Live Released October 2026

On-premises deployment for CALLAII. Training completed on 15 October 2026. Passed the same exams as Edge v1. Results below.

What improved in this version

  • Call analysis achieved complete output, rising from 1 to 527 out of 534 analyses
  • Scorable call separation on 40 real calls with pre-determined correct answers improved from 45.8% to 90.8%
  • Legal answers with procedure check went from 0 to 69 out of 70 questions
  • Map questions scored 57 to 90, shape counting from 58.3 to 88.3
  • Field task preparation on energy screens improved from 56.7 to 96.7
  • Data never leaves the organization, it runs on the organization's own server

Limitations and responsible use

  • Accuracy on resolution status and request type remains limited: 56% and 44% on real calls
  • Critical citation and quotation errors in legal documents have not decreased, and every document must be reviewed by an attorney

Training and measurements

4,390Step
1Epoch
0.3556Final validation loss
Training data70,236 samples, 8,000 of which are visual
Validation set186 samples
Step4,390 · 1 epoch · 16 samples per step
Training context28,672 tokens
Training dateOctober 2026
Data for every screen was generated by the live system's own prompt. The model is trained on the prompt it sees.
Petition, formal notice, and legal chat content comes from the source library. Procedure review relies solely on the provided articles.
Call scoring was taught using examples where the missing behavior was known in advance. In real calls, the median of three separate evaluations was used, and classification fields were corrected against human labels.
In live chat, not sharing internal rules, speaking in the company's own name in every language, and not inventing unknown products were taught separately.
Sales, cross-selling, and competency scenarios were validated against known ground truths.
Real calls reserved for evaluation were excluded from training.
Training and validation loss
Training loss Validation loss
0.0 0.5 1.0 1.5 2.0 1,000 2,000 3,000 4,000 Step Training loss Validation loss Training loss: 40 · 1.53 Training loss: 80 · 0.92 Training loss: 120 · 0.76 Training loss: 160 · 0.69 Training loss: 200 · 0.65 Training loss: 240 · 0.63 Training loss: 280 · 0.62 Training loss: 320 · 0.58 Training loss: 360 · 0.60 Training loss: 400 · 0.60 Training loss: 440 · 0.59 Training loss: 480 · 0.57 Training loss: 520 · 0.58 Training loss: 560 · 0.58 Training loss: 600 · 0.56 Training loss: 640 · 0.57 Training loss: 680 · 0.57 Training loss: 720 · 0.57 Training loss: 760 · 0.55 Training loss: 800 · 0.53 Training loss: 840 · 0.52 Training loss: 880 · 0.54 Training loss: 920 · 0.54 Training loss: 960 · 0.54 Training loss: 1,000 · 0.54 Training loss: 1,040 · 0.53 Training loss: 1,080 · 0.55 Training loss: 1,120 · 0.52 Training loss: 1,160 · 0.54 Training loss: 1,200 · 0.54 Training loss: 1,240 · 0.54 Training loss: 1,280 · 0.54 Training loss: 1,320 · 0.55 Training loss: 1,360 · 0.51 Training loss: 1,400 · 0.52 Training loss: 1,440 · 0.52 Training loss: 1,480 · 0.53 Training loss: 1,520 · 0.53 Training loss: 1,560 · 0.53 Training loss: 1,600 · 0.51 Training loss: 1,640 · 0.52 Training loss: 1,680 · 0.53 Training loss: 1,720 · 0.52 Training loss: 1,760 · 0.52 Training loss: 1,800 · 0.51 Training loss: 1,840 · 0.52 Training loss: 1,880 · 0.52 Training loss: 1,920 · 0.50 Training loss: 1,960 · 0.49 Training loss: 2,000 · 0.52 Training loss: 2,040 · 0.49 Training loss: 2,080 · 0.53 Training loss: 2,120 · 0.51 Training loss: 2,160 · 0.48 Training loss: 2,200 · 0.50 Training loss: 2,240 · 0.48 Training loss: 2,280 · 0.49 Training loss: 2,320 · 0.48 Training loss: 2,360 · 0.51 Training loss: 2,400 · 0.49 Training loss: 2,440 · 0.49 Training loss: 2,480 · 0.49 Training loss: 2,520 · 0.51 Training loss: 2,560 · 0.50 Training loss: 2,600 · 0.50 Training loss: 2,640 · 0.48 Training loss: 2,680 · 0.49 Training loss: 2,720 · 0.48 Training loss: 2,760 · 0.49 Training loss: 2,800 · 0.49 Training loss: 2,840 · 0.49 Training loss: 2,880 · 0.49 Training loss: 2,920 · 0.49 Training loss: 2,960 · 0.49 Training loss: 3,000 · 0.49 Training loss: 3,040 · 0.47 Training loss: 3,080 · 0.50 Training loss: 3,120 · 0.46 Training loss: 3,160 · 0.49 Training loss: 3,200 · 0.47 Training loss: 3,240 · 0.47 Training loss: 3,280 · 0.49 Training loss: 3,320 · 0.48 Training loss: 3,360 · 0.49 Training loss: 3,400 · 0.48 Training loss: 3,440 · 0.48 Training loss: 3,480 · 0.48 Training loss: 3,520 · 0.50 Training loss: 3,560 · 0.46 Training loss: 3,600 · 0.46 Training loss: 3,640 · 0.49 Training loss: 3,680 · 0.48 Training loss: 3,720 · 0.47 Training loss: 3,760 · 0.47 Training loss: 3,800 · 0.47 Training loss: 3,840 · 0.48 Training loss: 3,880 · 0.47 Training loss: 3,920 · 0.47 Training loss: 3,960 · 0.48 Training loss: 4,000 · 0.48 Training loss: 4,040 · 0.48 Training loss: 4,080 · 0.46 Training loss: 4,120 · 0.49 Training loss: 4,160 · 0.47 Training loss: 4,200 · 0.48 Training loss: 4,240 · 0.48 Training loss: 4,280 · 0.46 Training loss: 4,320 · 0.48 Training loss: 4,360 · 0.47 Training loss: 4,390 · 0.47 Validation loss: 100 · 0.61 Validation loss: 200 · 0.53 Validation loss: 300 · 0.52 Validation loss: 400 · 0.47 Validation loss: 500 · 0.50 Validation loss: 600 · 0.46 Validation loss: 700 · 0.45 Validation loss: 800 · 0.51 Validation loss: 900 · 0.45 Validation loss: 1,000 · 0.44 Validation loss: 1,100 · 0.42 Validation loss: 1,200 · 0.43 Validation loss: 1,300 · 0.41 Validation loss: 1,400 · 0.41 Validation loss: 1,500 · 0.41 Validation loss: 1,600 · 0.41 Validation loss: 1,700 · 0.41 Validation loss: 1,800 · 0.40 Validation loss: 1,900 · 0.40 Validation loss: 2,000 · 0.39 Validation loss: 2,100 · 0.39 Validation loss: 2,200 · 0.40 Validation loss: 2,300 · 0.39 Validation loss: 2,400 · 0.38 Validation loss: 2,500 · 0.38 Validation loss: 2,600 · 0.37 Validation loss: 2,700 · 0.38 Validation loss: 2,800 · 0.37 Validation loss: 2,900 · 0.37 Validation loss: 3,000 · 0.37 Validation loss: 3,100 · 0.36 Validation loss: 3,200 · 0.36 Validation loss: 3,300 · 0.36 Validation loss: 3,400 · 0.36 Validation loss: 3,500 · 0.36 Validation loss: 3,600 · 0.36 Validation loss: 3,700 · 0.36 Validation loss: 3,800 · 0.36 Validation loss: 3,900 · 0.36 Validation loss: 4,000 · 0.36 Validation loss: 4,100 · 0.35 Validation loss: 4,200 · 0.36 Validation loss: 4,300 · 0.36 Validation loss: 4,390 · 0.36

Each point is the average of 8 consecutive records.

Show values as a table
Training loss
401.5271
2400.6349
4400.5892
6400.5677
8400.5240
1,0400.5314
1,2400.5365
1,4400.5235
1,6400.5234
1,8400.5176
2,0400.4950
2,2400.4778
2,4400.4893
2,6400.4778
2,8400.4898
3,0400.4738
3,2400.4671
3,4400.4767
3,6400.4941
3,8400.4777
4,0400.4774
4,2400.4781
4,3900.4748
Validation loss
1000.6060
3000.5230
5000.5014
7000.4486
9000.4476
1,1000.4190
1,3000.4133
1,5000.4132
1,7000.4074
1,9000.3983
2,1000.3949
2,3000.3911
2,5000.3768
2,7000.3757
2,9000.3729
3,1000.3639
3,3000.3602
3,5000.3579
3,7000.3570
3,9000.3557
4,1000.3548
4,3000.3553
4,3900.3556

Comparison of CALLAII Edge v1 with the previous main model

Edge v2 was compared with two models. One is the previous Edge version. The other is the previous main model currently running in production (CALLAII v7). All three took the same exams under the same conditions. Measurement was performed on the same exam server on October 15, 2026. Call analysis and procedure exam were run three times, others were run once. Calls and prompts were identical across all three models. In on-premises deployment, Edge runs in compressed mode, so results may differ slightly. Validation losses were measured with different sets, so they are not comparable across versions.

45,847 → 70,236 samplesTraining data
4,096 → 28,672 tokensTraining context
1 out of 534 → 527 out of 534Complete analysis output
0 out of 70 → 69 out of 70Starting with procedural check
88 → 95 pointsAddress parsing
2 → 4Critical citation and quotation error in law (70 documents, live setting)
531 out of 534 · Edge v2 527 out of 534With the previous main model: complete analysis output
22.7% · Edge v2 43.9%With the previous main model: request type (40 real calls)
8 · Edge v2 4With the previous main model: critical citation and quotation error in law (70 documents)
92.1 · Edge v2 95 pointsWith the previous main model: address parsing
  • In call analysis, complete output went from 1 to 527 out of 534 analyses. Edge v1 often left today's analysis template incomplete or could not write it in a valid format.
  • On 40 real calls with pre-determined correct answers, scorable call separation rose from 45.8% to 90.8%, and risk level from 36.4% to 75.8%.
  • Resolution status went from 18.2% to 56.1%, and request type from 13.6% to 43.9%. These two fields remain the weakest areas.
  • In legal procedure review, starting the answer with a procedure check went from 0 to 69 out of 70 questions.
  • On map questions, the score went from 57 to 90, and on shape counting from 58.3 to 88.3.
  • On energy screens, the largest increase was in field task preparation (from 56.7 to 96.7) and manager route (from 60 to 96.7).
  • The citation and quotation errors deemed critical by the audit in legal documents did not decrease. In live mode, out of 70 documents, Edge v1 had 2 and Edge v2 had 4. In the no-thinking run, 3 and 1. The sample is small, document quality was not separately scored this round.
  • Legal output truncated at the length limit dropped from 1 to 0.
  • Edge v2 reached the call analysis level of the previous main model in live mode and surpassed it in some areas. In request type, 40 real calls: 43.9% vs 22.7%. 138 targeted calls: 88.9% vs 70.4%.
  • Edge v2 leads in separating scoreable calls: 40 real calls 90.8% vs 84.2%, targeted calls 88.2% vs 64.7%.
  • Where Edge v2 lags: analyses producing no valid output 7 out of 534, previous main model 3. Resolution status in targeted calls 71 vs 73, risk level in real calls 50 vs 51. Procedure check at start 69 vs 70.
  • Edge v2 outperforms the previous main model on image and energy screens. Map 90 vs 60, route plan evaluation 100 vs 20, field task preparation 96.7 vs 53.3.
  • Citation and quotation errors deemed critical by the audit in legal documents: 8 in the previous main model vs 4 in Edge v2 out of 70 documents. Truncated documents 0 vs 1.
Call analysis: 40 real calls with pre-determined correct answers (%)
Edge v1 Previous main model (CALLAII v7) Edge v2
0% 25% 50% 75% 100% Scorable Risk level Resolution status Request type Scorable · Edge v1: 45.8% 45.8% Scorable · Previous main model (CALLAII v7): 84.2% 84.2% Scorable · Edge v2: 90.8% 90.8% Risk level · Edge v1: 36.4% 36.4% Risk level · Previous main model (CALLAII v7): 77.3% 77.3% Risk level · Edge v2: 75.8% 75.8% Resolution status · Edge v1: 18.2% 18.2% Resolution status · Previous main model (CALLAII v7): 54.5% 54.5% Resolution status · Edge v2: 56.1% 56.1% Request type · Edge v1: 13.6% 13.6% Request type · Previous main model (CALLAII v7): 22.7% 22.7% Request type · Edge v2: 43.9% 43.9%

Same calls and same prompts, sum of 3 runs. Analyses that did not produce valid output were counted as incorrect.

Show values as a table
Edge v1Previous main model (CALLAII v7)Edge v2
Scorable45.8084.2090.80
Risk level36.4077.3075.80
Resolution status18.2054.5056.10
Request type13.6022.7043.90
Call analysis: 138 targeted exam calls (%)
Edge v1 Previous main model (CALLAII v7) Edge v2
0% 25% 50% 75% 100% Scorable Risk level Resolution status Request type Scorable · Edge v1: 17.6% 17.6% Scorable · Previous main model (CALLAII v7): 64.7% 64.7% Scorable · Edge v2: 88.2% 88.2% Risk level · Edge v1: 31.0% 31.0% Risk level · Previous main model (CALLAII v7): 66.7% 66.7% Risk level · Edge v2: 69.0% 69.0% Resolution status · Edge v1: 29.3% 29.3% Resolution status · Previous main model (CALLAII v7): 59.3% 59.3% Resolution status · Edge v2: 57.7% 57.7% Request type · Edge v1: 0.0% 0.0% Request type · Previous main model (CALLAII v7): 70.4% 70.4% Request type · Edge v2: 88.9% 88.9%

Same calls and same prompts, sum of 3 runs. Analyses that did not produce valid output were counted as incorrect.

Show values as a table
Edge v1Previous main model (CALLAII v7)Edge v2
Scorable17.6064.7088.20
Risk level31.0066.7069.00
Resolution status29.3059.3057.70
Request type0.0070.4088.90
Legal procedure review: 70 questions (%)
Edge v1 Previous main model (CALLAII v7) Edge v2
0% 25% 50% 75% 100% Starting with proc… Procedure points Starting with procedure control · Edge v1: 0.0% 0.0% Starting with procedure control · Previous main model (CALLAII v7): 100.0% 100.0% Starting with procedure control · Edge v2: 98.6% 98.6% Procedure points · Edge v1: 7.1% 7.1% Procedure points · Previous main model (CALLAII v7): 85.7% 85.7% Procedure points · Edge v2: 85.7% 85.7%

Thinking mode, median of 3 runs. The previous base model started with procedure control in 70 out of 70 questions across all three runs.

Show values as a table
Edge v1Previous main model (CALLAII v7)Edge v2
Starting with procedure control0.00100.0098.60
Procedure points7.1085.7085.70
Image and map (%)
Edge v1 Previous main model (CALLAII v7) Edge v2
0% 25% 50% 75% 100% Map Other image Shape counting Map · Edge v1: 57.0% 57.0% Map · Previous main model (CALLAII v7): 60.0% 60.0% Map · Edge v2: 90.0% 90.0% Other image · Edge v1: 71.3% 71.3% Other image · Previous main model (CALLAII v7): 82.3% 82.3% Other image · Edge v2: 100.0% 100.0% Shape counting · Edge v1: 58.3% 58.3% Shape counting · Previous main model (CALLAII v7): 80.0% 80.0% Shape counting · Edge v2: 88.3% 88.3%

Definitive-answer questions, single run. Map: 100 questions, other image: 18-point question, shape counting: 60 questions.

Show values as a table
Edge v1Previous main model (CALLAII v7)Edge v2
Map57.0060.0090.00
Other image71.3082.30100.00
Shape counting58.3080.0088.30
Energy distribution screens and address parsing (score)
Edge v1 Previous main model (CALLAII v7) Edge v2
0 puan 25 puan 50 puan 75 puan 100 puan Route plan eva… Field assistan… Field task pre… Representative… Manager route Address parsin… Route plan evaluation · Edge v1: 90.0 puan Route plan evaluation · Previous main model (CALLAII v7): 20.0 puan Route plan evaluation · Edge v2: 100.0 puan Field assistant · Edge v1: 93.3 puan Field assistant · Previous main model (CALLAII v7): 96.7 puan Field assistant · Edge v2: 100.0 puan Field task preparation · Edge v1: 56.7 puan Field task preparation · Previous main model (CALLAII v7): 53.3 puan Field task preparation · Edge v2: 96.7 puan Representative assistant · Edge v1: 86.7 puan Representative assistant · Previous main model (CALLAII v7): 80.0 puan Representative assistant · Edge v2: 96.7 puan Manager route · Edge v1: 60.0 puan Manager route · Previous main model (CALLAII v7): 93.3 puan Manager route · Edge v2: 96.7 puan Address parsing · Edge v1: 88.0 puan Address parsing · Previous main model (CALLAII v7): 92.1 puan Address parsing · Edge v2: 95.0 puan

30 tasks per screen, 38 addresses, single run.

Show values as a table
Edge v1Previous main model (CALLAII v7)Edge v2
Route plan evaluation90.0020.00100.00
Field assistant93.3096.70100.00
Field task preparation56.7053.3096.70
Representative assistant86.7080.0096.70
Manager route60.0093.3096.70
Address parsing88.0092.1095.00
Open technical sheet

Previous versions

CALLAII Edge v1 · Previous version
CALLAII Edge v1 Previous version Released September 2026

CALLAII Edge is currently in training. The goal is to consolidate CALLAII's legal and call center capabilities into a model that runs on the organization's own server. The graph below is updated regularly during training.

What improved in this version

  • Goal: Identify the competent and authorized court, litigation prerequisites, and deadlines for a legal question
  • Goal: Introduce itself as CALLAII under all circumstances
  • Results will be published here after training is complete, using the same exam taken before training

Training and measurements

0.49Epoch
0.6186Final validation loss
StatusIn training
Training data45,847 samples
Validation set200 samples never seen during training
Training context4,096 tokens
Training dateSeptember 2026
Training and validation loss
Training loss Validation loss
0 2 4 6 0.1 0.2 0.3 0.4 Epoch Training loss Validation loss Training loss: 0.0 · 5.09 Training loss: 0.0 · 2.96 Training loss: 0.0 · 1.32 Training loss: 0.0 · 1.02 Training loss: 0.0 · 0.95 Training loss: 0.0 · 0.88 Training loss: 0.0 · 0.85 Training loss: 0.0 · 0.83 Training loss: 0.0 · 0.79 Training loss: 0.1 · 0.76 Training loss: 0.1 · 0.78 Training loss: 0.1 · 0.78 Training loss: 0.1 · 0.79 Training loss: 0.1 · 0.73 Training loss: 0.1 · 0.71 Training loss: 0.1 · 0.76 Training loss: 0.1 · 0.72 Training loss: 0.1 · 0.73 Training loss: 0.1 · 0.77 Training loss: 0.1 · 0.75 Training loss: 0.1 · 0.71 Training loss: 0.1 · 0.71 Training loss: 0.1 · 0.71 Training loss: 0.1 · 0.70 Training loss: 0.1 · 0.67 Training loss: 0.1 · 0.70 Training loss: 0.1 · 0.68 Training loss: 0.1 · 0.70 Training loss: 0.2 · 0.68 Training loss: 0.2 · 0.67 Training loss: 0.2 · 0.70 Training loss: 0.2 · 0.67 Training loss: 0.2 · 0.63 Training loss: 0.2 · 0.67 Training loss: 0.2 · 0.69 Training loss: 0.2 · 0.67 Training loss: 0.2 · 0.66 Training loss: 0.2 · 0.69 Training loss: 0.2 · 0.65 Training loss: 0.2 · 0.66 Training loss: 0.2 · 0.66 Training loss: 0.2 · 0.62 Training loss: 0.2 · 0.68 Training loss: 0.2 · 0.68 Training loss: 0.2 · 0.66 Training loss: 0.2 · 0.66 Training loss: 0.2 · 0.67 Training loss: 0.3 · 0.64 Training loss: 0.3 · 0.66 Training loss: 0.3 · 0.66 Training loss: 0.3 · 0.66 Training loss: 0.3 · 0.65 Training loss: 0.3 · 0.65 Training loss: 0.3 · 0.67 Training loss: 0.3 · 0.62 Training loss: 0.3 · 0.64 Training loss: 0.3 · 0.66 Training loss: 0.3 · 0.66 Training loss: 0.3 · 0.65 Training loss: 0.3 · 0.65 Training loss: 0.3 · 0.64 Training loss: 0.3 · 0.64 Training loss: 0.3 · 0.63 Training loss: 0.3 · 0.60 Training loss: 0.3 · 0.63 Training loss: 0.3 · 0.64 Training loss: 0.4 · 0.65 Training loss: 0.4 · 0.64 Training loss: 0.4 · 0.63 Training loss: 0.4 · 0.61 Training loss: 0.4 · 0.68 Training loss: 0.4 · 0.65 Training loss: 0.4 · 0.60 Training loss: 0.4 · 0.61 Training loss: 0.4 · 0.62 Training loss: 0.4 · 0.60 Training loss: 0.4 · 0.64 Training loss: 0.4 · 0.66 Training loss: 0.4 · 0.63 Training loss: 0.4 · 0.61 Training loss: 0.4 · 0.63 Training loss: 0.4 · 0.59 Training loss: 0.4 · 0.63 Training loss: 0.4 · 0.65 Training loss: 0.4 · 0.62 Training loss: 0.5 · 0.64 Training loss: 0.5 · 0.63 Training loss: 0.5 · 0.62 Training loss: 0.5 · 0.62 Training loss: 0.5 · 0.62 Training loss: 0.5 · 0.61 Training loss: 0.5 · 0.60 Training loss: 0.5 · 0.62 Training loss: 0.5 · 0.60 Validation loss: 0.1 · 0.74 Validation loss: 0.1 · 0.74 Validation loss: 0.3 · 0.63 Validation loss: 0.3 · 0.63 Validation loss: 0.4 · 0.62 Validation loss: 0.4 · 0.62

Each point is the average of 3 consecutive records.

Show values as a table
Training loss
0.015.0917
0.030.9474
0.050.7875
0.070.7862
0.090.7166
0.110.7118
0.130.6670
0.150.6756
0.170.6324
0.190.6646
0.220.6551
0.240.6583
0.260.6638
0.280.6530
0.300.6599
0.320.6420
0.340.6337
0.360.6260
0.380.6009
0.400.6391
0.420.6311
0.450.6200
0.470.6200
0.490.6187
0.490.6022
Validation loss
0.070.7438
0.350.6291
0.420.6186
Open technical sheet
CALLAII® by CALLMENTA

CALLAII Edge Small Business

Language model

A compact CALLAII language model prepared to run on small businesses' own hardware.

CALLAII Edge Small Business v1 Live Released September 2026

CALLAII Edge Small Business is designed for businesses that want a compact CALLAII running on their own server. It automatically checks procedural requirements in legal questions and reads document images.

What improved in this version

  • First release
  • Procedural check in legal questions: performed in 67 out of 70 exam questions, none before training
  • Procedural check points improved from 9 to 57 across 70 questions
  • Text extraction and image description from document images were fully preserved after training

What it does

  • Legal: petition and contract drafting, procedural check on legal questions
  • Procedural check: competent and authorized court, conditions of action, and deadlines
  • Call center: call summary, customer response, and quality evaluation
  • Visual reading: understanding text and content in documents and screenshots

Limitations and responsible use

  • Generated text is a draft; petitions, contracts, and official documents must be approved by an authorized person
  • Must be used together with a legislation library for current laws, regulations, and rulings; the model's memory is not up to date
  • Audio recordings are transcribed with CALLAII ASR; this model is not used for speech recognition
  • Quality in languages other than Turkish has not been measured in as much detail as Turkish

Training and measurements

1Epoch
0.810 → 0.605Validation loss
In 67 out of 70 questionsProcedural check
Training data45,847 samples
Validation set200 samples never seen during training
Steps2,861 · 1 epoch · 16 samples per step
Training context4,096 tokens
Validation loss0.810 → 0.605
Exam70 procedural questions, same questions before and after training
Training dateSeptember 2026
Training and validation loss
Training loss Validation loss
0 1 2 3 0.2 0.4 0.6 0.8 1.0 Epoch Training loss Validation loss Training loss: 0.0 · 2.09 Training loss: 0.0 · 0.99 Training loss: 0.0 · 0.88 Training loss: 0.0 · 0.81 Training loss: 0.0 · 0.83 Training loss: 0.1 · 0.82 Training loss: 0.1 · 0.78 Training loss: 0.1 · 0.81 Training loss: 0.1 · 0.76 Training loss: 0.1 · 0.78 Training loss: 0.1 · 0.77 Training loss: 0.1 · 0.81 Training loss: 0.1 · 0.74 Training loss: 0.1 · 0.76 Training loss: 0.1 · 0.72 Training loss: 0.1 · 0.73 Training loss: 0.1 · 0.74 Training loss: 0.2 · 0.70 Training loss: 0.2 · 0.70 Training loss: 0.2 · 0.68 Training loss: 0.2 · 0.70 Training loss: 0.2 · 0.69 Training loss: 0.2 · 0.72 Training loss: 0.2 · 0.68 Training loss: 0.2 · 0.68 Training loss: 0.2 · 0.68 Training loss: 0.2 · 0.67 Training loss: 0.2 · 0.68 Training loss: 0.3 · 0.68 Training loss: 0.3 · 0.69 Training loss: 0.3 · 0.66 Training loss: 0.3 · 0.69 Training loss: 0.3 · 0.65 Training loss: 0.3 · 0.65 Training loss: 0.3 · 0.68 Training loss: 0.3 · 0.67 Training loss: 0.3 · 0.66 Training loss: 0.3 · 0.64 Training loss: 0.3 · 0.66 Training loss: 0.3 · 0.66 Training loss: 0.4 · 0.65 Training loss: 0.4 · 0.63 Training loss: 0.4 · 0.71 Training loss: 0.4 · 0.65 Training loss: 0.4 · 0.65 Training loss: 0.4 · 0.63 Training loss: 0.4 · 0.65 Training loss: 0.4 · 0.63 Training loss: 0.4 · 0.62 Training loss: 0.4 · 0.67 Training loss: 0.4 · 0.63 Training loss: 0.5 · 0.64 Training loss: 0.5 · 0.65 Training loss: 0.5 · 0.63 Training loss: 0.5 · 0.62 Training loss: 0.5 · 0.64 Training loss: 0.5 · 0.66 Training loss: 0.5 · 0.63 Training loss: 0.5 · 0.63 Training loss: 0.5 · 0.63 Training loss: 0.5 · 0.62 Training loss: 0.5 · 0.64 Training loss: 0.6 · 0.60 Training loss: 0.6 · 0.65 Training loss: 0.6 · 0.63 Training loss: 0.6 · 0.62 Training loss: 0.6 · 0.61 Training loss: 0.6 · 0.63 Training loss: 0.6 · 0.64 Training loss: 0.6 · 0.62 Training loss: 0.6 · 0.64 Training loss: 0.6 · 0.63 Training loss: 0.6 · 0.63 Training loss: 0.6 · 0.64 Training loss: 0.7 · 0.62 Training loss: 0.7 · 0.60 Training loss: 0.7 · 0.62 Training loss: 0.7 · 0.60 Training loss: 0.7 · 0.61 Training loss: 0.7 · 0.62 Training loss: 0.7 · 0.61 Training loss: 0.7 · 0.61 Training loss: 0.7 · 0.61 Training loss: 0.7 · 0.61 Training loss: 0.7 · 0.63 Training loss: 0.8 · 0.60 Training loss: 0.8 · 0.59 Training loss: 0.8 · 0.60 Training loss: 0.8 · 0.59 Training loss: 0.8 · 0.62 Training loss: 0.8 · 0.63 Training loss: 0.8 · 0.59 Training loss: 0.8 · 0.62 Training loss: 0.8 · 0.59 Training loss: 0.8 · 0.63 Training loss: 0.8 · 0.60 Training loss: 0.8 · 0.58 Training loss: 0.9 · 0.59 Training loss: 0.9 · 0.59 Training loss: 0.9 · 0.58 Training loss: 0.9 · 0.60 Training loss: 0.9 · 0.59 Training loss: 0.9 · 0.63 Training loss: 0.9 · 0.61 Training loss: 0.9 · 0.60 Training loss: 0.9 · 0.60 Training loss: 0.9 · 0.67 Training loss: 0.9 · 0.58 Training loss: 1.0 · 0.59 Training loss: 1.0 · 0.58 Training loss: 1.0 · 0.63 Training loss: 1.0 · 0.60 Training loss: 1.0 · 0.59 Training loss: 1.0 · 0.61 Training loss: 1.0 · 0.59 Validation loss: 0.1 · 0.78 Validation loss: 0.1 · 0.78 Validation loss: 0.2 · 0.70 Validation loss: 0.2 · 0.70 Validation loss: 0.3 · 0.68 Validation loss: 0.3 · 0.68 Validation loss: 0.3 · 0.66 Validation loss: 0.3 · 0.66 Validation loss: 0.4 · 0.65 Validation loss: 0.4 · 0.65 Validation loss: 0.5 · 0.65 Validation loss: 0.5 · 0.65 Validation loss: 0.6 · 0.64 Validation loss: 0.6 · 0.64 Validation loss: 0.6 · 0.64 Validation loss: 0.6 · 0.64 Validation loss: 0.7 · 0.62 Validation loss: 0.7 · 0.62 Validation loss: 0.8 · 0.61 Validation loss: 0.8 · 0.61 Validation loss: 0.8 · 0.61 Validation loss: 0.8 · 0.61 Validation loss: 0.9 · 0.61 Validation loss: 0.9 · 0.61 Validation loss: 1.0 · 0.60 Validation loss: 1.0 · 0.60 Validation loss: 1.0 · 0.61 Validation loss: 1.0 · 0.61

Each point is the average of 5 consecutive records.

Show values as a table
Training loss
0.012.0882
0.050.8227
0.100.7656
0.140.7346
0.180.6991
0.230.6836
0.270.6616
0.310.6667
0.360.6545
0.400.6291
0.450.6276
0.490.6401
0.530.6182
0.580.6186
0.620.6388
0.660.5980
0.710.6144
0.750.6025
0.800.6273
0.840.6025
0.880.6050
0.930.5961
0.970.6281
1.000.5878
Validation loss
0.140.7797
0.210.7025
0.280.6809
0.350.6608
0.420.6528
0.490.6549
0.560.6427
0.630.6439
0.700.6169
0.770.6117
0.840.6079
0.910.6055
0.980.6049
10.6050
Pre- and post-training exam
Before training After training
0% 25% 50% 75% 100% Court and jurisdic… Procedural points Court and jurisdiction · Before training: 0.0% 0.0% Court and jurisdiction · After training: 95.7% 95.7% Procedural points · Before training: 12.9% 12.9% Procedural points · After training: 81.4% 81.4%

70 procedural questions, same questions before and after training.

Show values as a table
Before trainingAfter training
Court and jurisdiction0.0095.70
Procedural points12.9081.40

Technical sheet

Versionv1
ReleaseSeptember 2026
InputText and image
OutputText
LanguageTurkish-focused; responds in the same language for other languages
Runs onthe business's own server
Open technical sheet
CALLAII® by CALLMENTA

CALLAIIKARAR Mini

Language model

Hızlı karar modeli. Metin yazmaz, soruya seçenek başına olasılık verir.

CALLAIIKARAR Mini v1 In training

Metin yazmayan, karar veren model. Her seçenek için olasılık verir, emin değilse bunu söyler.

What improved in this version

  • Hedef: Canlı destekte insan temsilci isteği, duygu ve niyet
  • Hedef: Çağrıda memnuniyet, çözüm, risk ve kalite kriteri
  • Hedef: Hukukta soru ve evrak kararları

Training and measurements

2.140Adım
0,1301Son doğrulama kaybı
61 karar görevi, her seçenek için olasılık veren etiketler
Kısa ve dağınık live destek mesajları (selam, teşekkür, deneme, yazım hatası) ve konu devamı ayrıca öğretildi
Öğretmen emin değilse düşünerek ikinci görüş alındı, çelişen etiket atıldı
Gerçek mesaj ve çağrı testlerini geçmeyen görev canlıda açılmaz
Eğitime giren karar örnekleri
0 20,000 40,000 60,000 Calls analizi Hukuk Customer Canlı destek Risk ve uyum Dil ve genel Sales Calls analizi · Etiket: 58,322.0 58,322.0 Hukuk · Etiket: 41,911.0 41,911.0 Customer · Etiket: 36,291.0 36,291.0 Canlı destek · Etiket: 36,255.0 36,255.0 Risk ve uyum · Etiket: 21,794.0 21,794.0 Dil ve genel · Etiket: 20,655.0 20,655.0 Sales · Etiket: 2,026.0 2,026.0

Her etiket seçenek başına olasılık dağılımıdır; öğretmen emin değilse ikinci görüş alınır, çelişen etiket atılır.

Show values as a table
Etiket
Calls analizi58,322.00
Hukuk41,911.00
Customer36,291.00
Canlı destek36,255.00
Risk ve uyum21,794.00
Dil ve genel20,655.00
Sales2,026.00
Eğitim kaybı
0.0 0.5 1.0 1.5 2.0 500 1,000 1,500 2,000 Adım Eğitim kaybı Eğitim kaybı: 20 · 1.98 Eğitim kaybı: 40 · 0.97 Eğitim kaybı: 60 · 0.96 Eğitim kaybı: 80 · 0.99 Eğitim kaybı: 100 · 1.03 Eğitim kaybı: 120 · 1.03 Eğitim kaybı: 140 · 1.03 Eğitim kaybı: 160 · 1.00 Eğitim kaybı: 180 · 0.95 Eğitim kaybı: 200 · 0.91 Eğitim kaybı: 220 · 0.90 Eğitim kaybı: 240 · 0.88 Eğitim kaybı: 260 · 0.86 Eğitim kaybı: 280 · 0.92 Eğitim kaybı: 300 · 0.86 Eğitim kaybı: 320 · 0.82 Eğitim kaybı: 340 · 0.76 Eğitim kaybı: 360 · 0.80 Eğitim kaybı: 380 · 0.77 Eğitim kaybı: 400 · 0.83 Eğitim kaybı: 420 · 0.83 Eğitim kaybı: 440 · 0.79 Eğitim kaybı: 460 · 0.77 Eğitim kaybı: 480 · 0.76 Eğitim kaybı: 500 · 0.73 Eğitim kaybı: 520 · 0.80 Eğitim kaybı: 540 · 0.73 Eğitim kaybı: 560 · 0.82 Eğitim kaybı: 580 · 0.78 Eğitim kaybı: 600 · 0.71 Eğitim kaybı: 620 · 0.70 Eğitim kaybı: 640 · 0.74 Eğitim kaybı: 660 · 0.70 Eğitim kaybı: 680 · 0.64 Eğitim kaybı: 700 · 0.73 Eğitim kaybı: 720 · 0.75 Eğitim kaybı: 740 · 0.67 Eğitim kaybı: 760 · 0.74 Eğitim kaybı: 780 · 0.62 Eğitim kaybı: 800 · 0.70 Eğitim kaybı: 820 · 0.65 Eğitim kaybı: 840 · 0.69 Eğitim kaybı: 860 · 0.70 Eğitim kaybı: 880 · 0.67 Eğitim kaybı: 900 · 0.67 Eğitim kaybı: 920 · 0.68 Eğitim kaybı: 940 · 0.66 Eğitim kaybı: 960 · 0.63 Eğitim kaybı: 980 · 0.65 Eğitim kaybı: 1,000 · 0.67 Eğitim kaybı: 1,020 · 0.64 Eğitim kaybı: 1,040 · 0.73 Eğitim kaybı: 1,060 · 0.65 Eğitim kaybı: 1,080 · 0.64 Eğitim kaybı: 1,100 · 0.62 Eğitim kaybı: 1,120 · 0.63 Eğitim kaybı: 1,140 · 0.68 Eğitim kaybı: 1,160 · 0.63 Eğitim kaybı: 1,180 · 0.61 Eğitim kaybı: 1,200 · 0.62 Eğitim kaybı: 1,220 · 0.60 Eğitim kaybı: 1,240 · 0.64 Eğitim kaybı: 1,260 · 0.58 Eğitim kaybı: 1,280 · 0.64 Eğitim kaybı: 1,300 · 0.58 Eğitim kaybı: 1,320 · 0.63 Eğitim kaybı: 1,340 · 0.59 Eğitim kaybı: 1,360 · 0.62 Eğitim kaybı: 1,380 · 0.59 Eğitim kaybı: 1,400 · 0.60 Eğitim kaybı: 1,420 · 0.66 Eğitim kaybı: 1,440 · 0.60 Eğitim kaybı: 1,460 · 0.60 Eğitim kaybı: 1,480 · 0.58 Eğitim kaybı: 1,500 · 0.59 Eğitim kaybı: 1,520 · 0.64 Eğitim kaybı: 1,540 · 0.57 Eğitim kaybı: 1,560 · 0.59 Eğitim kaybı: 1,580 · 0.57 Eğitim kaybı: 1,600 · 0.57 Eğitim kaybı: 1,620 · 0.62 Eğitim kaybı: 1,640 · 0.60 Eğitim kaybı: 1,660 · 0.57 Eğitim kaybı: 1,680 · 0.59 Eğitim kaybı: 1,700 · 0.60 Eğitim kaybı: 1,720 · 0.60 Eğitim kaybı: 1,740 · 0.55 Eğitim kaybı: 1,760 · 0.56 Eğitim kaybı: 1,780 · 0.57 Eğitim kaybı: 1,800 · 0.58 Eğitim kaybı: 1,820 · 0.53 Eğitim kaybı: 1,840 · 0.56 Eğitim kaybı: 1,860 · 0.56 Eğitim kaybı: 1,880 · 0.57 Eğitim kaybı: 1,900 · 0.57 Eğitim kaybı: 1,920 · 0.57 Eğitim kaybı: 1,940 · 0.57 Eğitim kaybı: 1,960 · 0.59 Eğitim kaybı: 1,980 · 0.59 Eğitim kaybı: 2,000 · 0.56 Eğitim kaybı: 2,020 · 0.57 Eğitim kaybı: 2,040 · 0.58 Eğitim kaybı: 2,060 · 0.51 Eğitim kaybı: 2,080 · 0.56 Eğitim kaybı: 2,100 · 0.56 Eğitim kaybı: 2,120 · 0.54 Eğitim kaybı: 2,140 · 0.61

Her nokta ardışık 2 kaydın ortalaması.

Show values as a table
Eğitim kaybı
201.9822
1201.0299
2200.9012
3200.8154
4200.8297
5200.8049
6200.6988
7200.7466
8200.6469
9200.6818
1,0200.6352
1,1200.6270
1,2200.5981
1,3200.6299
1,4200.6558
1,5200.6393
1,6200.6216
1,7200.6009
1,8200.5323
1,9200.5709
2,0200.5718
2,1200.5364
2,1400.6093
Eğitim ve doğrulama kaybı
Eğitim kaybı Doğrulama kaybı
0.0 0.5 1.0 1.5 2.0 500 1,000 1,500 Adım Eğitim kaybı Doğrulama kaybı Eğitim kaybı: 20 · 1.98 Eğitim kaybı: 40 · 0.97 Eğitim kaybı: 60 · 0.96 Eğitim kaybı: 80 · 0.99 Eğitim kaybı: 100 · 1.03 Eğitim kaybı: 120 · 1.03 Eğitim kaybı: 140 · 1.03 Eğitim kaybı: 160 · 1.00 Eğitim kaybı: 180 · 0.95 Eğitim kaybı: 200 · 0.91 Eğitim kaybı: 220 · 0.90 Eğitim kaybı: 240 · 0.88 Eğitim kaybı: 260 · 0.86 Eğitim kaybı: 280 · 0.92 Eğitim kaybı: 300 · 0.86 Eğitim kaybı: 320 · 0.82 Eğitim kaybı: 340 · 0.76 Eğitim kaybı: 360 · 0.80 Eğitim kaybı: 380 · 0.77 Eğitim kaybı: 400 · 0.83 Eğitim kaybı: 420 · 0.83 Eğitim kaybı: 440 · 0.79 Eğitim kaybı: 460 · 0.77 Eğitim kaybı: 480 · 0.76 Eğitim kaybı: 500 · 0.73 Eğitim kaybı: 520 · 0.80 Eğitim kaybı: 540 · 0.73 Eğitim kaybı: 560 · 0.82 Eğitim kaybı: 580 · 0.78 Eğitim kaybı: 600 · 0.71 Eğitim kaybı: 620 · 0.70 Eğitim kaybı: 640 · 0.74 Eğitim kaybı: 660 · 0.70 Eğitim kaybı: 680 · 0.64 Eğitim kaybı: 700 · 0.73 Eğitim kaybı: 720 · 0.75 Eğitim kaybı: 740 · 0.67 Eğitim kaybı: 760 · 0.74 Eğitim kaybı: 780 · 0.62 Eğitim kaybı: 800 · 0.70 Eğitim kaybı: 820 · 0.65 Eğitim kaybı: 840 · 0.69 Eğitim kaybı: 860 · 0.70 Eğitim kaybı: 880 · 0.67 Eğitim kaybı: 900 · 0.67 Eğitim kaybı: 920 · 0.68 Eğitim kaybı: 940 · 0.66 Eğitim kaybı: 960 · 0.63 Eğitim kaybı: 980 · 0.65 Eğitim kaybı: 1,000 · 0.67 Eğitim kaybı: 1,020 · 0.64 Eğitim kaybı: 1,040 · 0.73 Eğitim kaybı: 1,060 · 0.65 Eğitim kaybı: 1,080 · 0.64 Eğitim kaybı: 1,100 · 0.62 Eğitim kaybı: 1,120 · 0.63 Eğitim kaybı: 1,140 · 0.68 Eğitim kaybı: 1,160 · 0.63 Eğitim kaybı: 1,180 · 0.61 Eğitim kaybı: 1,200 · 0.62 Eğitim kaybı: 1,220 · 0.60 Eğitim kaybı: 1,240 · 0.64 Eğitim kaybı: 1,260 · 0.58 Eğitim kaybı: 1,280 · 0.64 Eğitim kaybı: 1,300 · 0.58 Eğitim kaybı: 1,320 · 0.63 Eğitim kaybı: 1,340 · 0.59 Eğitim kaybı: 1,360 · 0.62 Eğitim kaybı: 1,380 · 0.59 Eğitim kaybı: 1,400 · 0.60 Eğitim kaybı: 1,420 · 0.66 Eğitim kaybı: 1,440 · 0.60 Eğitim kaybı: 1,460 · 0.60 Eğitim kaybı: 1,480 · 0.58 Eğitim kaybı: 1,500 · 0.59 Eğitim kaybı: 1,520 · 0.64 Eğitim kaybı: 1,540 · 0.57 Eğitim kaybı: 1,560 · 0.59 Eğitim kaybı: 1,580 · 0.57 Eğitim kaybı: 1,600 · 0.57 Eğitim kaybı: 1,620 · 0.62 Eğitim kaybı: 1,640 · 0.60 Eğitim kaybı: 1,660 · 0.57 Eğitim kaybı: 1,680 · 0.59 Eğitim kaybı: 1,700 · 0.60 Eğitim kaybı: 1,720 · 0.60 Eğitim kaybı: 1,740 · 0.55 Eğitim kaybı: 1,760 · 0.56 Eğitim kaybı: 1,780 · 0.57 Eğitim kaybı: 1,800 · 0.58 Eğitim kaybı: 1,820 · 0.53 Eğitim kaybı: 1,840 · 0.56 Eğitim kaybı: 1,860 · 0.56 Eğitim kaybı: 1,880 · 0.57 Eğitim kaybı: 1,900 · 0.57 Doğrulama kaybı: 1,000 · 0.21 Doğrulama kaybı: 1,000 · 0.21

Her nokta ardışık 2 kaydın ortalaması.

Show values as a table
Eğitim kaybı
201.9822
1001.0276
1800.9538
2600.8646
3400.7571
4200.8297
5000.7338
5800.7818
6600.6963
7400.6748
8200.6469
9000.6656
9800.6455
1,0600.6512
1,1400.6755
1,2200.5981
1,3000.5790
1,3800.5855
1,4600.6023
1,5400.5664
1,6200.6216
1,7000.6044
1,7800.5672
1,8600.5622
1,9000.5678
Doğrulama kaybı
1,0000.2118
Doğrulama kaybı
0.125 0.150 0.175 0.200 0.225 1,000 1,200 1,400 1,600 1,800 2,000 Adım Doğrulama kaybı Doğrulama kaybı: 1,000 · 0.212 Doğrulama kaybı: 1,000 · 0.212 Doğrulama kaybı: 2,000 · 0.130 Doğrulama kaybı: 2,000 · 0.130

Eğitimde hiç görülmemiş örneklerle ölçüldü; düşmesi modelin ezberlemeden öğrendiğini gösterir.

Show values as a table
Doğrulama kaybı
1,0000.2118
2,0000.1301

Eğitim öncesiyle kıyas

Bu modelin ilk sürümü; önceki sürüm yok. Eğitimden önceki ve sonraki hâli aynı sınavlarla kıyaslanacak.

Open technical sheet
CALLAII® by CALLMENTA

CALLAII ASR

Speech recognition

A speech recognition model that separates call recordings into speaker channels and converts them to text. Analysis, coaching, and quality scoring operate on this transcription.

CALLAII ASR v4 Live Released September 2026

Splits call recordings into speaker channels and converts them to text. This version was trained on audio mimicking telephone line conditions across ten languages; error rates in call center conversations dropped significantly.

What improved in this version

  • Turkish word error rate in call center scenarios dropped from 25.70% to 6.32%
  • Error rates in Arabic, German, French, English, and Italian fell to one-quarter and one-fifth respectively; per-language results shown in the chart
  • Improved in read Turkish speech as well: 4.72% → 4.47% (800 recordings)
  • Trained on telephone line, noisy environment, and studio versions of the same text
  • Previous version's data was also included in training; new data was added without degrading learned knowledge

What it does

  • Transcribes customer and agent separately in two-channel recordings
  • Detects each channel's language independently
  • Produces timestamped, speaker-labeled transcripts
  • Leaves silent sections blank; trained to avoid generating fabricated sentences

Limitations and responsible use

  • Measurements were performed on test sets; reference-based error rate on real phone calls has not yet been measured
  • Error increases with overlapping speech and very low-quality recordings
  • Real-time transcription latency was not measured

Training and measurements

19,369Step
1Epoch
0.0981Final validation loss
9.41%Final word error rate
2.82%Final character error rate
Training data2,479,180 recordings, 3,654 hours of audio
New data share26.5%; the remainder is from the previous version
LanguagesTurkish, Arabic, German, French, Kazakh, Italian, English, Bashkir, Uzbek, Tatar
Audio typestelephone line simulation, noisy environment, studio
Validation set11,859 recordings
Steps19,369 · 1 epoch
Training dateSeptember 2026
Training loss
0 5 10 15 5,000 10,000 15,000 Step Training loss Training loss: 175 · 10.81 Training loss: 350 · 8.66 Training loss: 525 · 7.60 Training loss: 700 · 7.07 Training loss: 875 · 6.61 Training loss: 1,050 · 6.46 Training loss: 1,225 · 6.17 Training loss: 1,400 · 6.18 Training loss: 1,575 · 5.93 Training loss: 1,750 · 5.77 Training loss: 1,925 · 5.65 Training loss: 2,100 · 5.73 Training loss: 2,275 · 5.54 Training loss: 2,450 · 5.41 Training loss: 2,625 · 5.55 Training loss: 2,800 · 5.33 Training loss: 2,975 · 5.34 Training loss: 3,150 · 5.20 Training loss: 3,325 · 5.12 Training loss: 3,500 · 5.12 Training loss: 3,675 · 5.02 Training loss: 3,850 · 4.99 Training loss: 4,025 · 4.94 Training loss: 4,200 · 4.99 Training loss: 4,375 · 4.91 Training loss: 4,550 · 4.74 Training loss: 4,725 · 4.77 Training loss: 4,900 · 4.85 Training loss: 5,075 · 4.82 Training loss: 5,250 · 4.71 Training loss: 5,425 · 4.61 Training loss: 5,600 · 4.58 Training loss: 5,775 · 4.61 Training loss: 5,950 · 4.55 Training loss: 6,125 · 4.63 Training loss: 6,300 · 4.61 Training loss: 6,475 · 4.57 Training loss: 6,650 · 4.54 Training loss: 6,825 · 4.53 Training loss: 7,000 · 4.50 Training loss: 7,175 · 4.50 Training loss: 7,350 · 4.46 Training loss: 7,525 · 4.53 Training loss: 7,700 · 4.44 Training loss: 7,875 · 4.41 Training loss: 8,050 · 4.39 Training loss: 8,225 · 4.45 Training loss: 8,400 · 4.41 Training loss: 8,575 · 4.40 Training loss: 8,750 · 4.43 Training loss: 8,925 · 4.44 Training loss: 9,100 · 4.28 Training loss: 9,275 · 4.34 Training loss: 9,450 · 4.24 Training loss: 9,625 · 4.20 Training loss: 9,800 · 4.34 Training loss: 9,975 · 4.30 Training loss: 10,150 · 4.28 Training loss: 10,325 · 4.29 Training loss: 10,500 · 4.22 Training loss: 10,675 · 4.23 Training loss: 10,850 · 4.21 Training loss: 11,025 · 4.26 Training loss: 11,200 · 4.28 Training loss: 11,375 · 4.23 Training loss: 11,550 · 4.18 Training loss: 11,725 · 4.15 Training loss: 11,900 · 4.15 Training loss: 12,075 · 4.10 Training loss: 12,250 · 4.16 Training loss: 12,425 · 4.28 Training loss: 12,600 · 4.22 Training loss: 12,775 · 4.27 Training loss: 12,950 · 4.21 Training loss: 13,125 · 4.19 Training loss: 13,300 · 4.16 Training loss: 13,475 · 4.10 Training loss: 13,650 · 4.15 Training loss: 13,825 · 4.20 Training loss: 14,000 · 4.12 Training loss: 14,175 · 4.15 Training loss: 14,350 · 4.16 Training loss: 14,525 · 4.14 Training loss: 14,700 · 4.13 Training loss: 14,875 · 4.12 Training loss: 15,050 · 4.17 Training loss: 15,225 · 3.98 Training loss: 15,400 · 4.18 Training loss: 15,575 · 4.07 Training loss: 15,750 · 4.17 Training loss: 15,925 · 4.08 Training loss: 16,100 · 4.06 Training loss: 16,275 · 4.12 Training loss: 16,450 · 4.15 Training loss: 16,625 · 4.07 Training loss: 16,800 · 4.00 Training loss: 16,975 · 4.13 Training loss: 17,150 · 4.04 Training loss: 17,325 · 4.11 Training loss: 17,500 · 4.15 Training loss: 17,675 · 4.05 Training loss: 17,850 · 4.04 Training loss: 18,025 · 4.13 Training loss: 18,200 · 4.06 Training loss: 18,375 · 4.10 Training loss: 18,550 · 4.20 Training loss: 18,725 · 3.94 Training loss: 18,900 · 4.06 Training loss: 19,075 · 4.09 Training loss: 19,250 · 4.07 Training loss: 19,350 · 4.02

Each point is the average of 7 consecutive recordings.

Show values as a table
Training loss
17510.8073
1,0506.4649
1,9255.6508
2,8005.3350
3,6755.0206
4,5504.7431
5,4254.6104
6,3004.6143
7,1754.5038
8,0504.3882
8,9254.4358
9,8004.3362
10,6754.2276
11,5504.1843
12,4254.2788
13,3004.1626
14,1754.1483
15,0504.1727
15,9254.0753
16,8004.0031
17,6754.0504
18,5504.2040
19,3504.0197
Validation loss
0.00 0.05 0.10 0.15 0.20 5,000 10,000 15,000 Step Validation loss Validation loss: 1,000 · 0.19 Validation loss: 1,000 · 0.19 Validation loss: 2,000 · 0.16 Validation loss: 2,000 · 0.16 Validation loss: 3,000 · 0.14 Validation loss: 3,000 · 0.14 Validation loss: 4,000 · 0.13 Validation loss: 4,000 · 0.13 Validation loss: 5,000 · 0.13 Validation loss: 5,000 · 0.13 Validation loss: 6,000 · 0.12 Validation loss: 6,000 · 0.12 Validation loss: 7,000 · 0.11 Validation loss: 7,000 · 0.11 Validation loss: 8,000 · 0.11 Validation loss: 8,000 · 0.11 Validation loss: 9,000 · 0.11 Validation loss: 9,000 · 0.11 Validation loss: 10,000 · 0.10 Validation loss: 10,000 · 0.10 Validation loss: 11,000 · 0.10 Validation loss: 11,000 · 0.10 Validation loss: 12,000 · 0.10 Validation loss: 12,000 · 0.10 Validation loss: 13,000 · 0.10 Validation loss: 13,000 · 0.10 Validation loss: 14,000 · 0.10 Validation loss: 14,000 · 0.10 Validation loss: 15,000 · 0.10 Validation loss: 15,000 · 0.10 Validation loss: 16,000 · 0.10 Validation loss: 16,000 · 0.10 Validation loss: 17,000 · 0.10 Validation loss: 17,000 · 0.10 Validation loss: 18,000 · 0.10 Validation loss: 18,000 · 0.10 Validation loss: 19,000 · 0.10 Validation loss: 19,000 · 0.10 Validation loss: 19,369 · 0.10 Validation loss: 19,369 · 0.10

Measured on examples never seen in training; a falling curve shows the model learns rather than memorises.

Show values as a table
Validation loss
1,0000.1911
2,0000.1611
3,0000.1446
4,0000.1345
5,0000.1259
6,0000.1187
7,0000.1135
8,0000.1097
9,0000.1068
10,0000.1050
11,0000.1025
12,0000.1021
13,0000.1010
14,0000.0997
15,0000.0994
16,0000.0985
17,0000.0985
18,0000.0984
19,0000.0980
19,3690.0981
Error rate on validation (%)
Word error rate (WER) Character error rate (CER)
0% 5% 10% 15% 20% 5,000 10,000 15,000 Step Word error rate (WER) Character error rate (CER) Word error rate (WER): 1,000 · 15.03% Word error rate (WER): 1,000 · 15.03% Word error rate (WER): 2,000 · 13.22% Word error rate (WER): 2,000 · 13.22% Word error rate (WER): 3,000 · 12.06% Word error rate (WER): 3,000 · 12.06% Word error rate (WER): 4,000 · 10.78% Word error rate (WER): 4,000 · 10.78% Word error rate (WER): 5,000 · 10.51% Word error rate (WER): 5,000 · 10.51% Word error rate (WER): 6,000 · 10.01% Word error rate (WER): 6,000 · 10.01% Word error rate (WER): 7,000 · 9.60% Word error rate (WER): 7,000 · 9.60% Word error rate (WER): 8,000 · 9.72% Word error rate (WER): 8,000 · 9.72% Word error rate (WER): 9,000 · 9.52% Word error rate (WER): 9,000 · 9.52% Word error rate (WER): 10,000 · 9.43% Word error rate (WER): 10,000 · 9.43% Word error rate (WER): 11,000 · 9.36% Word error rate (WER): 11,000 · 9.36% Word error rate (WER): 12,000 · 9.35% Word error rate (WER): 12,000 · 9.35% Word error rate (WER): 13,000 · 9.38% Word error rate (WER): 13,000 · 9.38% Word error rate (WER): 14,000 · 9.52% Word error rate (WER): 14,000 · 9.52% Word error rate (WER): 15,000 · 9.23% Word error rate (WER): 15,000 · 9.23% Word error rate (WER): 16,000 · 9.36% Word error rate (WER): 16,000 · 9.36% Word error rate (WER): 17,000 · 9.35% Word error rate (WER): 17,000 · 9.35% Word error rate (WER): 18,000 · 9.36% Word error rate (WER): 18,000 · 9.36% Word error rate (WER): 19,000 · 9.24% Word error rate (WER): 19,000 · 9.24% Word error rate (WER): 19,369 · 9.41% Word error rate (WER): 19,369 · 9.41% Character error rate (CER): 1,000 · 4.24% Character error rate (CER): 1,000 · 4.24% Character error rate (CER): 2,000 · 3.85% Character error rate (CER): 2,000 · 3.85% Character error rate (CER): 3,000 · 3.47% Character error rate (CER): 3,000 · 3.47% Character error rate (CER): 4,000 · 3.13% Character error rate (CER): 4,000 · 3.13% Character error rate (CER): 5,000 · 3.01% Character error rate (CER): 5,000 · 3.01% Character error rate (CER): 6,000 · 2.94% Character error rate (CER): 6,000 · 2.94% Character error rate (CER): 7,000 · 2.87% Character error rate (CER): 7,000 · 2.87% Character error rate (CER): 8,000 · 2.89% Character error rate (CER): 8,000 · 2.89% Character error rate (CER): 9,000 · 2.89% Character error rate (CER): 9,000 · 2.89% Character error rate (CER): 10,000 · 2.85% Character error rate (CER): 10,000 · 2.85% Character error rate (CER): 11,000 · 2.88% Character error rate (CER): 11,000 · 2.88% Character error rate (CER): 12,000 · 2.81% Character error rate (CER): 12,000 · 2.81% Character error rate (CER): 13,000 · 2.92% Character error rate (CER): 13,000 · 2.92% Character error rate (CER): 14,000 · 2.93% Character error rate (CER): 14,000 · 2.93% Character error rate (CER): 15,000 · 2.80% Character error rate (CER): 15,000 · 2.80% Character error rate (CER): 16,000 · 2.83% Character error rate (CER): 16,000 · 2.83% Character error rate (CER): 17,000 · 2.77% Character error rate (CER): 17,000 · 2.77% Character error rate (CER): 18,000 · 2.80% Character error rate (CER): 18,000 · 2.80% Character error rate (CER): 19,000 · 2.72% Character error rate (CER): 19,000 · 2.72% Character error rate (CER): 19,369 · 2.82% Character error rate (CER): 19,369 · 2.82%

Lower is better. Measured on the validation set throughout training.

Show values as a table
Word error rate (WER)
1,00015.03
2,00013.22
3,00012.06
4,00010.78
5,00010.51
6,00010.01
7,0009.60
8,0009.72
9,0009.52
10,0009.43
11,0009.36
12,0009.35
13,0009.38
14,0009.52
15,0009.23
16,0009.36
17,0009.35
18,0009.36
19,0009.24
19,3699.41
Character error rate (CER)
1,0004.24
2,0003.85
3,0003.47
4,0003.13
5,0003.01
6,0002.94
7,0002.87
8,0002.89
9,0002.89
10,0002.85
11,0002.88
12,0002.81
13,0002.92
14,0002.93
15,0002.80
16,0002.83
17,0002.77
18,0002.80
19,0002.72
19,3692.82
Call center evaluation · Per-language word error rate (%)
v3 v4
0% 20% 40% 60% 80% Arabic German English French Italian Turkish Arabic · v3: 67.6% 67.6% Arabic · v4: 13.9% 13.9% German · v3: 54.6% 54.6% German · v4: 15.4% 15.4% English · v3: 59.9% 59.9% English · v4: 12.6% 12.6% French · v3: 62.4% 62.4% French · v4: 15.6% 15.6% Italian · v3: 38.9% 38.9% Italian · v4: 8.4% 8.4% Turkish · v3: 25.7% 25.7% Turkish · v4: 6.3% 6.3%

Lower is better. 300 samples per language.

Show values as a table
v3v4
Arabic67.5913.86
German54.5815.37
English59.9212.58
French62.3615.57
Italian38.888.42
Turkish25.706.32
Call center evaluation · Per-language character error rate (%)
v3 v4
0% 20% 40% 60% Arabic German English French Italian Turkish Arabic · v3: 48.6% 48.6% Arabic · v4: 5.8% 5.8% German · v3: 46.0% 46.0% German · v4: 7.2% 7.2% English · v3: 44.4% 44.4% English · v4: 6.2% 6.2% French · v3: 47.9% 47.9% French · v4: 7.3% 7.3% Italian · v3: 26.9% 26.9% Italian · v4: 3.5% 3.5% Turkish · v3: 10.5% 10.5% Turkish · v4: 2.0% 2.0%

Lower is better. 300 samples per language.

Show values as a table
v3v4
Arabic48.625.76
German46.007.23
English44.416.23
French47.927.34
Italian26.923.45
Turkish10.471.96
Read Turkish speech · Per-language word error rate (%)
v3 v4
0% 2% 4% 6% Turkish Turkish · v3: 4.7% 4.7% Turkish · v4: 4.5% 4.5%

Lower is better. 800 samples per language.

Show values as a table
v3v4
Turkish4.724.47
Read Turkish speech · Character error rate by language (%)
v3 v4
0.0% 0.5% 1.0% 1.5% Turkish Turkish · v3: 1.1% 1.1% Turkish · v4: 1.0% 1.0%

Lower is better. 800 samples per language.

Show values as a table
v3v4
Turkish1.110.95

Comparison with open models

CALLAII ASR was measured on the same Turkish audio clips using the same methodology as widely used open speech recognition models.

October 8, 2026Measurement date
1,000Number of clips
3Compared model
  • In word error rate, CALLAII ASR recorded the lowest value in all four tests.
  • In character error rate, the difference between read speech and phone line is 0.04 points; the models are at the same level.
  • The training portion of the volunteer recordings is included in CALLAII ASR training; the portion used in testing is not.
  • The phone line condition increases error across all models. This is the focus of the next version.
  • Reference-based measurement on real phone conversations has not yet been performed.
Word error rate comparison (%)
Qwen3-ASR 1.7B Whisper large-v3 CALLAII ASR
0% 10% 20% 30% 40% Read speech Telephone line Volunteer recordin… Call scenarios Read speech · Qwen3-ASR 1.7B: 9.8% 9.8% Read speech · Whisper large-v3: 7.0% 7.0% Read speech · CALLAII ASR: 6.7% 6.7% Telephone line · Qwen3-ASR 1.7B: 11.3% 11.3% Telephone line · Whisper large-v3: 8.3% 8.3% Telephone line · CALLAII ASR: 8.0% 8.0% Volunteer recordings · Qwen3-ASR 1.7B: 15.8% 15.8% Volunteer recordings · Whisper large-v3: 13.1% 13.1% Volunteer recordings · CALLAII ASR: 4.1% 4.1% Call scenarios · Qwen3-ASR 1.7B: 30.7% 30.7% Call scenarios · Whisper large-v3: 22.3% 22.3% Call scenarios · CALLAII ASR: 4.5% 4.5%

Lower is better. Same 1,000 Turkish clips and same measurement method. Read speech and telephone line: FLEURS; telephone line uses the same clips under 8 kHz telephone conditions. Volunteer recordings: Common Voice. Call scenarios: sentences prepared from call center dialogues; CALLAII ASR was trained on this domain's data. Measurement: October 8, 2026.

Show values as a table
Qwen3-ASR 1.7BWhisper large-v3CALLAII ASR
Read speech9.757.046.67
Telephone line11.338.328.03
Volunteer recordings15.8013.144.10
Call scenarios30.7422.334.45
Character error rate comparison (%)
Qwen3-ASR 1.7B Whisper large-v3 CALLAII ASR
0% 5% 10% 15% Read speech Phone line Volunteer recordin… Call scenarios Read speech · Qwen3-ASR 1.7B: 2.4% 2.4% Read speech · Whisper large-v3: 1.6% 1.6% Read speech · CALLAII ASR: 1.6% 1.6% Phone line · Qwen3-ASR 1.7B: 3.0% 3.0% Phone line · Whisper large-v3: 1.9% 1.9% Phone line · CALLAII ASR: 2.0% 2.0% Volunteer recordings · Qwen3-ASR 1.7B: 4.4% 4.4% Volunteer recordings · Whisper large-v3: 4.5% 4.5% Volunteer recordings · CALLAII ASR: 1.0% 1.0% Call scenarios · Qwen3-ASR 1.7B: 12.0% 12.0% Call scenarios · Whisper large-v3: 8.0% 8.0% Call scenarios · CALLAII ASR: 1.0% 1.0%

Lower is better. Same 1,000 Turkish clips and same measurement method. Read speech and telephone line: FLEURS; telephone line uses the same clips under 8 kHz telephone conditions. Volunteer recordings: Common Voice. Call scenarios: sentences prepared from call center dialogues; CALLAII ASR was trained on this domain's data. Measurement: October 8, 2026.

Show values as a table
Qwen3-ASR 1.7BWhisper large-v3CALLAII ASR
Read speech2.431.561.60
Phone line3.041.941.98
Volunteer recordings4.444.530.98
Call scenarios12.027.971.00

Technical sheet

Versionv4
ReleaseSeptember 8, 2026
InputAudio recording, single or dual channel
OutputSpeaker-labeled, timestamped text
Language detectionAutomatic per channel
Runs onServers managed by Callmenta
Open technical sheet

Previous versions

CALLAII ASR v3 · Previous version
CALLAII ASR v3 Previous version Released August 2026

Version with extensive multilingual training. Phone line simulation and silence detection were added in this version.

What improved in this version

  • Turkish word error rate dropped from 8.87% to 7.11% on the open benchmark set
  • Arabic dropped from 7.96% to 7.82%
  • Learned to leave silent sections blank; hallucinated sentences decreased
  • No degradation observed at the end of long calls (571-second real call)
  • Automatic language detection per channel has begun

Limitations and responsible use

  • The open benchmark set consists of read news sentences; it does not represent absolute success in phone calls
  • Error rates were high in languages other than Turkish in call center scenarios; resolved in v4

Training and measurements

11,540Step
1Epoch
0.1315Final validation loss
10.49%Final word error rate
3.01%Final character error rate
Training data1,477,006 records, 2,190.9 hours of audio
Silence samples25.7 hours
Audio typesphone band simulation (8 kHz), cross-mixing, dialogue segmentation
Steps11,540 · 1 epoch
Training dateAugust 2026
Training loss
0 1 2 3 2,500 5,000 7,500 10,000 Step Training loss Training loss: 100 · 2.63 Training loss: 200 · 1.90 Training loss: 300 · 1.80 Training loss: 400 · 1.82 Training loss: 500 · 1.72 Training loss: 600 · 1.67 Training loss: 700 · 1.67 Training loss: 800 · 1.68 Training loss: 900 · 1.67 Training loss: 1,000 · 1.66 Training loss: 1,100 · 1.60 Training loss: 1,200 · 1.57 Training loss: 1,300 · 1.54 Training loss: 1,400 · 1.56 Training loss: 1,500 · 1.56 Training loss: 1,600 · 1.56 Training loss: 1,700 · 1.53 Training loss: 1,800 · 1.50 Training loss: 1,900 · 1.52 Training loss: 2,000 · 1.51 Training loss: 2,100 · 1.52 Training loss: 2,200 · 1.47 Training loss: 2,300 · 1.48 Training loss: 2,400 · 1.45 Training loss: 2,500 · 1.44 Training loss: 2,600 · 1.43 Training loss: 2,700 · 1.45 Training loss: 2,800 · 1.44 Training loss: 2,900 · 1.44 Training loss: 3,000 · 1.42 Training loss: 3,100 · 1.41 Training loss: 3,200 · 1.46 Training loss: 3,300 · 1.37 Training loss: 3,400 · 1.41 Training loss: 3,500 · 1.46 Training loss: 3,600 · 1.35 Training loss: 3,700 · 1.37 Training loss: 3,800 · 1.35 Training loss: 3,900 · 1.39 Training loss: 4,000 · 1.40 Training loss: 4,100 · 1.38 Training loss: 4,200 · 1.38 Training loss: 4,300 · 1.36 Training loss: 4,400 · 1.38 Training loss: 4,500 · 1.34 Training loss: 4,600 · 1.35 Training loss: 4,700 · 1.36 Training loss: 4,800 · 1.31 Training loss: 4,900 · 1.30 Training loss: 5,000 · 1.34 Training loss: 5,100 · 1.33 Training loss: 5,200 · 1.31 Training loss: 5,300 · 1.30 Training loss: 5,400 · 1.31 Training loss: 5,500 · 1.33 Training loss: 5,600 · 1.33 Training loss: 5,700 · 1.27 Training loss: 5,800 · 1.32 Training loss: 5,900 · 1.33 Training loss: 6,000 · 1.31 Training loss: 6,100 · 1.30 Training loss: 6,200 · 1.27 Training loss: 6,300 · 1.26 Training loss: 6,400 · 1.33 Training loss: 6,500 · 1.29 Training loss: 6,600 · 1.29 Training loss: 6,700 · 1.31 Training loss: 6,800 · 1.22 Training loss: 6,900 · 1.25 Training loss: 7,000 · 1.31 Training loss: 7,100 · 1.27 Training loss: 7,200 · 1.23 Training loss: 7,300 · 1.26 Training loss: 7,400 · 1.27 Training loss: 7,500 · 1.25 Training loss: 7,600 · 1.26 Training loss: 7,700 · 1.24 Training loss: 7,800 · 1.25 Training loss: 7,900 · 1.24 Training loss: 8,000 · 1.24 Training loss: 8,100 · 1.26 Training loss: 8,200 · 1.23 Training loss: 8,300 · 1.24 Training loss: 8,400 · 1.24 Training loss: 8,500 · 1.23 Training loss: 8,600 · 1.27 Training loss: 8,700 · 1.23 Training loss: 8,800 · 1.22 Training loss: 8,900 · 1.20 Training loss: 9,000 · 1.22 Training loss: 9,100 · 1.22 Training loss: 9,200 · 1.22 Training loss: 9,300 · 1.24 Training loss: 9,400 · 1.23 Training loss: 9,500 · 1.21 Training loss: 9,600 · 1.23 Training loss: 9,700 · 1.18 Training loss: 9,800 · 1.23 Training loss: 9,900 · 1.21 Training loss: 10,000 · 1.18 Training loss: 10,100 · 1.22 Training loss: 10,200 · 1.21 Training loss: 10,300 · 1.26 Training loss: 10,400 · 1.20 Training loss: 10,500 · 1.20 Training loss: 10,600 · 1.21 Training loss: 10,700 · 1.21 Training loss: 10,800 · 1.23 Training loss: 10,900 · 1.17 Training loss: 11,000 · 1.19 Training loss: 11,100 · 1.18 Training loss: 11,200 · 1.18 Training loss: 11,300 · 1.20 Training loss: 11,400 · 1.20 Training loss: 11,500 · 1.17

Each point is the average of two consecutive records.

Show values as a table
Training loss
1002.6340
6001.6663
1,1001.6001
1,6001.5598
2,1001.5177
2,6001.4338
3,1001.4080
3,6001.3531
4,1001.3821
4,6001.3530
5,1001.3306
5,6001.3333
6,1001.3021
6,6001.2923
7,1001.2736
7,6001.2613
8,1001.2606
8,6001.2679
9,1001.2166
9,6001.2256
10,1001.2165
10,6001.2068
11,1001.1835
11,5001.1665
Validation loss
0.13 0.14 0.15 0.16 0.17 2,000 4,000 6,000 8,000 10,000 Step Validation loss Validation loss: 2,000 · 0.16 Validation loss: 2,000 · 0.16 Validation loss: 4,000 · 0.15 Validation loss: 4,000 · 0.15 Validation loss: 6,000 · 0.15 Validation loss: 6,000 · 0.15 Validation loss: 8,000 · 0.14 Validation loss: 8,000 · 0.14 Validation loss: 10,000 · 0.13 Validation loss: 10,000 · 0.13 Validation loss: 11,540 · 0.13 Validation loss: 11,540 · 0.13

Measured on examples never seen in training; a falling curve shows the model learns rather than memorises.

Show values as a table
Validation loss
2,0000.1633
4,0000.1528
6,0000.1456
8,0000.1383
10,0000.1340
11,5400.1316
Error rate on validation (%)
Word error rate (WER) Character error rate (CER)
0% 5% 10% 15% 20% 2,000 4,000 6,000 8,000 10,000 Step Word error rate (WER) Character error rate (CER) Word error rate (WER): 2,000 · 12.60% Word error rate (WER): 2,000 · 12.60% Word error rate (WER): 4,000 · 11.94% Word error rate (WER): 4,000 · 11.94% Word error rate (WER): 6,000 · 19.10% Word error rate (WER): 6,000 · 19.10% Word error rate (WER): 8,000 · 10.71% Word error rate (WER): 8,000 · 10.71% Word error rate (WER): 10,000 · 18.35% Word error rate (WER): 10,000 · 18.35% Word error rate (WER): 11,540 · 10.49% Word error rate (WER): 11,540 · 10.49% Character error rate (CER): 2,000 · 3.64% Character error rate (CER): 2,000 · 3.64% Character error rate (CER): 4,000 · 3.33% Character error rate (CER): 4,000 · 3.33% Character error rate (CER): 6,000 · 10.35% Character error rate (CER): 6,000 · 10.35% Character error rate (CER): 8,000 · 2.94% Character error rate (CER): 8,000 · 2.94% Character error rate (CER): 10,000 · 10.79% Character error rate (CER): 10,000 · 10.79% Character error rate (CER): 11,540 · 3.01% Character error rate (CER): 11,540 · 3.01%

Lower is better. Measured on the validation set throughout training.

Show values as a table
Word error rate (WER)
2,00012.60
4,00011.94
6,00019.10
8,00010.71
10,00018.35
11,54010.49
Character error rate (CER)
2,0003.64
4,0003.33
6,00010.35
8,0002.94
10,00010.79
11,5403.01
Open benchmark set (FLEURS) · Word error rate (%)
v2 v3
0.0% 2.5% 5.0% 7.5% 10.0% Turkish Arabic Turkish · v2: 8.9% 8.9% Turkish · v3: 7.1% 7.1% Arabic · v2: 8.0% 8.0% Arabic · v3: 7.8% 7.8%

Lower is better. 150 records per language, read news sentences. Valid for ranking; does not represent absolute success in phone calls.

Show values as a table
v2v3
Turkish8.877.11
Arabic7.967.82
Open benchmark set (FLEURS) · Character error rate (%)
v2 v3
0% 1% 2% 3% Turkish Arabic Turkish · v2: 2.8% 2.8% Turkish · v3: 1.7% 1.7% Arabic · v2: 2.7% 2.7% Arabic · v3: 2.5% 2.5%

Lower is better. 150 records per language.

Show values as a table
v2v3
Turkish2.831.67
Arabic2.652.46
Internal test set · Language-based word error rate (%)
0% 5% 10% 15% 20% Arabic German English French Italian Turkish Arabic · v3: 19.7% 19.7% German · v3: 6.8% 6.8% English · v3: 19.1% 19.1% French · v3: 12.7% 12.7% Italian · v3: 8.1% 8.1% Turkish · v3: 10.5% 10.5%

Lower is better. 200 samples per language.

Show values as a table
v3
Arabic19.72
German6.80
English19.10
French12.74
Italian8.11
Turkish10.49
Internal test set · Language-based character error rate (%)
0% 5% 10% 15% Arabic German English French Italian Turkish Arabic · v3: 7.4% 7.4% German · v3: 2.0% 2.0% English · v3: 11.9% 11.9% French · v3: 3.8% 3.8% Italian · v3: 2.2% 2.2% Turkish · v3: 3.0% 3.0%

Lower is better. 200 samples per language.

Show values as a table
v3
Arabic7.41
German1.95
English11.89
French3.75
Italian2.22
Turkish3.01

Technical sheet

Versionv3
ReleaseAugust 25, 2026
StatusSuperseded by v4
Open technical sheet
CALLAII ASR v2 · Previous version
CALLAII ASR v2 Previous version Released June 2026

Second version adapted to call center Turkish. The error rate on real recordings dropped to half of the first version.

What improved in this version

  • Word error rate on 10 real recordings dropped from %14.25 to %6.51
  • Character error rate dropped from %8.68 to %4.57
  • Domain prompt added for call center expressions (announcements and institution names)

Limitations and responsible use

  • Could enter a repetition loop at the end of long calls; fixed in v3
  • Output was corrupted when the language label was incorrectly assigned in Arabic recordings; fixed in v3 with automatic language detection

Training and measurements

Turkish validationword error rate %7.02, character error rate %2.15
Error rate (%) on 10 real recordings
v1 v2
0% 5% 10% 15% Word error rate (W… Character error ra… Word error rate (WER) · v1: 14.3% 14.3% Word error rate (WER) · v2: 6.5% 6.5% Character error rate (CER) · v1: 8.7% 8.7% Character error rate (CER) · v2: 4.6% 4.6%

Lower is better. The same 10 real call recordings were fed to both versions.

Show values as a table
v1v2
Word error rate (WER)14.256.51
Character error rate (CER)8.684.57

Technical sheet

Versionv2
ReleaseJune 2026
StatusReplaced by v3
Open technical sheet
CALLAII ASR v1 · Previous version
CALLAII ASR v1 Previous version Released April 2026

First CALLAII ASR version: The initial model that converts Turkish call recordings into text.

Technical sheet

Versionv1
ReleaseApril 2026
StatusReplaced by v2
Open technical sheet

Product release notes