Benchmarks & evaluation
Financial AI evaluation must cover evidence, calculations, policy, uncertainty, permissions and the quality of the resulting decision record.
Constraint-aware reasoning / system map
RA–01
Evaluation thesis
We decompose the decision path so that failures can be located, compared and corrected. A benchmark should expose what the system retrieved, inferred, calculated, ignored and escalated.
Decomposes a financial response into evidence selection, calculation, policy application, uncertainty and final action.
Evaluation dimensions
Tests domain terminology, ambiguity and instruction following in Turkish financial and banking contexts.
Evaluation dimensions
Measures whether an agent respects permissions, escalates uncertainty and leaves a usable decision record.
Evaluation dimensions
Release standard
What it measures—and explicitly does not measure.
How tasks, labels and edge cases were selected.
What is known about training-data exposure.
Where scores can mislead or fail to transfer.
How changes affect historical comparability.