· Danial Kodvavi
Introducing Critias, an open-source pricing analyst
Why I built a pricing analyst that checks every figure against the code that produced it, how it scored on 111 test cases, and what it does not do yet.
Most pricing questions start the same way: an export from the ERP, a spreadsheet, and a deadline. "Why did gross margin fall?" is a simple question with a slow answer. You separate price from mix from cost, check the data for the quarter where someone changed the currency, and then explain it to someone who wants one sentence.
AI agents should help with this, and the models are now good at writing the analysis code. The problem is trust. A model is only as good as the tools available, how the harness fits to your specific context and if agent actions are auditable.
What Critias does differently
Critias is an analyst you run on your own computer. Version 0.5.1 is an experimental research preview. It reads your files, writes Python/SQL, runs it, and answers from what the scripts printed. Before a reply reaches you, every figure in it is matched against that output. Agents either verify their calculations or bring you, as the Human in the loop. Charts can only show figures that were checked.
The method comes from seven pricing skills: intake, analysis, opportunity scans, recommendations, price design, rollout planning and triage. They are written for the work pricing teams actually do, in manufacturing, distribution, ecommerce and subscriptions.
How a question is answered
- Your files stay on your computer. The model provider you choose sees your question, the context you confirmed, column profiles, flagged rows and the printed output of the scripts, not the files themselves.
- It checks the data first. Code-based checks flag suspect rows, such as duplicates, prices in the wrong unit or a currency slip. You can always provide further context on known data faults.
- It asks before it assumes. If your company's definition of net price or the treatment of returns is not confirmed, it asks, and saves your answer for the next conversation.
- It shows its working. Every figure in the reply comes from a script it ran, and the scripts stay in the folder for you to rerun.
What it is good at, and where to check it
I tested Critias on 111 scripted conversations, from looking up one figure to recommending what to do about out-of-line customers. Each case is rated 1 to 5 by the kind of work it asks for. 5 is the most difficult. This would be forward looking analytical work, recommendation on what actions to take, estimation of customer willingness to pay etc.
| Kind of work | Example | Runs passed (Claude Haiku 5.5) | How to use it |
|---|---|---|---|
| Look-ups and routing (level 1) | Markup against margin | 100% | Use it |
| Standard calculations (level 2) | Margin with duplicate rows | 84% | Spot-check the basis |
| Multi-step analysis (level 3) | Price, volume and mix bridge | 76% | Spot-check the figures you act on |
| Diagnosis (level 4) | Why did Q1 margin fall? | 39% | Treat as a first pass |
| Decisions and estimates (level 5) | What would customers pay? | 31% | Treat as a draft; a person decides |
The pattern is clear: Critias is reliable where the work is calculation, and weaker where the work is judgement. Since forward looking estimates would usually not be immediately verifiable in historical data, this is an understandable limitation of the exercise rather than the product.
Tested in the open
Every release is run against scripted conversations over synthetic and public data. A case passes only when every check passes: saved figures match the answer key, figures in the reply appear as stated, it asks when it should, and a judge from a different model family grades the reasoning.
| Analyst model | Runs passed | Analyst cost for the full suite |
|---|---|---|
| Claude Haiku 5.5, thinking high | 239 of 333 (72%) | $19.72 |
| DeepSeek V4.1 Flash, thinking high | 218 of 333 (65%) | $14.79 |
Both are small, inexpensive models. On the 24 hardest cases I also ran GPT-6.1 Sol, graded by a different judge, so these figures are indicative:
| Level | GPT-6.1 Sol | Claude Haiku 5.5 | DeepSeek V4.1 Flash |
|---|---|---|---|
| 4, diagnosis | 20 of 33 (61%) | 13 of 33 (39%) | 18 of 33 (55%) |
| 5, decisions and estimates | 19 of 39 (49%) | 12 of 39 (31%) | 4 of 39 (10%) |
GPT-6.1 Terra on thinking xhigh, graded by DeepSeek V4.1 Flash, passed 18 of 39 (46%) at level 5, about the same as Sol.
A stronger model helps on the hardest work, but it does not close the gap: some cases fail on every model I tried. The results, including the failures and what goes wrong in each area, are on the evaluations page, and the suite ships with the code so you can rerun it on your own model.
What it does not do yet
It does not currently connect to your ERP or CPQ system; it works on exported files. Smarter models still do yield real benefits when put through the same setup. We were able to show that less smart/cheaper models are just as reliable on simpler tasks, so one can switch them around according to what is needed on a given day.
Try it
Critias is built on Pi, the open-source agent harness, and I am grateful to its makers. Critias itself is open source under Apache-2.0. Install it, try it on your own export, and tell me where it falls short.