Paying more for an AI model does not guarantee better results. DualEntry’s latest accounting benchmark puts Moonshot AI’s Kimi K3.1 Pro ahead of every tested model from OpenAI and Anthropic, although speed and reliability complicate the comparison.
Kimi scored 86.1 percent on the DualEntry Accounting AI Benchmark, placing fourth overall. Google’s Gemini 4 Argon led with 91.1 percent, while Zhipu AI’s GLM-5.3 Flash and Grok 4.7 tied for second at 88.1 percent. Two of the top four models are open-weight, giving businesses another reason to consider alternatives to proprietary AI.
The price comparison is particularly eye-catching. DualEntry reports that DeepSeek V4.1 Flash scored 84.2 percent at an estimated $0.003 per task. Anthropic’s Claude Fable 5.1 scored 85.1 percent at $1.18 per task. Based on those published figures, that is roughly 393 times the cost for a lead of less than one percentage point.
Those are benchmark execution costs, of course. They do not tell a business what it would spend integrating a model, checking its work, or fixing mistakes. Still, a price gap that large deserves attention, especially when the more expensive option barely edges ahead on the headline score.
Kimi offers another example. In its accompanying analysis, DualEntry puts Kimi’s average cost at $0.079 per task, making Claude Fable roughly 15 times as expensive. Kimi also finished ahead of Claude Opus 5.5, which scored 82.2 percent, and OpenAI’s GPT-6 Astra and GPT-6.1 Sol, both at 81.2 percent.
There is a catch, however. Kimi averaged 125.6 seconds per task, compared with 26 seconds for Claude Fable. It also passed 73.3 percent of tasks on all three attempts, below Fable’s 80.2 percent. If a finance team needs predictable results quickly, paying more could still make sense.
Performance also varied depending on the work. Kimi passed every bank reconciliation case, compared with 75 percent for overall leader Gemini 4 Argon. But Kimi’s month-end close score was just 66.7 percent, showing why picking a model based only on its overall ranking could be a mistake.
The scoring needs some explanation, too. DualEntry uses 101 synthetic accounting cases across eight categories, with each model running the full set three times. A case counts as passed when the model passes it in at least two runs. For most case types, a score of 80 percent is enough to pass.
That means the advertised “accuracy” percentage is a task pass rate under the benchmark’s rules. It does not mean every answer counted as a pass was completely correct. A business should not interpret these results as proof that a model can handle its accounting without supervision.
Readers should also consider who is doing the testing. DualEntry sells AI accounting software, so this is a commercial vendor’s evaluation. Synthetic cases can help compare systems under controlled conditions, but they cannot represent every company’s records, policies, and accounting problems.
I would be careful with the “open source” label as well. DualEntry describes these models as open-weight in its blog, which is the terminology used here. Available weights alone should not become a blanket claim about unrestricted use or complete transparency.
For businesses, the useful takeaway is that brand recognition and a higher price do not automatically identify the best model for a particular job. The cheaper options deserve testing. But when money is involved, the cost of checking an answer matters just as much as the cost of generating it.
Support independent tech journalism
NERDS.xyz is independently owned and operated. If you enjoy my coverage of Linux, AI, hardware, cybersecurity, and tech culture, consider supporting the site on Ko-fi.
Support NERDS.xyz
