CoIn: Counting the Invisible Reasoning Tokens in Commercial Opaque LLM APIs
Guoheng Sun, Ziyao Wang, Bowei Tian, Meng Liu, Zheyu Shen, Shwai He, Yexiao He, Wanghao Ye, Yiting Wang, Ang Li
2025-05-21
Summary
This paper talks about a tool called CoIn that helps people check how many hidden 'reasoning tokens' are being used by commercial AI language models, which is important for understanding what you're actually paying for.
What's the problem?
When companies use AI models through paid services, it's hard to see exactly how much of the model's work is being used because some of the processing is hidden, making it unclear if the billing is fair.
What's the solution?
The researchers created CoIn, a system that can audit and verify the number and type of these invisible tokens, so users can be sure they're being charged correctly and that the AI is really doing what it claims.
Why it matters?
This matters because it helps build trust between AI companies and their customers, making sure people aren't overcharged and that the AI services are as transparent and honest as possible.
Abstract
A verification framework named CoIn is proposed to audit and validate the quantity and authenticity of hidden tokens in large language models to ensure billing transparency.