When enterprises evaluate AI adoption, the items they check first are usually model performance, security, and data integration. But once operations begin, the problem budget owners face first shows up elsewhere. The bill arrives different from what was expected, and it's hard to explain where the discrepancy came from.
Consumption-based billing itself isn't new. Cloud infrastructure has been billed this way for a long time. What's different is the billing unit. Tokens are the unit in which AI models process input and output data; they fluctuate significantly with usage and are difficult to track in real time. On August 4, 2026, the Linux Foundation officially launched the Tokenomics Foundation together with 30 founding members to establish open industry standards, benchmarks, and best practices for AI economics. This means token cost is no longer a marketing topic for a specific vendor but is beginning to be treated as a subject for industry-wide standardization.
This article organizes the approaches available at each layer, based on the definition of AI tokenomics and the cost-control layers published by the Tokenomics Foundation, together with the cases of Cast AI and Splunk.
☑️ What Is AI Tokenomics
AI tokenomics is the discipline of converting energy and capital into AI, efficiently consuming AI services to produce intelligent outcomes, and translating those outcomes into business value. This is the definition published by the Tokenomics Foundation in its August 2026 draft, and the scope it covers spans the entire supply chain — from the point where tokens are generated from hardware to the point where those tokens are consumed and translated into business outcomes.
One premise comes attached to this. A significant portion of AI cost arises not from the tokens themselves but elsewhere — adjacent items such as compute, storage, databases, cache, and engineering labor costs. Even so, the reason tokens are used as the reference unit is that they are the usage unit that consistently connects all these costs.
The Foundation divides AI tokenomics into three areas — production, consumption, and value — with a working group for each. Production is the area of converting energy and capital into AI capacity; consumption is the area of using that acquired capacity well and understanding its cost; and value is the area concerned with what the output actually produced. There is one standard the consumption area proposes: expressing the entire cost breakdown of AI workloads not as cost-per-token but as cost-per-call.
☑️ Why Falling Token Prices and Rising Total Spend Occur Together
The expectation that cost problems will naturally ease as token unit prices continue to fall doesn't match the current situation. Cost per token fell sharply between 2023 and 2025, but that decline has since stopped, and the token prices of new models are actually rising. As a result, AI is becoming the largest and fastest-growing item in enterprise technology budgets.
Consumption itself is also growing rapidly. According to Goldman Sachs Research, global token usage is projected to grow 24-fold between 2026 and 2030. According to S&P Global data, AI infrastructure investment is expected to exceed $1 trillion by 2027, and the inference market is expected to expand from roughly $106 billion in 2025 to $255 billion by 2030.
The reason unit price and total spend move in opposite directions is that the consumption structure itself has changed. This difference is especially pronounced in agentic AI environments.
☑️ How Token Consumption Accumulates in Agentic AI Environments
Unlike conventional software, agentic AI costs don't increase linearly. AI agents that operate autonomously are designed to reason, deploy sub-agents, search databases, call external tools, verify results, and retry failed tasks. Each step consumes tokens, and a single misdirected AI agent can consume resources through repeated calls and re-queries without making progress on the task.
There's an additional structural factor. Because model API calls are stateless, the entire accumulated context is resent at every step. For example, 1,500 tokens of system instructions, 3,000 tokens of tool schema, and 2,500 tokens of retrieved customer data can turn a 20-token user question into 7,000 tokens of input before the model even generates a response. In subsequent calls this accumulates to 10,000, then 13,000, then 17,000 tokens, resulting in substantive work of under 10,000 tokens consuming more than 40,000 tokens.
The difference by application type is also large. A single simple RAG response is around 2,000–10,000 tokens, while a tool-using agent task ranges from 20,000–60,000 tokens, and a long-running research-style agent task ranges from 30,000–100,000 tokens. Since these figures come from different measurement bases — including production trace sampling and various public benchmarks — it's more appropriate to read them as relative differences in scale between types rather than as absolute numbers.
The Foundation has published a notation for classifying this growth pattern. It denotes the size of a request or input as n, the number of model calls per request as k, and agent depth as a, and uses these to distinguish what kind of curve a workload follows. The lowest tier is cases where the model isn't called at all, such as cache hits or static lookups; a linear tier, where there's one model call per request, is presented as the healthy default. From the tier where a request involves multiple calls, cost begins to multiply, and at this point the number of calls is usually invisible. Next comes the tier where an orchestrator spawns sub-agents that in turn call tools, and the final tier is loops with no termination condition. The purpose of this notation is to let organizations check which tier their own workload falls into before the bill arrives.
☑️ At Which Layer Can Token Cost Be Controlled
Once the consumption structure is understood, the next question is where intervention is possible. The Foundation has organized the points where action can be taken into five layers.
The lowest layer, L1, is the silicon layer. This covers which accelerators are used, of which generation, and the associated memory, interconnect, and power. It's the layer that converts energy into tokens, and it determines the floor for the cost-per-token of every layer above it.
L2 is the capacity layer. This is where it's determined how much computing resource is held, where that resource is located, and how much of it is actually doing work. Capacity left idle drives up the effective cost of the work that is actually performed.
L3 is the inference stack layer. This covers the serving engine, KV cache management, batching, and prefill/decode separation. It's the layer with the single largest optimization impact — and the first point where consumption itself is suppressed, rather than simply priced.
L4 is the model and quantization layer. This determines which model is run at what numerical precision. Using a frontier-grade model as-is for trivial tasks is cited as a typical form of waste here.
L5 is the routing and governance layer. This is where it's decided which model handles each request, how budgets and quotas are set, and what caps are placed on agent depth and retries. It's also the only layer where it can be decided that a given request doesn't need to call a model at all.
There's a reason for this order. Each layer inherits the cost and availability of the layer below it, so improvements accumulate upward. That said, the first results generally come from the inference stack layer and the routing/governance layer. A full explanation of all five layers is available on the Tokenomics Foundation's "The Five-Layer Tokenomics Stack" page.
▶ The Five-Layer Tokenomics Stack
Mapping this structure onto actual tools reveals a correspondence. The L2 capacity layer is the domain of infrastructure automation that provisions and reclaims GPU resources on Kubernetes, and this is the layer Cast AI addresses. The L5 routing and governance layer is the domain of deciding which model should handle which request and enforcing limits — and the data needed for that decision is produced by Splunk Agent Observability. Because the lower and upper layers each address a different problem, the two need to be viewed together.
☑️ The Approach at the Capacity Layer: Cast AI
The challenge at the L2 capacity layer is that provisioned resources aren't actually doing work. In customer environments, it's common for specifications sized to a peak measured months earlier to remain unchanged, and in that case the gap between actual usage and that specification becomes cost as-is. According to Cast AI research, enterprise GPU utilization sits around 5%, in which case most of the token cost ends up coming from idle hardware.
Cast AI is a platform that automates this area for AI infrastructure running on Kubernetes, and it participates as a founding member of the Tokenomics Foundation. What sets it apart is that it doesn't stop at showing cost status — it adjusts resource provisioning, scaling, and reclamation at the execution level. From a token-cost standpoint, it addresses four areas.
Continuous GPU rightsizing places inference workloads on the amount of computing resource they actually need. Because resources are determined based on current load rather than a specification set in the past, this affects the cost that used to arise from the gap between actual usage and provisioned specification.
Using spot instances for AI workloads is an option many teams have avoided because of the risk of interruption. Cast AI addresses this through live migration: when an interruption occurs, the workload continues on another node rather than being lost, and this is an area with significant cost savings potential for inference workloads.
Autoscaling handles rapidly surging agent traffic. As discussed earlier, in a structure where a single agent task branches out in multiple directions, resources need to scale up within seconds and be released just as quickly once the task ends. The reason release speed matters as much as scaling speed is that delayed release leaves idle cost sitting in place.
Multi-cloud capacity routing automatically moves workloads to whichever cloud has available resources. Its purpose is to avoid being locked into a particular region or cloud during periods when GPU availability is tight, which affects the waiting time — and the cost incurred during that wait — caused by failed provisioning.
☑️ The Approach at the Routing and Governance Layer: Splunk
The challenge at the L5 layer is different in nature. How many tokens were used can be seen from the bill, but what those tokens produced doesn't appear on the bill. To decide which request should go to which model, or where to stop, data at the level of individual agent executions has to exist first.
Splunk Agent Observability is a product built on the Galileo acquisition that secures this data. It's built so that an agent's cost and behavior can be viewed on the same screen, and its distinguishing feature is that it addresses token usage and output quality together.
First, it breaks down consumed tokens by type. Splunk classifies the tokens generated by an agent call into six categories. Context tokens are items carried along with every call, such as system instructions, conversation history, and tool schemas; reasoning tokens are what the model generates while planning. Retrieval tokens arise when grounding documents are fetched during RAG; tool tokens arise when a tool is called and its response is read. Coordination tokens are used for role prompts and state sharing in multi-agent configurations, and governance tokens arise during verification, evaluation, and human review.
This classification matters because each type behaves differently. The schema of a tool that won't be called for a given task, or prior conversation that has already become irrelevant, simply becomes overhead carried along with every call. Reasoning tokens improve quality on complex multi-step problems but only add cost on easy tasks. Governance tokens often don't show up clearly on the bill, so they frequently go unmanaged in the budget.
The cost of observation itself also falls into this category. Using a general-purpose large model for observation and evaluation can push observation cost close to the cost of the agent's own execution. Splunk Agent Observability uses small language models built specifically for evaluation, which allows the entire agent population to be observed without sampling. The fact that it doesn't sample matters operationally — if the agent causing the cost problem happens to fall outside the sample, the problem itself never becomes visible.
A central dashboard shows every agent across the environment. It tracks total request counts, latency, and input/output token counts together with their respective costs, and it can identify and terminate an agent stuck in a repeating loop before it exceeds budget. This is the actual point where the "loop with no termination condition" — identified as the final tier in the Foundation's notation — gets caught in practice.
A combined view of cost and quality is also provided. Token usage can be checked side by side with quality metrics such as hallucination, bias, and toxicity, and performance metrics such as latency and error rate. If a low-cost model achieves the same quality score as a high-cost model on a particular task, that makes it possible to route that task to the more efficient option. Complexity-based routing at the L5 layer is built on top of this data.
Control mechanisms belong to the same layer. This includes treating token usage like network traffic by setting dynamic thresholds, and building circuit breakers into agent systems that automatically terminate processes exceeding a set cost limit. It's also more effective to shift budget criteria from total token volume to workflow-level metrics such as cost per process, retry rate, and total execution time.
☑️ Metrics That Connect Token Spend to Outcomes
Even after layers are separated and action is taken, whether that action actually translated into results has to be confirmed with a separate metric. The Foundation sets the consumption area's measurement standard as cost-per-call, and has the value area's measurement start from the proportion of work processed without human intervention.
The metrics Splunk proposes point in the same direction. Token yield indicates how many successful sessions resulted per million tokens. The definition of "success" varies by workflow. For a customer support agent, success could be defined as a task that was completed, didn't exceed an acceptable error threshold, and wasn't escalated to a human. Because only sessions that pass every condition designated as mandatory count as successful, an evaluation framework that defines those criteria has to be in place before the metric can even be calculated.
Cost per accepted task is the actual price of one usable outcome, calculated as the sum of model cost, tool and runtime cost, and human review cost. Working through an example reveals the gap. If a support agent using a low-cost model has a cost-per-task of $0.02, that looks efficient. But if 40% of responses require human review, each review takes 15 minutes, and support staff cost $30/hour, the actual cost of a response requiring review comes to about $3.02. Responses that don't require review stay at $0.02, but the average across both cases ends up far higher than the bill alone would suggest.
Looking only at token usage, the low-cost model always looks better — but once review labor cost is included, the conclusion can change.
☑️ The Order for Starting Token Cost Management
The Foundation specifies an order of application: vocabulary comes before classification, classification comes before optimization, and instrumentation comes before all of it.
The first step is to check the numbers that are already available. It's better not to start by switching models or trimming context. Breaking tokens down into the six categories requires a level of tracking infrastructure most organizations don't yet have. The sequence is to first check the total input and output tokens per call that every API response already provides, in order to find where consumption is greatest, and add type-level classification once tracking infrastructure is in place.
The second step is to classify workloads. Whether a request involves one model call, multiple calls, or branches out to sub-agents changes the cost curve.
The third step is to decide which layer to address. If cost is coming from idle resources, the capacity layer comes first; if it's coming from uncontrolled calls and failed tasks, the routing/governance layer comes first.
The fourth step is to define success criteria and calculate the metrics. Whether it's token yield or cost per accepted task, no metric can be produced until what counts as success has been defined.
☑️ Frequently Asked Questions
What is AI tokenomics?
AI tokenomics is the discipline of converting energy and capital into AI and efficiently consuming AI services to translate them into business value. On August 4, 2026, the Linux Foundation launched the Tokenomics Foundation together with 30 founding members to develop open standards and benchmarks on this subject; it operates as a vendor-neutral organization working closely with the FinOps Foundation.
Why is agentic AI more expensive than general generative AI?
AI agents repeatedly reason, call tools, verify results, and retry failed tasks. Because model API calls are stateless, accumulated context is resent at every step, and in structures where an orchestrator spawns sub-agents, the number of model calls attached to a single request grows in a way that stays invisible.
If token unit prices keep falling, won't the cost problem resolve itself?
Cost per token fell sharply between 2023 and 2025, but that decline has since stopped, and the token prices of new models are rising. Combined with the fact that agent adoption is increasing consumption volume itself, it's difficult to say that falling unit prices alone will reduce total spend.
Which layer should be addressed first?
Based on the Foundation's five layers, the first results generally come from the inference stack layer and the routing/governance layer. However, since each layer inherits the cost and availability of the layer below it, if there's significant idle resource at the capacity layer, that needs to be addressed as well for improvements at higher layers to hold.
What layers do Cast AI and Splunk each handle?
Cast AI handles the capacity layer. It automates GPU resource provisioning, scaling, and reclamation in Kubernetes environments to reduce cost arising from idle resources. Splunk Agent Observability handles the routing and governance layer. It tracks per-agent token usage and cost alongside quality metrics to provide the basis for model routing and limit-setting decisions. Because they address different layers, this is a complementary configuration rather than an either-or choice.
☑️ Closing
The conclusion that tokenomics discussions consistently point to is a change in the unit of management: not how many tokens were used, but what outcome those tokens produced and what the actual price of a single outcome is. Only with this standard in place does a decision to switch models, or to impose a limit, have a basis. And this standard can't be built from a single layer alone. Even if idle resources are reduced at the capacity layer, if unnecessary re-calls continue at the routing layer, total cost won't fall; and strengthening observation and evaluation alone, without infrastructure automation, means a problem can be identified but not acted on. It also remains true that by the time the bill is checked, the spending has already happened.
CloudNetworks is an official partner of Cast AI and Splunk. We support customers from adoption review through implementation and operation, so that unused resources in Kubernetes and GPU infrastructure can be identified and addressed via Cast AI, and the actual breakdown of tokens consumed by agents can be checked and acted on via Splunk. If you're looking to bring your AI operating costs under control, please reach out to CloudNetworks.
▶ View the Cast AI product
▶ View the Splunk product
[Source : Tokenomics Foundation, "The Tokenomics Foundation - AI Value", https://www.tokeneconomics.com/, The Linux Foundation, "Linux Foundation Launches the Tokenomics Foundation to Define the Economics and ROI of AI Value", https://www.linuxfoundation.org/press/linux-foundation-launches-the-tokenomics-foundation-to-define-the-economics-and-roi-of-ai-value, The Linux Foundation, "Linux Foundation Announces the Intent to Launch the Tokenomics Foundation to Establish Open Standards for AI Cost Management", https://www.linuxfoundation.org/press/linux-foundation-announces-the-intent-to-launch-the-tokenomics-foundation-to-establish-open-standards-for-ai-cost-management, Splunk, "The New Currency of AI: Why Tokenomics is the Next Big Test for Tech Leaders", https://www.splunk.com/en_us/blog/artificial-intelligence/ai-tokenomics-govern-agentic-ai-costs-with-splunk.html, Splunk, "What is Agent Tokenomics?", https://www.splunk.com/en_us/blog/artificial-intelligence/what-is-agent-tokenomics.html, CAST AI, "What Is Tokenomics, And Why Your AI Infrastructure Is Now a FinOps Problem", https://cast.ai/blog/tokenomics-why-your-ai-infrastructure-is-now-a-finops-problem/]
