At OpenMeter, we work with companies to enable usage-based billing, Product-Led Growth (PLG) use cases and cost management. Building a robust metering system requires a careful balance between accuracy, auditability, real-time processing, and cost. This article explores the challenges of metering and helps to avoid common pitfalls.
What is Usage Metering?
Usage metering tracks and records the consumption of a resource or service over time. This usage can take various forms, such as API requests, compute time seconds, or tokens consumed. Usage metering is commonly event-based to ensure accuracy and auditable data.
Why Do You Need Metering?
Widespread metering and consumption use cases include:
- Usage-based billing
- Enforcing usage limits and entitlements
- Real-time consumption dashboards
- Quoting and forecasting for renewals
- Enabling Product-Led Growth and Product-Led Sales
- Cost-attribution and margin analysis
Why Does Metering Need to Be Auditable?
In usage-based billing, metering data directly impacts customer invoices and revenue. Because financial transactions are involved, metered data must be accurate and auditable. This ensures the integrity of data collection, processing, and reporting. The industry distinguishes between auditable and operational data as:
- Auditable Data: Loss of any data record is intolerable, and complete retention is necessary. Examples include transaction logs, replication logs, and billing events.
- Operational Data: Some data loss is acceptable to maintain cost efficiency. Examples include infrastructure observability and metrics.
For usage metering, auditable data ensures accuracy and accountability.
Learn more about auditable and operational data.
Why is Usage Metering Event-Based?
Event-based metering ensures that each billable transaction generates a the corresponding billable event, which the metering system aggregates into usage data over time. Event-based metering enables accuracy by allowing deduplication of events based on their unique IDs and timestamps, making revenue recognition possible.
Where Should Usage Be Measured?
Measure usage at the point when a billable transaction occurs. Consumption data can originate from multiple sources, such as API requests, tokens, or compute time. We recommend a "fire-and-forget" approach, where each billable transaction generates a single billable event sent directly to the metering system. This avoids the complexities of pre-aggregating usage or calculating workload execution duration from lifecycle events.
When metering data size like storage, fairness dictates that metering occurs at data ingest, ensuring customers only pay for what they send and what they can control. For example, disk size can vary significantly due to compression, disk fragmentation, or other factors, but customers shouldn't bear the burden of your system's inefficiencies and technical debt. Efficient data storage is your competitive edge.
Learn more about how to meter workload execution.
How To Meter Continues Workloads?
For continuous workloads, like compute or long-running jobs, metering can be challenging as it's not a single transaction. In this case, we recommend the heartbeat approach, where the workload sends a heartbeat event at regular intervals to the metering system. This is more accurate than calculating the workload's consumption from its lifecycle events, which can span multiple billing periods or be a victim of data retention or loss.
Learn more about heartbeat metering.
Do You Need a Metering System?
The short answer is yes. The need depends on your usage volume and consumption
data use cases. High usage volumes, typical in AI, Compute, and DevOps,
introduce engineering challenges. Use cases evolve with your product's maturity,
from simple billing to complex pricing models requiring real-time metering and
flexible data integrations.
You can read more about the topic here.
Why Is Implementing Metering Challenging?
Building a metering system that is accurate, auditable, real-time, and cost-effective is complex:
- Accurate: Observability and metrics systems often lack accuracy.
- Auditable: Each billable transaction results in a billable event.
- Real-time: Periodic batch processing in data warehouses introduces latency.
- Cost Effective: Database writes at scale are expensive.
Inaccuracy of Metrics Systems
Observability systems like Prometheus excel at gathering and processing operational data but often lack the consistency for auditable use cases like billing. Tracking individual user consumption on a large scale is challenging, especially with high cardinality.
Expensive Writes
Scaling databases for high-volume event ingestion is costly. Not only must you store every record, but also aggregate them during queries, driving up costs. Most databases perform best with large batch inserts, so metering often requires a buffer or stream processing pipeline to support the database.
Complex Stream Processing
Storing all records in a data warehouse and processing them in batches can save costs but results in stale data. Real-time usage metering requires complex stream processing to build out and maintain. For example, most companies use Kafka-based solutions to implement real-time usage metering.
Learn more about metering with stream processing.
Should You Pre-Aggregate Usage?
In usage-based billing, especially in AI and cloud computing, a large volume of consumption data is generated—potentially millions of records per second. While pre-aggregating usage data appears to reduce infrastructure costs at first sight, it carries weights and hidden costs in terms of architecture complexity and maintenance, delivering limited data granularity and reporting frequency and thus slowing down data-driven decisions and real-time use cases.
It Can Be Expensive to Implement
Pre-aggregation requires temporarily storing raw events before reporting the aggregated usage. Ensuring each raw usage is reported once and only once is complex and expensive. Pre-aggregation of data also requires extra components in your architecture to handle this task. These should be highly available and responsive, significantly burdening your engineering and SRE teams. Moreover, pre-aggregating data can limit future query flexibility, forcing you to double-record data in different aggregations, which adds complexity and slows down iteration.
Impacts Real-Time Use-Cases
Pre-aggregation frequency affects data freshness, which is critical for real-time use cases like consumption dashboards and usage limit enforcement. Systems relying on hourly or daily snapshots lose the ability to provide real-time feedback, negatively impacting customer experience and cost control.
Error-Prone Reporting
Pre-aggregated usage data can lead to mistakes if the aggregation logic isn't consistently applied. For example, if you incorrectly sum the maximum storage size of each database shard, you could over-report consumption, leading to incorrect business decisions. Such errors are easy to make with pre-aggregated data and can have significant consequences.
Limits Deduplication Window
Each aggregation step reduces the data available for deduplication, making it harder to track metrics like seat or license usage over time, impacting billing accuracy and complicating data management.
Read more about usage deduplication.
Summary
Implementing an effective metering system is essential for companies offering usage-based pricing. Event-based metering has become the industry's preferred approach to ensure accuracy, auditability, and real-time performance. However, building such a system presents challenges, from scaling databases to balancing cost and performance.
Pre-aggregating data can reduce infrastructure costs, limiting future flexibility and hindering real-time decision-making. Companies must carefully consider where and how to measure usage to ensure that their metering system adapts to evolving needs without sacrificing accuracy or increasing operational costs. By understanding these challenges, businesses can create robust metering solutions supporting billing and Product-Led Growth initiatives.