We are happy to announce that the OpenMeter Collector now supports event buffering. With buffering enabled, the collector retains events on a persistent volume during network failures and re-delivers them once network connectivity is recovered, regardless of whether it takes hours or days. Never lose an event and prevent revenue leakage caused by network issues.
What is the OpenMeter Collector?
The OpenMeter Collector is a standalone application you can install in your infrastructure. It allows you to point the OpenMeter SDKs to enhance the resilience of your event metering pipeline and meter various usage data sources out of the box.
The OpenMeter collector supports:
- Event buffering and retries for SDKs
- Metering Kubernetes and Run:ai workloads
- Converting Prometheus metrics to metered usage
- Importing usage from PostgreSQL, ClickHouse, and other databases
Once collected, this data is turned into billable usage events and processed through OpenMeter's platform. From there, you can price, invoice, and track revenue.
OpenMeter SDKs With Event Buffering
OpenMeter offers SDKs for Node.js, Python, Go, and other languages that communicate with the OpenMeter API via HTTP. Installing the OpenMeter Collector with buffer enabled lets you point your SDKs to the OpenMeter Collector instead of sending events directly to OpenMeter Cloud. In this configuration, the OpenMeter SDKs will first send the event to the OpenMeter Collector, which will forward the events to OpenMeter Cloud and buffer them, retrying in the case of network failure.
To increase the resilience of your metering pipeline, the OpenMeter Collector comes with:
- Buffering
- Retries and backoff
- Deduplication
- OTel Metrics and Logging
How Does Event Buffering Work?
When buffering is enabled, in the case of connectivity issues, the OpenMeter Collector stores events on an attached persistent volume until the network recovers. When the events are replayed, the OpenMeter Cloud deduplicates events based on their ID and source properties to ensure each event is processed once and exactly once. The OpenMeter Collector also provides visibility into the buffer and processing states by exposing Prometheus metrics. The available space on the attached disk determines the size of the buffer.
Getting Started With Event Buffering
You can enable buffering with the OpenMeter Collector in two simple steps:
- Install the OpenMeter Collector
- Point your SDKs to the installed collector:
Check out the event buffering guide for more details.
That's it, enjoy the benefits of a robust metering pipeline that can automatically recover from network failures.
