OpenTelemetry | OpenMeter

Non-sampled logs (e.g., access logs) are excellent sources of usage information. Combined with OpenTelemetry open standard for log forwarding, you can easily extract usage information from your logs and forward them to OpenMeter.

The OpenMeter Collector is powered by Redpanda Connect.

#Prerequisites

You will need an OpenTelemetry-compatible log-forwarding solution. The easiest way to get started is to use the OpenTelemetry Collector.

#Configuration

First, create a new YAML file for the collector configuration. You will have to use the otel_log Redpanda Connect input:

input:
  otel_log:
    # Point your log forwarder to this address using the OTLP gRPC protocol.
    address: 127.0.0.1:4317

OpenMeter Collector

This is a custom input plugin that is not part of the official Redpanda Connect distribution. You can find the source code of the plugin here.

Next, you need to configure the mapping from your log schema to CloudEvents using bloblang:

pipeline:
  processors:
    - mapping: |
        root = {
          "id": uuid_v4(),
          "specversion": "1.0",
          "type": "api-calls",
          "source": "otlp-log",
          "time": this.record.attributes.time,
          "subject": this.record.attributes.subject,
          "data": {
            "method": this.record.attributes.method,
            "path": this.record.attributes.path,
            "region": this.record.attributes.region,
            "zone": this.record.attributes.zone,
            "duration_ms": this.record.attributes.duration,
          },
        }

About log attributes

this.record.attributes contains the log attributes extracted by the otel_log input plugin.

Finally, you need to configure the OpenMeter output:

output:
  openmeter:
    url: https://openmeter.cloud # optional
    token: '<YOUR OPENMETER CLOUD TOKEN>'

Read more about configuring Redpanda Connect in the OpenMeter Collector guide.

Example on GitHub

Check out a fully working example here.

#Installation

Check out the OpenMeter Collector guide for installation instructions.