Metering OpenAI ChatGPT token usage

In this example we will track our customer's OpenAI token consumption. This is useful if you are building top of the OpenAI API like Chat GPT and want to meter your customers usage for reporting or billing purposes.

#Prerequisites

Source code for this example is available here.

Clone the OpenMeter repository and navigate to the example directory:

git clone [email protected]:openmeterio/openmeter.git
cd examples/ingest-openai-node

#Our Example

The OpenAI response contains token usage so all we need to do is to report it to OpenMeter with the corresponding user. For idempotency we use the OpenAI API response id and for time we use response's created property.

Check out the quickstart guide to see how to run OpenMeter.

OpenAI response:

{
   "id":"chatcmpl-abc123",
   "created":1677858242,
   "model":"gpt-3.5-turbo-0301",
   "usage":{
      "prompt_tokens":13,
      "completion_tokens":7,
      "total_tokens":20
   },
   ...
}

Report usage to OpenMeter:

const model = 'gpt-3.5-turbo';
await openmeter.events.ingest({
  specversion: '1.0',
  // We use Open AI response ID as idempotent key
  id: completion.id,
  source: 'my-app',
  type: 'openai',
  subject: 'my-awesome-user-id',
  // We get date from Open AI response
  time: new Date(completion.created * 1000),
  // We report usage with model
  data: {
    total_tokens: completion.usage.total_tokens,
    prompt_tokens: completion.usage.prompt_tokens,
    completion_tokens: completion.usage.completion_tokens,
    model,
  },
});

Note how we also collect the OpenAI model version. This is useful as OpenAI charges differently for varios models so you may want to group by them in OpenMeter.

Check out the full source code in the app.ts. You can also run it as:

npm install
OPENAI_ORG=org-.. OPENAI_API_KEY=sk-... npm start

#Streaming

Modern applications thrive on being responsive and fluid. With Large Language Models (LLMs) like ChatGPT, generating extensive outputs can take a while. Stream APIs allow for processing responses as soon as they become available. OpenAI's data-only stream API makes this possible, but it doesn’t return token usage metadata, which is by default included in OpenAI’s blocking API call response. To fill the gap and enable accurate usage metering with stream APIs, we implemented an example in app-stream.ts that tokenizes messages as they become available. Check out the full source code in the app-stream.ts.

#Scripts

To regenerate the OpenMeter TypeScript client run npm run typegen.