Datadog LLM Observability

danger

LLM Observability is in public beta, and its API is subject to change.

With Datadog LLM Observability, you can monitor, troubleshoot, and evaluate your LLM-powered applications, such as chatbots. You can investigate the root cause of issues, monitor operational performance, and evaluate the quality, privacy, and safety of your LLM applications.

This is an experimental community implementation, and it is not officially supported by Datadog. It is based on the Datadog LLM Observability API.

Setup

tip

See this section for general instructions on installing integration packages.

npm

npm install @langchain/community @langchain/core

Yarn

yarn add @langchain/community @langchain/core

pnpm

pnpm add @langchain/community @langchain/core

Usage

import { OpenAI } from "@langchain/openai";
import { DatadogLLMObsTracer } from "@langchain/community/experimental/callbacks/handlers/datadog";

/**
 * This example demonstrates how to use the DatadogLLMObsTracer with the OpenAI model.
 * It will produce a "llm" span with the input and output of the model inside the meta field.
 *
 * To run this example, you need to have a valid Datadog API key and OpenAI API key.
 */
export const run = async () => {
  const model = new OpenAI({
    model: "gpt-4",
    temperature: 0.7,
    maxTokens: 1000,
    maxRetries: 5,
  });

  const res = await model.invoke(
    "Question: What would be a good company name a company that makes colorful socks?\nAnswer:",
    {
      callbacks: [
        new DatadogLLMObsTracer({
          mlApp: "my-ml-app",
        }),
      ],
    }
  );

  console.log({ res });
};

API Reference:

• OpenAI from @langchain/openai
• DatadogLLMObsTracer from @langchain/community/experimental/callbacks/handlers/datadog