OpenInference

OpenInference is an open-source standard designed to bring observability to AI and Large Language Model (LLM) applications. Built as a set of semantic conventions and plugins on top of OpenTelemetry (OTel), it provides a vendor-neutral way to trace the complex, multi-step operations typical of modern AI systems.

Why OpenInference Exists

While OpenTelemetry is the standard for distributed tracing, AI applications have unique requirements that generic OTel models do not address. OpenInference defines AI-specific schemas:

• Structured AI Data: Handles complex, nested data like multi-turn message arrays, system prompts, tool definitions, and multimodal content.
• Token Economics: Provides standardized tracking for prompt, completion, cached, and reasoning token counts.
• Agentic Control Flow: Establishes a taxonomy to represent “hops” in agentic workflows (loops, sub-agent delegation, tool invocations).
• Privacy: Allows for per-field masking of sensitive data (like PII in prompts) before export.

How It Works

OpenInference works by “tagging” OpenTelemetry spans with specific AI semantic conventions (e.g., llm.model_name, llm.token_count.*, openinference.span.kind).

• Compatibility: Every OpenInference trace is a valid OpenTelemetry trace, ensuring compatibility with any OTel-compliant backend.
• Auto-Instrumentation: Offers libraries for popular frameworks (LangChain, LlamaIndex, CrewAI, AutoGen, OpenAI, Anthropic), allowing trace capture without code changes.
• Vendor Agnostic: Enables collecting observability data once and sending it to various platforms (e.g., Arize Phoenix, Dynatrace) without proprietary lock-in.