The new evaluation service reads OpenTelemetry spans, so any framework compatible with the standard can be scored.

Amazon Bedrock AgentCore Evaluations decouples agent evaluation from the underlying framework. Teams can use the service to score agents built on LangGraph, LlamaIndex, the OpenAI Agents SDK, Google ADK, the Claude Agent SDK, or Strands Agents, provided their telemetry flows through OpenTelemetry.

The service reads spans exported over the OpenTelemetry Protocol and collected by the AWS Distro for OpenTelemetry. The ADOT backend routes span and event records to Amazon CloudWatch, and the evaluation service scores the agent based on those spans, regardless of the SDK underneath.

For builders, this means evaluation pipelines no longer break when a team switches frameworks. An agent’s execution produces spans for model calls, tool calls, retrieval, reranking, embeddings, guardrail checks, and orchestration steps, and the service can read all of them through OpenTelemetry.

Coverage already extends to any framework that emits OpenTelemetry-compatible telemetry. Watch for the evaluation service to support additional agent frameworks as they emit the expected span attributes.

What matters

• AgentCore Evaluations scores any agent framework that emits OpenTelemetry spans.
• Builders avoid pipeline rework when switching between LangGraph, LlamaIndex, or the OpenAI SDK.
• The service will cover more frameworks as their telemetry follows OpenTelemetry conventions.

Why it matters

The service will cover more frameworks as their telemetry follows OpenTelemetry conventions.

This GenAI News article was prepared in original wording using reporting and materials published by AWS Machine Learning Blog. Source reference: https://aws.amazon.com/blogs/machine-learning/evaluate-any-agent-framework-with-amazon-bedrock-agentcore-evaluations/.

Drafted by the GenAI News review pipeline.