# What Exactly Changed With AgentCore Evaluations?

According to the AWS Machine Learning Blog, published August 26, 2026, Bedrock AgentCore Evaluations now accepts OpenTelemetry telemetry from any agent framework. The announcement explicitly lists LangGraph, LlamaIndex, the OpenAI Agents SDK, Google ADK, the Claude Agent SDK, and Strands Agents as supported frameworks. This means the evaluation service no longer cares about the underlying orchestration layer — it only cares whether your agent emits standardized traces, metrics, and logs.

The technical contract is simple: if your agent emits OpenTelemetry signals, AgentCore can score it. This is a fundamental shift from the previous model where evaluation tools were often tied to a specific framework's internal state or instrumentation. AWS is effectively saying that the evaluation layer should be framework-agnostic, and OpenTelemetry is the universal language that makes that possible.

What this means operationally is that a team using LangGraph can now evaluate their agents against the same benchmarks and quality gates as a team using the OpenAI Agents SDK, without writing custom evaluation harnesses. The evaluation infrastructure becomes portable, and switching frameworks no longer means throwing away your evaluation pipeline.

# Why Does OpenTelemetry Matter More Than the Framework Itself?

OpenTelemetry has been the observability standard for microservices for years, but this move extends its reach into the agent runtime itself. The AWS announcement positions OpenTelemetry as the contract that makes agent evaluation possible across diverse frameworks. This is not just about tracing HTTP calls — it's about capturing the decision-making process, tool calls, and intermediate reasoning steps that define agent behavior.

The OpenTelemetry project documentation describes traces as the way to capture the full path of a request through a distributed system. For agents, this translates to capturing every step from user input to tool invocation to final response. AgentCore Evaluations leverages this existing standard rather than inventing a proprietary telemetry format, which means any agent that already emits OpenTelemetry for debugging purposes can immediately be evaluated without additional instrumentation.

This is the key insight: the barrier to entry for evaluation has collapsed. According to the AWS blog post, the service can score agents emitting OTel telemetry — that's it. No SDK installation, no framework-specific adapters, no re-platforming. For developers, this means the evaluation layer is now a commodity, and the real differentiator becomes how well your framework emits telemetry in the first place.

# Who Wins and Who Loses in the New Evaluation Economy?

The immediate winners are enterprises running heterogeneous agent stacks. A company with one team on LangGraph and another on the Claude Agent SDK can now standardize on Bedrock AgentCore Evaluations without forcing either team to change frameworks. The loser is any vendor that built proprietary evaluation tooling tied to a single framework — those tools now look like expensive lock-in rather than value-add.

Consider the comparison across the major frameworks:

For framework vendors, the strategic calculus just changed. LangSmith and similar proprietary evaluation tools now compete directly with a service that works across all frameworks. The value proposition of LangSmith was deep integration with LangGraph internals; AgentCore can't replicate that, but it doesn't need to — it offers cross-framework consistency instead.

# What Should Teams Do Differently Starting Today?

First, audit your agent's existing telemetry. According to the OpenTelemetry specification, traces must include span names, timestamps, and attributes to be useful. If your agent framework emits OTel already, you're 90% of the way to AgentCore compatibility. If not, you need to add instrumentation — and the AWS blog post suggests this is straightforward for the listed frameworks.

Second, stop treating evaluation as framework-specific. The AWS announcement makes it clear that evaluation should be a cross-cutting concern, not a feature of your orchestration layer. Start designing evaluation harnesses that measure agent quality independent of the framework, using OTel telemetry as the input.

Third, consider the strategic implications for your vendor relationships. If you're paying for a proprietary evaluation tool, you're now paying for something that Bedrock AgentCore does for free (or at least at AWS pricing). The value of LangSmith, for example, has just been significantly reduced for teams that don't need its deep LangGraph integration.

1. By Q2 2027, LangChain will add native Bedrock AgentCore Evaluations integration, conceding that cross-framework evaluation is the enterprise standard.
2. By Q1 2027, OpenAI will enhance the OpenAI Agents SDK's OTel emission to include tool-call-level spans, specifically to remain competitive in Bedrock-based evaluations.
3. By Q3 2027, at least two major enterprises will publicly report switching from proprietary evaluation tools to Bedrock AgentCore Evaluations for multi-framework agent stacks.

# What's the Bottom Line for Agent Development Teams?

The bottom line is that evaluation is no longer a reason to choose a framework. Amazon has made it a neutral service, and the only thing that matters is whether your agent emits good telemetry. This is a massive simplification for teams that were previously locked into a framework because of its evaluation tooling.

The strategic takeaway is that AWS is positioning itself as the Switzerland of the agent ecosystem — providing the evaluation layer that works across all frameworks, while the frameworks themselves compete on orchestration quality, not lock-in. For developers, this is the best possible outcome: more choice, less risk, and a clear standard for measuring agent quality.

• Framework choice is now decoupled from evaluation — pick the best orchestration tool, not the best evaluation tool.
• OpenTelemetry is the universal contract; invest in rich, accurate telemetry emission regardless of framework.
• Proprietary evaluation tools face an existential threat from cross-framework alternatives.
• The next competitive battleground is telemetry quality, not framework features.
• Enterprises with heterogeneous agent stacks are the immediate winners of this announcement.