Generative AI platforms & models · AWS vs Google Cloud
Amazon Bedrock Evaluations / Amazon Bedrock AgentCore vs Gen AI evaluation service
Bedrock Evaluations maps to the Gen AI evaluation service on Agent Platform, which scores models with adaptive or static rubrics, computation metrics, and custom functions.
Agent evaluation is a separate feature.
Side-by-side comparison
| Aspect | Amazon Web Services | Google Cloud |
|---|---|---|
| Service | Amazon Bedrock Evaluations Amazon Bedrock AgentCore | Gen AI evaluation service |
| Lifecycle | Generally available Generally available | Generally available |
| What it is | Automatic, LLM-as-a-judge, and human evaluation jobs for models and RAG sources, including sources outside Bedrock. Modular agent platform. Harness (managed agent loop), Runtime, Memory, Gateway (MCP), Identity, Code Interpreter, Browser, Observability, Evaluations, Optimization, Policy, Registry, Payments. Works with any framework and any model. | Evaluates generative models with adaptive rubrics, static rubrics, computation-based metrics (such as ROUGE and BLEU), and custom Python functions. Agent evaluation is a separate feature. |
| Pricing model | You pay for the model inference used (generator and judge models) plus any human-evaluation task fees. AgentCore Evaluations is consumption-based. | Billed for the model calls the evaluation makes (for example the autorater model's tokens). See the Agent Platform pricing page. |
| Docs | Amazon Bedrock Evaluations docs Amazon Bedrock AgentCore docs | Gen AI evaluation service docs |
| Pricing page | Amazon Bedrock Evaluations pricing Amazon Bedrock AgentCore pricing | Gen AI evaluation service pricing |
Key differences
- Adaptive rubrics generate pass/fail tests per prompt using an autorater model.
- Computation-based metrics such as ROUGE and BLEU need ground-truth references.
- Bedrock also offers human-based evaluation jobs with a managed workforce; Google's service is automated.
Pricing summary
Pricing models only. No prices are quoted because they change by region and over time.
- AWS
- You pay for the model inference used (generator and judge models) plus any human-evaluation task fees. AgentCore Evaluations is consumption-based. Amazon Bedrock Evaluations pricing → Amazon Bedrock AgentCore pricing →
- GCP
- Billed for the model calls the evaluation makes (for example the autorater model's tokens). See the Agent Platform pricing page. Gen AI evaluation service pricing →
And on Azure?
Direct match Azure's equivalent is Foundry observability & evaluations. AWS vs Azure comparison →
FAQ
What is the Google Cloud equivalent of Amazon Bedrock Evaluations and Amazon Bedrock AgentCore?
Bedrock Evaluations maps to the Gen AI evaluation service on Agent Platform, which scores models with adaptive or static rubrics, computation metrics, and custom functions. Agent evaluation is a separate feature.
How does pricing compare between Amazon Bedrock Evaluations and Amazon Bedrock AgentCore and Gen AI evaluation service?
AWS: You pay for the model inference used (generator and judge models) plus any human-evaluation task fees. AgentCore Evaluations is consumption-based. Google Cloud: Billed for the model calls the evaluation makes (for example the autorater model's tokens). See the Agent Platform pricing page. Check the official pricing pages for current rates.
What is the Azure equivalent?
Foundry observability & evaluations (direct match). Bedrock Evaluations and AgentCore Evaluations map to Foundry observability and evaluations (built-in and custom evaluators, continuous evaluation, and tracing).