The workflow links trace findings to evaluations, then verifies harness changes on development and held-out tasks.
NVIDIA outlines a NeMo workflow that turns recorded agent failures into prioritized issues and custom evaluations. NeMo Compass analyzes stored agent traces and links each finding to specific trace IDs, while NeMo Eval Author provides coding agent skills and workflows for Harbor or NVIDIA NeMo Gym formats.
The tutorial uses a Hermes enterprise assistant for a fictional company inside an NVIDIA OpenShell sandbox. The assistant connects through Model Context Protocol to company chat, email, files, and ticketing systems, and recorded traces show where tool calls, routing rules, or missing context derail tasks.
Builders can update prompts, tools, routing, memory, skills, control flow, and harness code, then measure changes against domain requirements. The loop reruns evaluations on development and held-out tasks to verify improvements and detect regressions before teams trust an agent in production.
NVIDIA points builders to the NemoClaw blueprint, a workplace assistant tutorial on GitHub, and a CAD guide for engineering workflow optimization. In a Dassault Systèmes engineering example, the same loop cut malformed tool calls from 142 to zero and raised the overall evaluation score by 28% relative with 35% fewer tool calls per run across six scenarios.
What matters
- NVIDIA NeMo Compass links stored agent traces to prioritized failures and custom evaluations.
- Builders can test prompts, tools, routing, memory, skills, and harness code against domain requirements.
- The NemoClaw blueprint runs a full optimization cycle on an example Hermes enterprise assistant.
Why it matters
The NemoClaw blueprint runs a full optimization cycle on an example Hermes enterprise assistant.
This GenAI News article was prepared in original wording using reporting and materials published by NVIDIA Developer Blog. Source reference: https://developer.nvidia.com/blog/use-nvidia-nemo-to-turn-agent-failures-into-tests-and-harness-fixes/.
Drafted by the GenAI News review pipeline.