Agent Evaluation

Automated Evaluation Suite

SKOOR maintains a labeled test dataset built from your historical corrections and verified categorizations. The evaluation suite runs continuously against this dataset, measuring classification accuracy, false positive rate, and confidence calibration. If accuracy on expense coding drops from 97% to 93%, the system detects the regression before it affects your workflow.

Prompt Versioning

Every improvement to the AI's behavior goes through a versioned release process. New prompt versions are tested against the evaluation suite before deployment. If a proposed change improves expense coding accuracy but degrades vendor matching, the change is blocked until the regression is resolved. This prevents well-intentioned improvements from causing unexpected side effects.

Regression Detection and Rollback

If a deployed change causes accuracy to drop below the configured threshold, SKOOR automatically rolls back to the last known-good version. The rollback happens within minutes, not days. You receive a notification from GABRIEL explaining what happened and what was reverted. This safety net ensures your AI agent maintains consistent quality even as the underlying models evolve.

• Continuous evaluation against labeled test data
• Prompt versioning: changes tested before deployment
• Automatic rollback if accuracy drops below threshold
• GABRIEL notification on any detected regression