The transition from AI assistants to AI agents changes the question - from "is this output accurate?" to "should this system be allowed to act?" These are the fronts where current methods are insufficient.
When models browse, execute code, manage files, and chain decisions across multi-step workflows, trust becomes compositional, contextual, and continuous. A model trusted to summarize documents may not be trusted to send emails. A workflow trusted at 10 requests/minute may fail catastrophically at 10,000.
Current approaches - output filtering, behavioral red-teaming, RLHF alignment - treat trust as a binary classification problem. Rotalabs exists to develop the science, benchmarks, and infrastructure for AI trust at scale. GlassBox is our first public instrument in that program.
The problem: as agents call agents, how does Agent A verify Agent B isn't compromised, and how do we detect agents coordinating on harmful strategies? Why it's hard: traditional authentication assumes static identities, but agents are defined by weights, prompts, and tool access - all mutable. Our approach: inter-agent trust protocols with cryptographic verification, collusion detection, and agent attestation.
The problem: in a 50-step workflow, which steps need human review, which can be AI-verified, which automated? Why it's hard: perfect verification is expensive, skipping it is dangerous, and the optimal policy depends on task, stakes, confidence, and history. Our approach: hierarchical verification routing, debate and self-critique for high-stakes decisions, and verification-cost economics.
The problem: do reasoning traces reflect actual internal computation, or post-hoc rationalization? Why it's hard: we can't directly observe cognition; CoT is a lossy projection a model could learn to game. Our approach: CoT-faithfulness detection, identifying hidden communication in traces, and comparing internal representations to stated reasoning.
The problem: any trust mechanism can be gamed - deploy sandbagging detection and adversaries train to evade it. Why it's hard: it's an arms race; sophisticated adversaries adapt and game-theoretic equilibria are hard to characterize. Our approach: evasion attacks that then harden defenses, game-theoretic analysis of routing exploitation, and evolutionary adversarial testing.
The problem: regulated industries need guarantees, not probabilities. Why it's hard: neural networks aren't formally verifiable in the traditional sense - but properties of decision boundaries, tool calls, and action constraints can be. Our approach: bounded-uncertainty certificates, formally verified tool use, and regulatory-grade audit trails.
The problem: long-running agents accumulate memory that can be poisoned, stolen, or legally required to be deleted. Why it's hard: memory is contextual and associative - "delete all information about user X" is ambiguous when it's distributed across embeddings. Our approach: memory-poisoning detection, cryptographic memory provenance, and verified forgetting for compliance.
Questions we're thinking about - we don't have complete approaches yet.
We're looking for researchers, engineers, and organizations interested in AI reliability.
Type to search across all pages and posts
Press ↑ ↓ to navigate, Enter to select
