Building an AI product for hundreds of thousands of internal users is a different problem than building one for a demo.
The failure modes are different. The success criteria are different. The stakeholders you need aligned, the compliance reviews you need to pass, and the operational infrastructure you need in place are all different. And the gap between "it works" and "it works at scale" is larger than most teams think.
I learned this firsthand owning the Remote Identity Verification platform at Amazon — a system that touched 650,000+ corporate employees across global operations. Here's what the experience taught me about the difference between enterprise AI pilots and enterprise AI products.
The Problem
Identity verification at that scale is a security-critical flow. Get it wrong and you've either locked out employees who need support or you've created a pathway for unauthorized access to internal systems handling sensitive data.
The existing approach was fragmented. Employees reached support through three different channels — an IT portal, a chatbot, and case management — and each channel had its own identity verification process. Verification was slow (averaging several minutes per case), inconsistent across channels, and generated significant friction for the 1,500+ IT engineers who handled the resulting support queue daily.
The AI/ML opportunity was real: pattern-based verification could dramatically reduce verification time while improving accuracy. But "integrating AI/ML verification" across three separate channels, at 650,000-user scale, with zero tolerance for security failures, is not a sprint-sized task.
The Approach
I owned the full product lifecycle — discovery, roadmap, cross-functional delivery, and launch — which meant I was accountable for both the business outcome and the technical feasibility. That dual accountability shaped every decision.
Discovery came first. I spent six weeks talking to IT engineers, security stakeholders, HR, and end users before writing a single requirement. The goal wasn't to validate a solution — it was to understand the problem precisely enough that the solution would be obvious. What I found: the bottleneck wasn't verification speed in isolation. It was the re-verification burden. Employees who had verified identity in one channel were forced to verify again in another. Fixing that — building a shared verification state across channels — would eliminate more friction than any model improvement could.
The roadmap reflected this. Phase one: shared identity state across the IT portal, chatbot, and case management. Phase two: AI/ML-powered verification replacing manual review steps. Phase three: automated workflows for the 1,500 IT engineers handling the resulting case queue, reducing the human-in-the-loop requirement for routine cases.
The cross-functional alignment was as important as the technical delivery. Security needed to sign off on the verification model's false acceptance and false rejection rates. HR needed to understand the implications for onboarding flows. Legal needed to review data handling. Getting all of those reviews running in parallel rather than sequentially — which is how enterprise compliance review usually works, serially, with each review waiting on the previous — compressed a timeline that would otherwise have stretched to 18 months into eight.
The Results
- 25% reduction in identity verification time across all support channels
- AI/ML verification integrated across IT portal, chatbot, and case management
- Automated workflows built for 1,500 IT engineers
- $1B in hardware and software assets brought under streamlined management
- Zero security incidents during or after rollout
The Real Lesson
The AI/ML component — the verification models, the integration work, the inference pipeline — was maybe 30% of the actual complexity of the project. The other 70% was product management: defining the right problem, aligning stakeholders across security, HR, legal, and engineering, sequencing the rollout to manage risk, and building the measurement infrastructure to know whether it was working.
This is the pattern I see consistently underestimated in enterprise AI work. Teams focus on the model and the integration because those are tractable, well-understood engineering problems. The ambiguous, relationship-intensive, communication-heavy work of getting a large organization aligned and moving in the same direction gets scoped as "change management" and treated as secondary.
At 650,000 users, there is no secondary. Everything is the critical path. The PM who treats the organizational challenge as secondary to the technical challenge ships a technically excellent product that nobody uses. The PM who treats them as equally important ships something that actually changes how the organization operates.
That's the job.
