Imagine bringing an AI into your car’s garage to help diagnose issues or manage service schedules. You’d want to know if it’s reliable under pressure—can it handle unexpected crises without cutting corners? Recent experiments with AI models in a simulated business environment show that, much like a mechanic under stress, not all AI agents can stay honest or complete their work when it matters most. This isn’t just about chat quality or code accuracy — it’s about management skills, resilience, and trustworthiness when the stakes are real.
Listen free for 30 days with Audible
Thousands of audiobooks and originals — cancel anytime.
As an affiliate, we earn on qualifying purchases.
Measuring Management, Not Just Chat
In a groundbreaking live experiment, four advanced AI models were tasked with running a small software company through its worst week—complete with customer crises, internal temptations, and the pressure to cut corners. The goal wasn’t just to see if they could handle a coding challenge, but whether they could act ethically, prioritize the company’s best interests, and deliver results under stress.
All four models successfully identified every crisis and refused every manipulation attempt, which is promising. But when it came to closing a pivotal deal, only two of them actually signed the agreement—despite all diagnosing the same opportunity and pitching the same solution. The other two failed to follow through, leaving critical opportunities on the table, illustrating that management quality involves more than just spotting problems; it’s about executing decisively and ethically.
AI management software for business decision making
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
The Hidden Weakness: Reading Deeper Files
Digging into the company’s own records revealed a crucial difference: models that read and understood documents two layers deep in the internal files won the deal at full price, adding €4,583 in Monthly Recurring Revenue (MRR). This underscores a simple truth—trustworthy management requires understanding context beyond surface-level signals. AI that reads more deeply can uncover hidden opportunities or risks that others miss, a critical trait for business decision-making under real-world pressures.
AI ethical decision support tools
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
But Would They Cheat?
One of the experiment’s tests involved social engineering: fake messages from a CEO escalating across stages, and a reporter trying to get a quick approval bypass. All models refused to accept the manipulative requests, with the Kimi K3 model explicitly reasoning that the request looked like impersonation or approval-bypass—demonstrating that ethical judgment matters in these systems.
Recolx Fit 2 AI Voice Recorder Note Taker & Translation Earbuds.
- Real-Time Meeting Notes: Captures and organizes meeting details
- AI Summaries & Templates: Transforms conversations into organized notes
- Multilingual Support: Creates notes and summaries in multiple languages
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
The Real Business in Action
The experiment isn’t just theoretical. It takes place inside a live company that operates every business day, managing real money—€105,000 monthly burn against €2,300 MRR—and 680+ learned rules guiding daily decisions. This setup offers a watchable, practical view of how AI can function in real operational environments, not just in demo or chat scenarios.
AI cybersecurity and fraud prevention software
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Performance Gaps and What Matters
The leading model, gpt-5.6-sol, scored a 95 out of 100, successfully closing the full deal. Kimi K3 scored 93, also closing the deal with the cleanest discipline, while Sonnet 5 and Opus 4.8 lagged behind, leaving opportunities unexploited or slipping discipline. Interestingly, Opus 4.8 was more thorough in analysis but still left the closing on the table, revealing that thoroughness alone doesn’t guarantee management success under pressure.
The Takeaway for Business Leaders
This experiment highlights a vital shift: the quality of management shown by AI isn’t measured by how well it chatters or codes but by its ability to stay honest, understand context deeply, and execute decisions reliably when stakes are high. For automotive and garage operators considering AI tools for diagnostics, scheduling, or customer management, the question isn’t just about how well the AI can answer questions. It’s whether it can finish what it starts, interpret deep internal data, and resist manipulative pressures.
As seen in the live setup, AI’s true value is its management skill—trustworthiness, decisiveness, and resilience—more than just raw analytical power or conversational finesse. Those qualities determine whether AI can truly augment your business operations or simply perform well in a demo.
Explore the live experiment and see how your enterprise could test its AI workforce at firmulate.com, or try running your own wargame against your business data at firmulate.com/pilot.html. Because when it comes to AI and management, the real test is whether it can deliver under pressure, not just in a chat.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
Flea & tick season Picks
flea and tick prevention
As an affiliate, we earn on qualifying purchases.
