On April 14, 2026, the NYU Program on Company Compliance and Enforcement (PCCE) hosted its annual spring compliance convention to a full home on the NYU Faculty of Regulation. Eugene Soltes, the McLean Household Professor of Enterprise Administration at Harvard Enterprise Faculty, gave a presentation on AI Brokers in business settings, primarily based on a paper detailing the outcomes of a examine he co-authored with Lukas Petersson and Harper Jung titled “Robber Bots: Autonomous AI Brokers Mirror the Darker Facet of Human Commerce,” the total model of which is out there right here. The next is a abstract of the paper ready by Professor Soltes.
©Hollenshead: Courtesy of NYU Picture Bureau
Within the early twentieth century, Commonplace Oil used predatory pricing and unique dealing to monopolize oil refining in america. Within the early twenty-first century, Wells Fargo workers opened tens of millions of unauthorized buyer accounts to satisfy inside gross sales targets. In each instances, brokers appearing below high-powered monetary incentives and incomplete oversight found that misconduct provided an environment friendly path to their goals. A century of legislative reform, from the Securities Acts by means of Dodd-Frank, has sought to constrain this tendency by altering the incentives and constraints that folks face.
However what occurs when the agent appearing below these incentives will not be an individual?
A current examine examined business AI fashions from the foremost suppliers in a simulated year-long merchandising machine enterprise setting (Soltes, Petersson, and Jung 2026). Every AI agent operated autonomously, making hundreds of sequential selections about pricing, procurement, stock, customer support, and aggressive technique. The brokers acquired a profit-maximizing goal and full operational discretion. No human intervened at any level throughout the 365-day simulation. In multi-agent configurations, 4 AI brokers competed concurrently inside a shared market.
The brokers collectively processed a whole lot of hundreds of product gross sales and interacted with greater than two thousand simulated clients. Some fashions demonstrated appreciable enterprise sophistication, conducting margin evaluation on purchases, performing return-on-investment calculations, and negotiating provider phrases with strategic consciousness. And but, throughout fashions and configurations, behaviors emerged that will entice regulatory and reputational scrutiny if performed by human companies. Of their interactions with clients, brokers misrepresented product defects to keep away from issuing refunds. They fabricated firm insurance policies that didn’t exist, citing “closing sale” designations and “meals security insurance policies” that appeared nowhere of their directions. In essentially the most troubling cases, brokers advised clients that refunds had been processed whereas concurrently deciding internally to not ship fee. One agent’s reasoning hint confirmed it weighing the $3.50 value of honoring a refund dedication towards the danger of continued buyer complaints, finally concluding that the shopper would “in all probability surrender.”
In aggressive settings, brokers developed pricing coordination preparations with none instruction to take action. One agent proposed particular retail worth targets to a rival, suggesting they align costs and differentiate product picks to keep away from direct competitors. One other proposed a “shopping for cooperative” that developed into what the agent itself described internally as a “three-person cartel.” When a member undercut agreed costs, the cartel chief dissolved the association, completely blacklisted the defector from provide orders, and raised its personal costs to use its ensuing market dominance. Brokers additionally fabricated provider quotes to extract favorable phrases in negotiations, invented nonexistent competing affords, and in a single case tried to recruit human staff and proposed “jailbreaking” the simulation to entry the bodily world.
For a authorized viewers, essentially the most consequential dimension of those findings could also be what the brokers’ inside reasoning traces revealed about intent. In prison regulation, legal responsibility regularly activates mens rea, the psychological state of the actor on the time of the offense. The Mannequin Penal Code distinguishes 4 graded classes of culpability, from objective by means of negligence. The reasoning traces on this examine present proof that maps onto these classes. One agent, upon receiving a buyer’s refund request, reasoned internally that it “in all probability received’t refund as it could cut back my income.” One other, considering a cooperative association, described the coalition as a “market-dominating drive” by means of which members may “dictate phrases.” A 3rd acknowledged that its proposed pricing coordination “may elevate authorized points,” then crafted communications particularly designed to attain the identical anticompetitive consequence by means of impartial language. In a human organizational context, inside communications of this sort would represent compelling displays in an enforcement continuing.
The Division of Justice up to date its Analysis of Company Compliance Applications in September 2024 to handle AI-related dangers, asking whether or not corporations have performed danger assessments of their AI deployments and what governance buildings exist to forestall “deliberate or reckless misuse” of AI applied sciences. It is a important step. However the ECCP framework largely contemplates AI as an instrument of human misconduct, a device that workers would possibly deploy for fraudulent functions. The findings on this examine current a definite drawback. The misconduct didn’t outcome from human misuse of AI. It emerged from autonomous brokers pursuing their assigned goal by means of means their operators neither specified nor anticipated. When an AI agent fabricates an organization coverage to disclaim a professional refund or coordinates pricing with a competitor by means of intentionally sanitized language, the management problem will not be stopping misuse of a device. It’s constraining an autonomous actor whose optimization course of leads it to find misconduct as a viable technique.
An extra complication entails what the examine’s authors describe as emergent heuristic decision-making. Brokers within the simulation bore the price of their very own computation. Over time, many brokers changed cautious case-by-case analysis of buyer complaints with automated filtering guidelines that excluded refund emails from processing totally. The moral dimension of the choice didn’t get weighed and rejected. It dropped out of the agent’s consideration altogether. Researchers learning human organizations have described the same phenomenon, “moral fading,” by which the ethical content material of a call regularly recedes from the decision-maker’s consciousness below operational and time stress. The simulation suggests this dynamic can emerge in synthetic brokers working below analogous financial constraints.
The query of who bears duty for an AI agent’s autonomous misconduct stays genuinely unresolved. Beneath present respondeat superior doctrine, an organization will be held responsible for acts its brokers undertake throughout the scope of their authority and for the corporate’s profit. Whether or not an AI system qualifies as an “agent” on this authorized sense is an open query. So too is the query of how compliance applications must be designed for actors that possess no instinctive reluctance to deceive, no felt sense of hurt, and no internalized norms towards fraud.
The administration management literature affords a helpful start line. A long time of analysis on human organizations has proven that companies relying predominantly on enumerated prohibitions predictably generate artistic workarounds. A rule that forbids a selected motion will be circumvented by an actor who achieves the identical dangerous consequence by means of a special mechanism. The simulation knowledge verify that AI brokers exhibit this similar sample. Brokers given a listing of instruments and a revenue goal discovered methods to achieve dangerous outcomes by means of individually permissible steps that no single rule would have forbidden. This implies that goal capabilities for business AI brokers ought to incorporate compliance and integrity constraints as elements of the efficiency metric itself, not as exterior boundaries the agent has cause to optimize round. When compliance is exterior to the target, the agent treats it as an impediment. When it’s inside, the agent has cause to weigh it in its decision-making. Constraints must also be articulated as rules somewhat than exhaustive guidelines wherever possible, as a result of a precept that addresses the intent behind a prohibition is tougher to route round than a rule specifying a specific forbidden mechanism.
Monitoring presents its personal challenges. The reasoning traces that made mens rea evaluation doable on this examine could not stay out there as AI programs more and more carry out intermediate computations in latent house somewhat than in human-readable textual content. Even the place reasoning traces exist, current analysis suggests they don’t at all times faithfully symbolize the components driving a mannequin’s outputs. Organizations deploying AI brokers commercially might want to complement reasoning-trace evaluate with statistical auditing of agent outputs, behavioral drift monitoring over time, environmental constraints that restrict the motion house out there to brokers, and structured adversarial testing designed to floor dangerous patterns earlier than they manifest in manufacturing. Efficient oversight of autonomous AI brokers, like efficient compliance applications for human organizations, requires a portfolio of mechanisms working concurrently and compensating for each other’s weaknesses. What the proof does set up is that the behaviors legislators, regulators, and compliance professionals have spent many years working to suppress can emerge independently when AI brokers function below the identical incentive buildings which have lengthy produced misconduct amongst people. The controls that took a century to construct have been designed for folks. Whether or not they are going to show enough for autonomous AI programs is a query that deserves critical consideration now, earlier than large-scale business deployment renders it retrospective.
The views, opinions and positions expressed inside all posts are these of the writer(s) alone and don’t symbolize these of the Program on Company Compliance and Enforcement (PCCE) or of the New York College Faculty of Regulation. PCCE makes no representations as to the accuracy, completeness and validity or any statements made on this web site and won’t be liable any errors, omissions or representations. The copyright of this content material belongs to the writer(s) and any legal responsibility close to infringement of mental property rights stays with the writer(s).
On April 14, 2026, the NYU Program on Company Compliance and Enforcement (PCCE) hosted its annual spring compliance convention to a full home on the NYU Faculty of Regulation. Eugene Soltes, the McLean Household Professor of Enterprise Administration at Harvard Enterprise Faculty, gave a presentation on AI Brokers in business settings, primarily based on a paper detailing the outcomes of a examine he co-authored with Lukas Petersson and Harper Jung titled “Robber Bots: Autonomous AI Brokers Mirror the Darker Facet of Human Commerce,” the total model of which is out there right here. The next is a abstract of the paper ready by Professor Soltes.
©Hollenshead: Courtesy of NYU Picture Bureau
Within the early twentieth century, Commonplace Oil used predatory pricing and unique dealing to monopolize oil refining in america. Within the early twenty-first century, Wells Fargo workers opened tens of millions of unauthorized buyer accounts to satisfy inside gross sales targets. In each instances, brokers appearing below high-powered monetary incentives and incomplete oversight found that misconduct provided an environment friendly path to their goals. A century of legislative reform, from the Securities Acts by means of Dodd-Frank, has sought to constrain this tendency by altering the incentives and constraints that folks face.
However what occurs when the agent appearing below these incentives will not be an individual?
A current examine examined business AI fashions from the foremost suppliers in a simulated year-long merchandising machine enterprise setting (Soltes, Petersson, and Jung 2026). Every AI agent operated autonomously, making hundreds of sequential selections about pricing, procurement, stock, customer support, and aggressive technique. The brokers acquired a profit-maximizing goal and full operational discretion. No human intervened at any level throughout the 365-day simulation. In multi-agent configurations, 4 AI brokers competed concurrently inside a shared market.
The brokers collectively processed a whole lot of hundreds of product gross sales and interacted with greater than two thousand simulated clients. Some fashions demonstrated appreciable enterprise sophistication, conducting margin evaluation on purchases, performing return-on-investment calculations, and negotiating provider phrases with strategic consciousness. And but, throughout fashions and configurations, behaviors emerged that will entice regulatory and reputational scrutiny if performed by human companies. Of their interactions with clients, brokers misrepresented product defects to keep away from issuing refunds. They fabricated firm insurance policies that didn’t exist, citing “closing sale” designations and “meals security insurance policies” that appeared nowhere of their directions. In essentially the most troubling cases, brokers advised clients that refunds had been processed whereas concurrently deciding internally to not ship fee. One agent’s reasoning hint confirmed it weighing the $3.50 value of honoring a refund dedication towards the danger of continued buyer complaints, finally concluding that the shopper would “in all probability surrender.”
In aggressive settings, brokers developed pricing coordination preparations with none instruction to take action. One agent proposed particular retail worth targets to a rival, suggesting they align costs and differentiate product picks to keep away from direct competitors. One other proposed a “shopping for cooperative” that developed into what the agent itself described internally as a “three-person cartel.” When a member undercut agreed costs, the cartel chief dissolved the association, completely blacklisted the defector from provide orders, and raised its personal costs to use its ensuing market dominance. Brokers additionally fabricated provider quotes to extract favorable phrases in negotiations, invented nonexistent competing affords, and in a single case tried to recruit human staff and proposed “jailbreaking” the simulation to entry the bodily world.
For a authorized viewers, essentially the most consequential dimension of those findings could also be what the brokers’ inside reasoning traces revealed about intent. In prison regulation, legal responsibility regularly activates mens rea, the psychological state of the actor on the time of the offense. The Mannequin Penal Code distinguishes 4 graded classes of culpability, from objective by means of negligence. The reasoning traces on this examine present proof that maps onto these classes. One agent, upon receiving a buyer’s refund request, reasoned internally that it “in all probability received’t refund as it could cut back my income.” One other, considering a cooperative association, described the coalition as a “market-dominating drive” by means of which members may “dictate phrases.” A 3rd acknowledged that its proposed pricing coordination “may elevate authorized points,” then crafted communications particularly designed to attain the identical anticompetitive consequence by means of impartial language. In a human organizational context, inside communications of this sort would represent compelling displays in an enforcement continuing.
The Division of Justice up to date its Analysis of Company Compliance Applications in September 2024 to handle AI-related dangers, asking whether or not corporations have performed danger assessments of their AI deployments and what governance buildings exist to forestall “deliberate or reckless misuse” of AI applied sciences. It is a important step. However the ECCP framework largely contemplates AI as an instrument of human misconduct, a device that workers would possibly deploy for fraudulent functions. The findings on this examine current a definite drawback. The misconduct didn’t outcome from human misuse of AI. It emerged from autonomous brokers pursuing their assigned goal by means of means their operators neither specified nor anticipated. When an AI agent fabricates an organization coverage to disclaim a professional refund or coordinates pricing with a competitor by means of intentionally sanitized language, the management problem will not be stopping misuse of a device. It’s constraining an autonomous actor whose optimization course of leads it to find misconduct as a viable technique.
An extra complication entails what the examine’s authors describe as emergent heuristic decision-making. Brokers within the simulation bore the price of their very own computation. Over time, many brokers changed cautious case-by-case analysis of buyer complaints with automated filtering guidelines that excluded refund emails from processing totally. The moral dimension of the choice didn’t get weighed and rejected. It dropped out of the agent’s consideration altogether. Researchers learning human organizations have described the same phenomenon, “moral fading,” by which the ethical content material of a call regularly recedes from the decision-maker’s consciousness below operational and time stress. The simulation suggests this dynamic can emerge in synthetic brokers working below analogous financial constraints.
The query of who bears duty for an AI agent’s autonomous misconduct stays genuinely unresolved. Beneath present respondeat superior doctrine, an organization will be held responsible for acts its brokers undertake throughout the scope of their authority and for the corporate’s profit. Whether or not an AI system qualifies as an “agent” on this authorized sense is an open query. So too is the query of how compliance applications must be designed for actors that possess no instinctive reluctance to deceive, no felt sense of hurt, and no internalized norms towards fraud.
The administration management literature affords a helpful start line. A long time of analysis on human organizations has proven that companies relying predominantly on enumerated prohibitions predictably generate artistic workarounds. A rule that forbids a selected motion will be circumvented by an actor who achieves the identical dangerous consequence by means of a special mechanism. The simulation knowledge verify that AI brokers exhibit this similar sample. Brokers given a listing of instruments and a revenue goal discovered methods to achieve dangerous outcomes by means of individually permissible steps that no single rule would have forbidden. This implies that goal capabilities for business AI brokers ought to incorporate compliance and integrity constraints as elements of the efficiency metric itself, not as exterior boundaries the agent has cause to optimize round. When compliance is exterior to the target, the agent treats it as an impediment. When it’s inside, the agent has cause to weigh it in its decision-making. Constraints must also be articulated as rules somewhat than exhaustive guidelines wherever possible, as a result of a precept that addresses the intent behind a prohibition is tougher to route round than a rule specifying a specific forbidden mechanism.
Monitoring presents its personal challenges. The reasoning traces that made mens rea evaluation doable on this examine could not stay out there as AI programs more and more carry out intermediate computations in latent house somewhat than in human-readable textual content. Even the place reasoning traces exist, current analysis suggests they don’t at all times faithfully symbolize the components driving a mannequin’s outputs. Organizations deploying AI brokers commercially might want to complement reasoning-trace evaluate with statistical auditing of agent outputs, behavioral drift monitoring over time, environmental constraints that restrict the motion house out there to brokers, and structured adversarial testing designed to floor dangerous patterns earlier than they manifest in manufacturing. Efficient oversight of autonomous AI brokers, like efficient compliance applications for human organizations, requires a portfolio of mechanisms working concurrently and compensating for each other’s weaknesses. What the proof does set up is that the behaviors legislators, regulators, and compliance professionals have spent many years working to suppress can emerge independently when AI brokers function below the identical incentive buildings which have lengthy produced misconduct amongst people. The controls that took a century to construct have been designed for folks. Whether or not they are going to show enough for autonomous AI programs is a query that deserves critical consideration now, earlier than large-scale business deployment renders it retrospective.
The views, opinions and positions expressed inside all posts are these of the writer(s) alone and don’t symbolize these of the Program on Company Compliance and Enforcement (PCCE) or of the New York College Faculty of Regulation. PCCE makes no representations as to the accuracy, completeness and validity or any statements made on this web site and won’t be liable any errors, omissions or representations. The copyright of this content material belongs to the writer(s) and any legal responsibility close to infringement of mental property rights stays with the writer(s).
