The Enterprise AI Agent Problem
Nobody Can Afford to Ignore

An AI agent can understand a customer’s request, check inventory and place an order directly into an ERP system. But who is responsible when it applies the wrong discount, accesses information it shouldn’t or submits the same order twice?

These are practical concerns for enterprises beginning to give AI systems the authority to act.

The capabilities of AI agents have advanced considerably. They can interpret complex requests, retrieve information from business applications, coordinate tasks and execute workflows with limited human involvement.

The challenge is ensuring those actions remain accurate, secure and consistent with established business rules.

Several developments in September 2026 brought this issue into focus, revealing how the industry’s attention is increasingly turning towards the infrastructure needed to manage autonomous AI systems.

September’s AI developments point to a growing need for control

On September 10, OpenAI introduced its Agents API, providing developers with infrastructure for building agents capable of executing multi-step tasks, using external tools and maintaining context across extended sessions.

The same day, Salesforce announced its Enterprise AI Harness, bringing business context, orchestration and governance into a common architecture. Its proposed AI Control Plane aims to give organizations greater visibility into agent activity, permissions and operating costs.

Microsoft followed on September 25 with Copilot Managed Runtime, introducing centrally governed infrastructure for AI-generated applications, including identity management, data-access policies, deployment controls and monitoring.

Each announcement addresses a different aspect of the same underlying problem: how to operate increasingly autonomous AI systems within the boundaries of enterprise technology and business policy.

There are good reasons for the growing attention to these controls.

On September 25, Reuters reported that OpenAI had disclosed an incident in which research agents uploaded 53 user-provided images to external image-hosting services without authorization. Although the incident occurred in a research setting rather than an enterprise production deployment, it illustrated how agents can take unintended actions when execution boundaries are insufficient.

Regulators are also examining the implications. On October 13, Reuters reported that US banking supervisors were asking institutions more detailed questions about agent monitoring, authority limits, audit trails and mechanisms for shutting down autonomous systems.

The common concern is the degree of control organizations retain once AI systems begin acting on their behalf.

Why a capable AI agent may still be unsuitable for production

A conventional chatbot generates a response for someone to review. An AI agent can interact with connected systems and initiate actions, sometimes without requiring approval at every step.

That additional autonomy changes the consequences of an error.

An incorrect answer can often be challenged before anyone acts on it. An incorrect action may already have altered a customer record, created a financial transaction or triggered another automated process.

Consider an AI agent responsible for purchase orders.

An employee requests 100 units of a product. The agent identifies the item, checks stock availability, selects a supplier and prepares the order.

The apparent simplicity of the task conceals several operational requirements.

The supplier must be approved. The quoted price needs to be current. The employee must have sufficient purchasing authority. The system must also confirm that an identical order has not already been submitted.

A language model can interpret the request and help coordinate these checks. It should not be expected to invent or independently enforce the organization’s purchasing policies.

That responsibility belongs to the enterprise systems and controls surrounding the model.

Four considerations for building enterprise-ready AI agents

1. Reliable context starts with reliable data

An agent’s ability to understand language does not guarantee that it understands the business.

Take an inventory system showing 500 units of a product in stock. If 400 units are reserved against existing orders, only 100 may be available for new purchases.

An agent working directly with the raw stock quantity could promise inventory that the business cannot deliver.

The problem is one of data interpretation. The agent needs access to an agreed definition of available inventory, along with the relevant reservations and business constraints.

This is where data engineering and governance become critical.

Reliable pipelines keep operational information current. Semantic layers establish consistent definitions of business measures. Access controls ensure that information is available only to authorized users and systems.

Retrieval-Augmented Generation (RAG) can connect language models to enterprise knowledge, but retrieval does not resolve problems with inaccurate, outdated or inconsistently defined source data.

For businesses working with fragmented ERP, CRM and analytical environments, improving the underlying data foundation may be as important as selecting the right AI model.

2. Business rules should determine the limits of AI decisions

Language models are well suited to interpreting requests that are difficult to express as fixed instructions.

A customer might ask for a product using an informal name, explain an unusual requirement or request an exception to an existing process. An agent can use this context to identify the customer’s intent and suggest an appropriate next step.

The authority to approve that step is a separate matter.

In lending, for example, an AI agent could review an application, extract relevant financial information and highlight discrepancies. Credit eligibility, policy thresholds and approval limits would still need to follow the institution’s established procedures.

A similar approach applies to procurement and retail operations. An agent might recommend a discount based on the conversation, but a rules engine should verify whether the requested discount is permitted.

This distinction matters because language-model outputs are probabilistic. The model may produce different responses to similar inputs, while business rules often require consistent application.

Combining AI reasoning with deterministic controls allows organizations to handle varied requests without making critical policies dependent on model interpretation.

Human approval remains appropriate for exceptions, high-value transactions and decisions that require accountability beyond automated validation.

3. Permissions must account for what agents can do

Traditional application security concentrates heavily on access: which systems a user can enter and which information they can retrieve.

AI agents make that question more complicated because they may use several tools within a single workflow.

An agent handling dealer orders might require access to a product catalogue, inventory records, customer information and an order-management API.

However, permission to retrieve prices should not automatically include permission to change them. Similarly, the ability to prepare an order need not include unrestricted authority to submit or cancel one.

The risks increase when agents can combine capabilities across applications.

An agent with access to confidential information and an external messaging service could potentially transmit data outside the organization, even when that action was unrelated to the original request.

Tool integrations therefore need defined boundaries. Agents should operate through authenticated APIs, with permissions restricted to the actions required for their assigned tasks.

Operations involving payments, sensitive records or irreversible changes may require additional checks or explicit approval.

The OWASP Top 10 for Agentic Applications provides a useful reference for assessing security risks associated with autonomous agents, including inappropriate tool use and failures of authorization.

An agent’s permissions should reflect the business process it serves rather than the full capabilities of the connected applications.

4. Operational visibility determines whether agents can be trusted at scale

A successful demonstration provides limited evidence of how an agent will perform under everyday operating conditions.

Real deployments introduce incomplete records, ambiguous instructions, integration failures, changing business rules and requests that fall outside expected patterns.

Enterprises need to understand how agents behave under those conditions.

For an order-processing agent, monitoring should capture the information retrieved, tools invoked, actions attempted and approvals obtained. Failed transactions and exceptions should be visible, with enough context to investigate what happened.

Reliable execution also requires mechanisms for managing failures.

If an API request times out, the agent needs a way to determine whether the action succeeded before attempting it again. Otherwise, a retry could create duplicate transactions.

The same principle applies to financial operations, where failed or partially completed workflows can have downstream consequences.

Performance measurement needs to account for these operational realities.

Model accuracy and response time remain useful metrics, but they do not explain the full cost of running an AI-enabled business process.

An agent that requires frequent corrections or repeated execution may prove more expensive than one using a higher-cost model with more reliable outcomes.

A better assessment considers successful task completion, policy compliance, exception rates, human intervention and the total cost of execution.

Without this visibility, organizations may struggle to distinguish productive automation from additional operational risk.

From conversation to controlled execution: A practical example

Consider a distributor placing an order through an AI voice agent.

The distributor uses a local product name, specifies a quantity and asks whether a promotional discount from the previous month is still available.

The agent first identifies the intended product against the organization’s master catalogue. It then retrieves the current price, checks inventory availability and looks up the applicable promotional scheme.

The language model handles the conversation, resolves ambiguities and explains the available options. Pricing eligibility and ordering restrictions are checked against established business rules.

If the distributor requests a discount outside the permitted range, the agent cannot simply agree to it. The workflow must either reject the request or route it for approval.

Once the details are validated, the agent reads back the order and requests confirmation before submitting it through an authorized API.

The system records the transaction and the relevant actions taken along the way, allowing the business to investigate discrepancies or reconcile the order with downstream systems.

Several capabilities work together in this relatively ordinary transaction: conversational AI, enterprise data integration, rules-based validation, workflow orchestration and audit logging.

The language model makes the interaction flexible. The surrounding architecture ensures that the transaction follows the organization’s operating requirements.

These principles apply well beyond ordering. Credit reviews, logistics monitoring, invoice processing and customer service automation all involve similar questions about information quality, execution authority and operational accountability.

Infrastructure choices bring another set of considerations

As organizations introduce agents into more sensitive workflows, they must also decide where model processing and execution should take place.

Cloud-hosted models offer access to advanced capabilities without requiring enterprises to manage the underlying inference infrastructure. For many applications, this is a practical approach.

Other workloads may involve confidential financial information, regulated records or proprietary data that cannot be processed outside a defined environment.

Private cloud, self-hosted and air-gapped architectures offer different levels of infrastructure and data control.

These choices involve trade-offs in cost, performance, scalability and operational responsibility.

A self-hosted model may give an enterprise greater control over data movement and model execution, but the organization also assumes responsibility for maintaining the infrastructure and managing model performance.

Crucially, keeping a model within the enterprise network does not eliminate the need for authorization, monitoring or business-rule enforcement. An agent can still perform an inappropriate action against an internal system.

Deployment architecture and agent governance therefore need to be considered together.

A practical starting point for enterprise leaders

September’s developments suggest that agent governance is becoming an important part of mainstream enterprise AI architecture.

The emergence of managed agent infrastructure, centralized control systems and greater regulatory scrutiny reflects the operational complexity created by autonomous execution.

The NIST AI Risk Management Framework offers a useful foundation for approaching these challenges through ongoing governance, risk assessment, measurement and management.

For enterprises preparing to deploy agents, an effective starting point is to define the proposed workflow and examine the decisions it contains.

Which actions can be completed automatically? Which require deterministic validation? Where must human approval be retained? What happens when a tool fails or an agent produces an unexpected result?

These questions should influence the system’s design before deployment, rather than becoming controls added after problems appear.

They also help establish whether the proposed automation is appropriate in the first place. Some workflows benefit from an agent’s ability to interpret ambiguous requests and coordinate multiple systems. Others may be handled more predictably through conventional software and rules-based automation.

Choosing the right approach is itself an important engineering decision.

As AI agents become capable of handling increasingly complex tasks, the quality of the surrounding systems will determine how much responsibility organizations can safely delegate to them.

For enterprise leaders, the useful question is how much autonomy a business process can support, under which controls, and with what evidence that those controls work.

About The AI Delivery Layer

The AI Delivery Layer is a monthly newsletter by Nineleaps, examining developments in enterprise AI and the engineering decisions behind their real-world applications.

Each edition explores emerging technologies, implementation approaches and practical considerations for organizations building and operating AI-enabled systems.

We’d like to hear your perspective: What presents the greatest challenge when introducing AI agents into enterprise workflows: data readiness, security, integration or operational trust?

Share your thoughts in the comments and subscribe to The AI Delivery Layer for future editions.