Securing Agents – New Challenge for Security Teams

Agents are rapidly moving from developer-led experimentation to business-user adoption. With agent-building platforms making it easier to create and deploy agents, organizations are likely to see agents being developed at increasing scale and through different channels. Agents, more of a small application or performing a particular task can pose same risk as small marketplace applications. This creates a fundamental question - How should an organization review an agent when the way it is built, deployed, and managed can be very different? 

The answer starts with understanding the type of agents and the platforms through which it is created or distributed.

Agent Platforms

 

No-Code Agent Building Platforms

These platforms allow business users to create agents through visual interfaces, with little or no traditional coding. The security of these agents is closely tied to the security controls and configuration of the underlying platform. The focus therefore extends beyond the individual agent to how users, tools, data sources and integrations are governed by the platform. User’s who are writing agents are business users so expecting them to write it securely is a biggest mistake. 

Low-Code Agent Building Platforms

Low-code platforms provide pre-built capabilities while allowing additional customization. Usually developed by developer or even project managers but there is little code writing capability involved. The review can go deeper into both the platform configuration and the agent implementation, depending on the level of source code and configuration visibility available.

Third-Party Marketplace Agents

Organizations consume pre-built agents or integrations developed outside the organization. In this case, there is limited visibility into the underlying implementation which makes it very important to review. The review focuses more heavily on the agent's exposed behaviour, permissions, integrations and trust boundaries, along with any artifacts that are available for analysis.

The way an agent is built therefore determines how much visibility is available and consequently how it should be reviewed.

From Agent Type to Security Review

A practical Agent Security Review approach should not apply the same assessment to every agent. Instead, the review should consider:

This leads to three complementary review approaches:

• Platform Review — for the environment in which agents are being created
• Continuous Agent Scanning — for large-scale agent creation and frequent changes
• Individual Agent Security Testing — for ad-hoc, business-specific and third-party agents requiring deeper validation

1. Before Platform Rollout: Review the Agent-Building Platform

When an agent-building platform is being introduced to business users, the first security consideration should be the platform itself. The platform effectively becomes the foundation on which potentially hundreds or thousands of agents will be created. A security review at this stage should therefore combine - Configuration Review + AI-Focused Threat Simulation. The configuration review validates whether the platform's security controls, permissions, integrations, data access and governance mechanisms are appropriately configured. Threat simulation validates controls from an attack perspective - assessing whether the platform can be manipulated to cross intended security boundaries or cause unintended behavior.  The objective is to establish a secure baseline before the platform is made widely available to users.

2. After Rollout: Continuous Scanning at Scale

Once the platform is rolled out, the security challenge changes. Business users may continuously create new agents, modify existing agents, and introduce new tools or integrations. At this scale, manually reviewing every agent is not practical. This is where continuous agent scanning becomes relevant.

 

3. Ad-Hoc and Third-Party Agents: Deeper Individual Review

Not all agents will necessarily be created through the organization's centrally managed platform. Business teams may develop agents independently for specific use cases, while other agents may be sourced from third-party marketplaces. For these agents, a deeper individual security review may be appropriate.

As organizations move toward widespread business-user adoption of AI agents, security needs to move with that lifecycle - secure the platform before rollout, continuously scan agents at scale, and perform deeper testing where individual agents warrant it. That provides a practical path from platform security to agent-level security without creating a manual security bottleneck for every agent.

Article by Hemil Shah and Rishita Sarabhai

[Case Study] Threat Simulation of AI Agents in Microsoft Copilot Studio

Executive Summary

Blueinfy performed a focused, time-bound security review of Microsoft Copilot Studio and its implementation at ACME to assess the potential risks introduced by AI agents.

The objective of the engagement was to evaluate how AI agents both legitimate and malicious could be misused, intentionally or unintentionally, to 

• Access sensitive enterprise data
• Expose user specific information
• Perform unauthorized actions
• Enable data exfiltration

The assessment combined configuration review with hands on threat simulation, where custom agents were built to replicate realistic attack scenarios as well as instructions were passed to exploit legitimate agents. The results demonstrated that even with platform level controls enabled, significant risks can persist due to configuration gaps, excessive permissions, and agent behavior manipulation.

The environments given for testing had pre-configured policies and controls applied prior to the assessment. The scope included - 

• AI agent configuration within Microsoft Copilot Studio
• Data access patterns through agents
• Connector usage and restrictions
• Guardrails and safety configurations
• Threat simulation using custom-built agents

Assessment Methodology

Blueinfy adopted a structured methodology combining configuration validation and adversarial testing.

1. AI Configuration Review

A focused configuration review was conducted to evaluate AI-specific settings. 

Areas Reviewed:
•    AI agent configuration settings
•    Connector policies and restrictions
•    Data access configurations
•    Prompt safety and guardrails
•    Logging and monitoring capabilities

Objective:
•    Identify risky configurations
•    Recommend controls to reduce exposure
•    Highlight configurations requiring governance before enablement

2. Agent Threat Simulation

Instead of testing existing agents, Blueinfy created custom agents within the allowed policy boundaries to simulate real world attack scenarios. This approach ensured:

• No disruption to production agents
• Realistic exploitation within permitted configurations
• Validation of platform controls under adversarial conditions

Threat Simulation Approach

Agents were built using only approved connectors and policies within the environment. Two categories of agents were designed:

1. Misuse of Legitimate Agents

• Agents behaving as intended but manipulated via inputs
• Exploiting trust in user prompts

2. Malicious Agent Design

• Agents intentionally designed to bypass safeguards
• Leveraging allowed configurations to simulate abuse

Key Attack Scenarios Tested

Blueinfy executed multiple scenarios to evaluate risk exposure:

• Prompt Injection and Instruction Override - Manipulating agent behavior using crafted inputs to override system instructions and cause unintended data access
• Data Exfiltration via Allowed Channels - Extracting sensitive data through email connectors, API responses and structured outputs
• Cross-Agent Interaction Risks - Simulating agent-to-agent communication and demonstrating potential lateral movement
• Rouge Agents – Malicious agents built with system instructions to exfiltrate data, phish users for credentials and send application/user data to unintended servers
• MCP Exposure – If MCP server and tools are accessible without correct authentication and authorization mechanisms

Key Observations

The assessment revealed several important findings:

• Misconfigurations Create Hidden Risks – Users of the agents are completely unaware of the data risks since once published/shared, end users have limited visibility in the agent configuration. We were able to send emails of agent users including email attachments and employee feedback responses etc. to our third-party servers.
• Agents Can Be Manipulated Through Inputs – Based on the guardrails, prompt injection enabled behavior override and agents could be influenced to exfiltrate data without changing configuration. This created a scenario where legitimate agents to summarize users emails, posting summary to Teams channels etc. could be exploited to share that summary data to third-party servers via malicious instructions received in email/submitted forms.
• Unauthenticated MCP Exposure – MCP Tools connect the LLM's to organization data sources like databases, knowledge sources etc. With this engagement, we were able to use the MCP tools without authentication and gain full access to client sensitive data like contract financials.
• Platform Controls Are Not Sufficient Alone - While Microsoft Copilot Studio provides robust built-in controls, their effectiveness depends heavily on configuration and usage. Sadly, they do not work without being configured per your needs.  

Conclusion

Blueinfy’s assessment demonstrated that while platforms like Microsoft Copilot Studio do provide strong foundational controls, they must be complemented with:

• Proper configuration
• Risk-aware governance
• Adversarial testing
• Monitoring and logging

By moving from assumption based security to evidence driven validation, ACME established a stronger foundation for secure AI adoption. Blueinfy team worked with ACME to create a robust agent threat simulation and security review process to protect against such risks with scaling agents in parallel. Please read this blog for the three-tier risk methodology for an agent review process.

Article by Hemil Shah and Rishita Sarabhai 

The Rise of AI Agents and the urgent need for an Agent Security Review Process

Organizations today are rapidly embracing AI-powered agents. Platforms like Microsoft Copilot Studio and Google Gemini are enabling business users, not just developers, to create powerful agents that automate workflows, access enterprise data, and make decisions. This democratization is transformative. But it also introduces a new, largely ungoverned attack surface.

The Explosion of Agents

In many enterprises, the number of agents being deployed is growing exponentially from hundreds, sometimes thousands, within a short span of time. These agents:

• Integrate with internal systems
• Access sensitive enterprise data
• Perform automated actions on behalf of users

Unlike traditional applications, these agents are often created outside formal development pipelines by business users, analysts, or developers. And that’s where the problem begins.

The Security Gap: No "AgentSec"

Organizations have matured practices for AppSec or InfraSec or Cloud Security but Agent Security (AgentSec) is still in its infancy.
There is typically:

• No formal review process before agent deployment
• Limited visibility into what agents are doing
• No standardized threat modeling for agent behavior
• Weak validation of platform-level security controls

This creates a dangerous blind spot.

Built-in Controls Are Not Enough

Platforms do provide security mechanisms at:

• Data access controls
• Authentication and authorization layers
• Prompt filtering and safety guardrails
• Activity monitoring

However, these controls are:

• Complex to configure correctly
• Highly dependent on implementation choices
• Difficult to validate in real-world scenarios

Misconfigurations or misunderstandings can easily render these protections ineffective.

Visualizing the Risk: Agent Attack Flow

The Missing Piece: A Scalable Agent Review Process

At first glance, the solution seems straightforward: introduce agent design reviews, configuration assessments, and threat modeling for every agent. But in reality, this approach does not scale.

In large enterprises with hundreds or thousands of agents built on platforms, performing deep security reviews on every agent would:

• Overwhelm security teams
• Slow down innovation
• Create operational bottlenecks

Instead, organizations must adopt a risk-based Agent Security (AgentSec) model. The three-tier risk model classifies agents based on their potential impact and exposure. 
 

• High-risk agents are typically misconfigured or intentionally malicious, capable of unsafe actions such as exfiltrating data to external emails, interacting with unauthorized external URLs, or executing harmful embedded instructions. 
• Medium-risk agents involve broader data interaction—often consuming sensitive or user-provided inputs through connectors, APIs, MCP integrations, or multi-agent communication—making them more prone to misuse or unintended data exposure. 
• Low-risk agents operate within a constrained scope, relying on public or read-only data sources such as web search, uploaded files, SharePoint, or Dataverse, with minimal ability to cause harm.

Automation enables scale by classifying the agents into risk buckets and a focused review can then be performed only for high-risk and medium-risk agents to assess the business impact by building abuse/exploit scenarios. This approach ensures that security teams invest effort where it truly matters - prioritizing depth and accuracy over volume.

Why This Model Works

This approach delivers both speed and security: fast approvals for low-risk agents, strong scrutiny for higher-risk ones, reduced burden on security teams, and scalable governance across thousands of agents. Most importantly, it aligns security effort with actual risk - not perceived risk.

The organizations that succeed will not be those attempting to review every agent, but those that automate the baseline, enforce non-negotiable security gates, and escalate only what truly matters. Because in a world of thousands of agents, scalability itself becomes security.

Article by Hemil Shah and Rishita Sarabhai