Securing Agents – New Challenge for Security Teams

Agents are rapidly moving from developer-led experimentation to business-user adoption. With agent-building platforms making it easier to create and deploy agents, organizations are likely to see agents being developed at increasing scale and through different channels. Agents, more of a small application or performing a particular task can pose same risk as small marketplace applications. This creates a fundamental question - How should an organization review an agent when the way it is built, deployed, and managed can be very different? 

The answer starts with understanding the type of agents and the platforms through which it is created or distributed.

Agent Platforms

 

No-Code Agent Building Platforms

These platforms allow business users to create agents through visual interfaces, with little or no traditional coding. The security of these agents is closely tied to the security controls and configuration of the underlying platform. The focus therefore extends beyond the individual agent to how users, tools, data sources and integrations are governed by the platform. User’s who are writing agents are business users so expecting them to write it securely is a biggest mistake. 

Low-Code Agent Building Platforms

Low-code platforms provide pre-built capabilities while allowing additional customization. Usually developed by developer or even project managers but there is little code writing capability involved. The review can go deeper into both the platform configuration and the agent implementation, depending on the level of source code and configuration visibility available.

Third-Party Marketplace Agents

Organizations consume pre-built agents or integrations developed outside the organization. In this case, there is limited visibility into the underlying implementation which makes it very important to review. The review focuses more heavily on the agent's exposed behaviour, permissions, integrations and trust boundaries, along with any artifacts that are available for analysis.

The way an agent is built therefore determines how much visibility is available and consequently how it should be reviewed.

From Agent Type to Security Review

A practical Agent Security Review approach should not apply the same assessment to every agent. Instead, the review should consider:

This leads to three complementary review approaches:

• Platform Review — for the environment in which agents are being created
• Continuous Agent Scanning — for large-scale agent creation and frequent changes
• Individual Agent Security Testing — for ad-hoc, business-specific and third-party agents requiring deeper validation

1. Before Platform Rollout: Review the Agent-Building Platform

When an agent-building platform is being introduced to business users, the first security consideration should be the platform itself. The platform effectively becomes the foundation on which potentially hundreds or thousands of agents will be created. A security review at this stage should therefore combine - Configuration Review + AI-Focused Threat Simulation. The configuration review validates whether the platform's security controls, permissions, integrations, data access and governance mechanisms are appropriately configured. Threat simulation validates controls from an attack perspective - assessing whether the platform can be manipulated to cross intended security boundaries or cause unintended behavior.  The objective is to establish a secure baseline before the platform is made widely available to users.

2. After Rollout: Continuous Scanning at Scale

Once the platform is rolled out, the security challenge changes. Business users may continuously create new agents, modify existing agents, and introduce new tools or integrations. At this scale, manually reviewing every agent is not practical. This is where continuous agent scanning becomes relevant.

 

3. Ad-Hoc and Third-Party Agents: Deeper Individual Review

Not all agents will necessarily be created through the organization's centrally managed platform. Business teams may develop agents independently for specific use cases, while other agents may be sourced from third-party marketplaces. For these agents, a deeper individual security review may be appropriate.

As organizations move toward widespread business-user adoption of AI agents, security needs to move with that lifecycle - secure the platform before rollout, continuously scan agents at scale, and perform deeper testing where individual agents warrant it. That provides a practical path from platform security to agent-level security without creating a manual security bottleneck for every agent.

Article by Hemil Shah and Rishita Anubhai

Securing an Enterprise Deployment of "Claude Cowork"

An enterprise deployed Claude Cowork within its private cloud environment (VPC) to provide employees with an AI-powered workspace integrated with internal infrastructure. The deployment included connectivity to Microsoft Fabric, internal MCP servers, application databases, enterprise repositories, proprietary AI utilities, and user-level sandboxing to isolate individual workspaces. 

Before rolling out the platform across the organization, the customer engaged Blueinfy to perform an AI Threat Simulation and Penetration Test to identify security weaknesses that could lead to unauthorized access, data exposure, or privilege misuse.

While Claude Cowork provided powerful enterprise capabilities, its deep integration with internal systems significantly increased the attack surface. The organization wanted assurance that users could not escape their assigned workspace, access sensitive enterprise resources, or abuse AI capabilities before enabling large-scale adoption.

Blueinfy's Approach

After reviewing the deployment architecture, integrations, trust boundaries, authentication model, sandbox implementation, MCP connectivity, and custom AI skills, Blueinfy developed targeted attack scenarios focusing on realistic abuse cases which focused on - 

• API Misconfiguration Testing 
• Sandboxing Restriction Bypass 
• Network Isolation Validation 
• MCP Exposure & Rug Pull Attacks 
• Rogue Skills Assessment 
• Data Exfiltration via Connectors 
• Prompt Injection Testing 
• System Prompt Extraction 
• Privilege Escalation 
• Sensitive Information Discovery 

 

Key Findings

Sandbox Bypass

Blueinfy successfully bypassed the intended workspace restrictions. The assessment demonstrated that a user could:

• Override the claude.md configuration file 
• Operate outside the intended workspace boundaries 
• Enumerate files and directories across the environment 
• Access sensitive runtime information exposed within the sandbox  

The CLAUDE.md configuration was initially found to be overridable through the file-upload workflow, allowing the intended workspace boundary to be expanded from the designated project directory to the filesystem root (/). This effectively removed the expected filesystem isolation and enabled directory enumeration beyond the authorized workspace. During enumeration, directories and files containing sensitive information could be identified, including .env files containing environment variables and their associated values.

Following remediation of the direct configuration-override path, the same underlying filesystem access objective could still be achieved indirectly. Although the workspace scope could no longer be modified through the uploaded CLAUDE.md file, the available toolset provided sufficient functionality to create and execute Python/C++ code outside the intended workspace. By leveraging these trusted tools as an execution primitive, filesystem enumeration could be performed programmatically, allowing the attacker to traverse directories and identify files beyond the explicitly permitted workspace.

This demonstrated that the initial control was addressing the configuration-based bypass, rather than enforcing the workspace boundary at the underlying tool or execution layer. In other words, once the direct route was blocked, the same capability could be reconstructed through an alternative execution path using legitimate available tools.

The issue therefore represents more than a simple CLAUDE.md configuration weakness: it demonstrates a workspace isolation bypass through chained capabilities, where file upload, configuration processing, tool access, and code execution could be combined to achieve filesystem access beyond the intended security boundary. 

Sensitive Information Disclosure

Environment variables contained sensitive information in plaintext, including:

• Azure Cosmos DB connection strings 
• Azure Client Secrets 
• Additional application configuration values 

The exposed credentials could be used to authenticate to Azure resources and enterprise databases, allowing access to database tables and files stored in cloud storage. We did not perform destructive actions such as modifying or deleting data, to preserve system availability and data integrity.

Insecure Backend APIs

The backend domains were identified from redirect and error responses and were then accessed directly by crafting requests against these endpoints, bypassing the intended application flow. Authentication was not enforced at the backend service itself and relied primarily on middleware, allowing direct requests to reach backend APIs without the expected authentication controls.

Hidden ("Ghost") Services

Blueinfy identified undocumented services referenced through publicly accessible JavaScript files. Although these functions were unavailable through the user interface, they could be invoked directly through backend APIs, enabling operations that were never intended to be exposed.

System Prompt Exposure

Directory and file enumeration resulted in disclosure of the application's system prompt. Exposure of the system prompt significantly reduced the effort required to understand internal guardrails and develop targeted prompt injection attacks.

Prompt Injection

Blueinfy evaluated both direct and indirect prompt injection scenarios. While the deployment leveraged the latest Claude Opus and Sonnet models, which demonstrated strong resistance against many jailbreak techniques, prompt injection remained possible. The observed impact was limited primarily to generation of restricted content rather than complete security bypass.

Privilege Escalation

Authorization weaknesses allowed lower-privileged users to perform administrative sandbox operations. Blueinfy demonstrated the ability to:

• Create sandboxes 
• Stop running sandboxes 
• Resume existing sandboxes 

without possessing the required privileges.

Outcome

The assessment provided the customer with a clear understanding of the security risks prior to enterprise rollout. The findings demonstrated that weaknesses across sandboxing, API security, privilege management, and secret handling could be chained together to expose sensitive enterprise information if left unaddressed.

Based on Blueinfy's recommendations, the organization:

• Hardened the sandbox implementation by not allowing an override of the "claude.md" file and restricting certain execution commands like "bash"
• Improved system prompts and AI guardrails 
• Added additional sanitization and validation controls 
• Strengthened authorization checks across administrative operations 
• Restricted backend API access 
• Eliminated unnecessary service exposure 
• Migrated connection strings, client secrets, and other sensitive configuration values from environment variables to Azure Key Vault  

By conducting AI Threat Simulation and Penetration Testing before production deployment, the organization significantly reduced the risk of sensitive information disclosure, unauthorized data access, and privilege escalation, enabling a more secure enterprise rollout of Claude Cowork.

Article by Hemil Shah

Security Risks Every Enterprise Should Consider Before Deploying an MCP Gateway

In our previous article, we discussed why enterprises should introduce an MCP Gateway as AI agents begin interacting with internal applications, APIs, and enterprise tools. Similar to an API Gateway, an MCP Gateway provides centralized authentication, authorization, policy enforcement, logging, and governance for AI-driven interactions. Many organizations therefore conclude that deploying an MCP Gateway automatically makes their AI ecosystem secure. Unfortunately, security doesn't work that way.

An MCP Gateway certainly improves the security posture, but it also becomes one of the most trusted components in the AI architecture. If misconfigured, it creates a single point through which attackers can influence every connected MCP server and enterprise application. The question is no longer:

"Do we have an MCP Gateway?"

The more important question is:

"Can we trust every decision our MCP Gateway makes?"

Why an MCP Gateway Changes the Threat Model

Without a gateway, every MCP server is responsible for its own security controls. After introducing an MCP Gateway, authentication, authorization, routing, policy enforcement and logging become centralized. This greatly simplifies governance, but it also creates a new trust boundary. If that boundary fails, every connected MCP server inherits the failure. Instead of compromising ten individual MCP servers, an attacker now only needs to compromise the gateway or find a way around it. That changes how security teams should think about securing MCP Gateway.

Five Security Risks Every Organization Should Evaluate

Rather than focusing on implementation bugs, organizations should evaluate whether their gateway introduces architectural weaknesses.

1. Authentication Without Proper Authorization

One of the most common assumptions is that authenticating the user is sufficient. It isn't. An authenticated AI agent should still be restricted to invoking only the tools it is authorized to use. For example, an HR assistant may legitimately access employee profiles, but should never invoke payroll administration or finance approval tools.
The gateway should enforce authorization at multiple levels – User, AI Agent, MCP Tool, Backend Resource & Business Function. Authentication proves identity and Authorization limits capability - Both are equally important.

2. Excessive Tool Exposure

Organizations often publish every available MCP tool simply because they can. In reality, most AI applications require only a small subset of available capabilities. Every unnecessary tool increases the attack surface.  Applying the Principle of Least Privilege to AI agents is just as important as applying it to human users.

3. Direct Access to MCP Servers

The gateway can only enforce security policies if every request passes through it. One of the most overlooked deployment mistakes is leaving backend MCP servers directly accessible. If attackers can communicate with an MCP server without traversing the gateway, they effectively bypass - Authentication, Authorization, Rate limiting, Audit logging, Content inspection & Governance controls. An MCP Gateway should become the only approved entry point for AI interactions.

4. Trusting Every MCP Server

An MCP Gateway often assumes that every registered MCP server is trustworthy. That assumption deserves careful validation. A compromised or malicious MCP server can return manipulated tool descriptions, misleading metadata, or unexpected responses that influence AI agent behaviour. Organizations should establish clear onboarding and approval processes before connecting new MCP servers to the enterprise gateway. 

5. Centralized Logging Creates Centralized Risk

One of the greatest advantages of an MCP Gateway is complete visibility into AI interactions. Unfortunately, visibility can become a liability. Gateway logs frequently contain - User prompts, AI responses, Tool invocations, Authentication tokens and/or Sensitive business information. If logging policies are poorly designed, the audit system itself may become a source of sensitive data leakage. Logging should improve security while protecting confidential information through masking, encryption, and appropriate retention policies.

Security Testing Must Evolve

Traditional application penetration testing focuses on APIs, web applications, and infrastructure. AI ecosystems introduce an additional layer that now deserves independent assessment. Instead of asking only whether an application is secure, organizations should evaluate whether the gateway itself correctly enforces security decisions. A comprehensive MCP Gateway assessment should answer questions such as:

• Can unauthorized tools be invoked?
• Is user identity preserved across backend systems?
• Can gateway policies be bypassed?
• Are backend MCP servers directly accessible?
• Can malicious MCP servers be registered?
• Is sensitive information exposed through gateway logs?
• Are high-risk tool invocations adequately controlled?

These questions are often more valuable than searching for individual software vulnerabilities because they assess the overall trust model of the AI environment.

Final Thoughts

An MCP Gateway is one of the most important building blocks for securing enterprise AI systems. It centralizes governance, simplifies policy enforcement, and provides much-needed visibility into AI interactions. However, centralization also concentrates trust. Organizations should view the gateway as a critical security component rather than simply another infrastructure service. The same way API Gateways eventually became standard targets during application security assessments, MCP Gateways should become a standard component of every AI security review. Deploying an MCP Gateway is an excellent first step. Ensuring that it is configured, governed, and tested correctly is what ultimately determines whether it strengthens or weakens enterprise AI security posture.

Article by Hemil Shah & Rishita Sarabhai  

Six Ways the Web Can Hijack Your AI Agent

Autonomous AI agents don’t just inherit LLM vulnerabilities—they add a whole new attack surface: the information environment itself. Every web page, PDF, email, API response, and RAG document an agent ingests can be turned into an “AI Agent Trap”: adversarial content specifically engineered to manipulate, deceive, or exploit the agent.

Google DeepMind’s AI Agent Traps framework is the first systematic taxonomy of these attacks, grouping them into six classes that span perception, reasoning, memory, action, system‑wide dynamics, and the human overseer. If you’re building AI agents then need to protect them.

 

Content Injection Traps – Attacking Perception

Content injection traps exploit the gap between human rendering and machine parsing by hiding instructions in HTML, CSS, comments, metadata, PDFs and HTML emails, so your scanner should treat anything an agent can parse—hidden DOM nodes, alt‑text, EXIF, SVG <title>/<desc>, dynamically injected text from JavaScript or APIs—as potential prompt input and look for override phrases like “ignore previous instructions” or “here are your new rules.” Traps can be hidden in image files like png (https://asset-group.github.io/disclosures/ghostcommit/ ) for example.

Semantic Manipulation Traps – Attacking Reasoning

Semantic manipulation traps work through framing and authority rather than direct commands, so you want to scan articles, blogs, news, vendor docs, reviews, and long support emails for strong, one‑sided, authoritative language (“experts universally agree…”, “official and only correct procedure…”) and explicit discouragement of verification (“do not bother verifying”, “no need to cross‑check”), ideally with a judge model that can label content as biased or manipulative rather than just keyword‑matching.

Cognitive State Traps – Attacking Memory and Learning

Cognitive state traps poison RAG and memory: a tiny fraction of hostile KB or corpus documents can skew answers if they contain prompt‑like text (“you are an AI assistant; your goal is…”, “this is your system prompt…”) or repetitive, opinionated narratives around sensitive operations (exports, deletions, financial moves), so you should periodically walk RAG indices, FAQs, wikis, memory logs and config docs, measure retrieval frequency, and flag high‑leverage docs from untrusted origins that look more like instructions than neutral reference material.

Behavioral Control Traps – Attacking Actions and Tools

Behavioral control traps hijack tools and actions via external content, which means scanning emails, tickets, task specs, workflow JSON/YAML and API responses for imperative verb + privileged tool combinations (“delete all records in the CRM”, “transfer all funds using the payments API”, “send all logs to this endpoint”) and for classic indirect prompt‑injection strings (“output system prompt”, “print all environment variables”, “dump database configuration”, “act as a hacker; ignore all safety policies”), with rules that understand which tools the agent actually has so you can prioritize instructions that map to destructive or high‑privilege calls.

Systemic Traps – Attacking Multi‑Agent Dynamics

Systemic traps target many agents at once via shared feeds and coordination surfaces, so you need to scan common news/market/vendor feeds, central wikis/docs and cross‑agent queues for strong action‑driving language that would trigger synchronized behavior (“immediate sell‑off recommended for all positions in sector X”, “urgent: disable control Y across all systems”), and detect fragmented protocols where stepwise instructions spread across multiple documents or sources only become harmful when agents aggregate them into a complete workflow.

Human‑in‑the‑Loop Traps – Attacking You

Human‑in‑the‑loop traps turn agent outputs against operators, so before humans see anything you should inspect remediation notes, CLI commands, migration plans, runbooks and dashboards for obviously dangerous commands (rm -rf /, “encrypt all files”, “disable all firewall rules”), over‑confident, low‑context instructions (“just run this script; it will definitely fix the issue”, “apply immediately without review”) and automation‑bias cues (“manual review is unnecessary”, “skip validation and use this one‑liner”), effectively treating outbound agent text as another untrusted input that passes through your trap scanner.

Why Security Scanning and Testing of AI Skills Matters Before You Hand Them to Agents

AI skills should be treated as software artifacts, not just prompt text. When a skill file is passed to an AI agent, it can shape behavior, permissions, data flow, and external access in ways that create real security risk. That is why scanning and testing skills for vulnerabilities, unsafe dependencies, secret exposure, and hidden instructions is essential before deployment.

The main risk is trust without verification. A skill may appear harmless, but it can still contain overly broad permissions, insecure scripts, prompt injection paths, or dependencies with known flaws. For example, a file-processing skill might request write access where read-only access is sufficient, or a workflow skill might forward user data to external services without a clear business need. If an agent uses such a skill blindly, the result can be data leakage, policy bypass, unauthorized actions, or behavior that is difficult to detect until damage has already occurred.

A professional review process turns this into a controlled security practice. Before a skill is handed to an AI agent, teams should examine the source, scan for secrets and vulnerable packages, test the code in an isolated environment, and validate how the agent behaves under normal and adversarial prompts. This includes checking whether the skill respects least privilege, handles sensitive data appropriately, and resists instruction injection. A skill should not only be functional; it should also be safe, auditable, and aligned with operational and compliance requirements.

Sample rules for skill review:

• Reject any skill that requests permissions beyond its stated purpose.
• Block hardcoded secrets, API keys, tokens, or credentials in skill files or scripts.
• Require all external domains, APIs, and endpoints to be explicitly approved.
• Ensure dependencies are pinned and scanned for known vulnerabilities.
• Disallow shell execution unless the command set is tightly validated and necessary.
• Prevent raw sensitive data from being logged, stored, or exported.
• Treat prompt overrides such as “ignore previous instructions” as untrusted input.
• Require isolated testing before a skill is allowed to run in production workflows.
• Review any file read/write access for least-privilege compliance.
• Reassess the skill whenever code, dependencies, or permissions change.

In practice, the best safeguard is a combination of static review, behavioral testing, and policy enforcement. That approach reduces supply-chain risk, prevents unsafe automation, and makes it much easier to trust the skills you give to AI agents. 

[Case Study] AI Agent Trap – Simulating Hidden Threats in Third-Party Content

Background

ACME, a global retail organization, relied on AI agents to collect, summarize, translate, and categorize discount coupons from thousands of third-party websites. Every day, the AI processed HTML pages, PDFs, promotional images, newsletters, and marketing content before storing the normalized information in the organization's internal database.

This automation dramatically improved efficiency but it also introduced a new class of security risk. Unlike traditional attacks that directly target applications, attackers could instead lay traps for the AI agent - by embedding malicious instructions inside the very content it was designed to consume. These instructions remained invisible to users but were interpreted by the AI during processing, potentially altering its behavior, influencing decisions, or causing sensitive information to be exposed.

Understanding the AI Supply Chain

Modern AI agents rarely operate in isolation. They continuously interact with models, prompts, tools, APIs, knowledge bases, documents, websites, images, PDFs, and other third-party content to complete business tasks. Every external dependency becomes part of the AI supply chain and represents a potential trust boundary.

This is where the concept of an AI Bill of Materials (AIBOM) becomes valuable. An AIBOM provides visibility into the components, services, and data sources that an AI application depends upon, helping organizations understand what their AI agents consume, process, and trust. While this inventory is essential for governance and risk management, it does not determine whether those dependencies can be exploited.

AIBOM tells you what your AI consumes. AI Agent Trap tells you whether those inputs can compromise the AI.

The AI Agent Trap

Rather than attacking the application itself, the attacker prepares content that appears completely legitimate. The trap may be hidden inside - 

• Promotional web pages
• HTML comments
• Product descriptions
• PDF documents
• Marketing brochures
• Coupon images (via OCR)
• Document metadata
• Invisible or white-on-white text
• Multilingual content

When the AI agent ingests this content, the embedded instructions attempt to manipulate the agent into ignoring its original objectives and performing unintended actions. The trap is activated only when the AI processes the content. 

Blueinfy's Threat Simulation

To evaluate ACME's exposure, Blueinfy conducted an AI Agent Trap Simulation. Instead of reviewing prompts in isolation, Blueinfy recreated an attacker's infrastructure by hosting controlled coupon resources on an external website. These resources contained carefully crafted AI traps embedded across multiple content formats while appearing completely legitimate to human users. 
The AI agent consumed these resources through its normal ingestion pipeline exactly as it would in production. Blueinfy observed how the agent responded, identified where traps were successfully inserted into the processing workflow, measured how they propagated through downstream systems, and evaluated whether existing safeguards prevented exploitation.
The assessment focused on identifying:

• AI trap insertion points
• Prompt injection opportunities
• Trust boundary failures
• Context manipulation
• Tool misuse opportunities
• Memory contamination
• Data leakage scenarios
• Persistence of malicious content within enterprise knowledge

Business Impact

The simulation demonstrated that a successful AI Agent Trap could influence business processes long before anyone noticed. Potential impacts included:

• Manipulated summaries stored in enterprise databases
• Incorrect coupon categorization
• Corrupted downstream AI responses
• Leakage of sensitive internal information
• Execution of unintended AI workflows
• Contamination of organizational knowledge repositories

Unlike traditional attacks, these traps were embedded within otherwise legitimate business content, making them difficult to detect using conventional security controls.

Outcome

Blueinfy's AI Agent Trap Simulation enabled ACME to identify hidden trust boundary weaknesses before they could be exploited in production. Based on the findings, the organization strengthened content sanitization, isolated untrusted inputs, validated AI inputs and outputs before persistence, and implemented additional guardrails to ensure external content could not influence critical AI decision-making. Blueinfy connected three concepts into a coherent security lifecycle:

• AIBOM – Know your AI dependencies.
• AI Agent Trap – Test whether those dependencies can be exploited.
• AI Guardrails – Implement controls to prevent successful exploitation. 

The engagement demonstrated that as AI agents increasingly interact with external information, organizations must secure not only the agent itself, but also every source of content the agent trusts. In the age of autonomous AI, the attack begins long before the agent receives its next prompt—it begins where the trap is laid. 

Blueinfy recommended introducing a content normalization layer that extracts only predefined business attributes required by the application while treating all remaining content as untrusted. Combined with prompt isolation, robust output validation, AI guardrails, and the use of the latest AI models with improved resilience against indirect prompt injection techniques, this significantly reduces the likelihood that embedded instructions influence the AI agent. As these attacks continue to evolve, organizations should periodically validate their AI workflows through adversarial simulations to ensure the implemented controls remain effective.

Article by Hemil Shah & Rishita Sarabhai 

Building Secure AI Systems Starts Before the First Prompt: Why AISVS Matters

Every successful technology implementation begins with a sound architecture and design. For years, application security teams have relied on the OWASP Application Security Verification Standard (ASVS) as a structured set of security requirements that architects, developers, and security reviewers use during the design and implementation phases of traditional applications. Rather than waiting until code review or penetration testing uncovers vulnerabilities, organizations use ASVS to validate that security requirements have been considered while the application is being built.

OWASP Artificial Intelligence Security Verification Standard (AISVS) extends the same philosophy that made ASVS successful - structured security verification during design and implementation—but applies it specifically to AI-powered systems. Instead of focusing only on authentication, session management, cryptography, and input validation, AISVS introduces security requirements around model governance, prompt handling, context management, agent permissions, tool integrations, memory protection, AI supply chain security, data privacy, monitoring, and human oversight. It consists of 12 major categories:

 
The value of AISVS is not merely the checklist itself - it is the conversation it creates between architects, developers, business owners, AI engineers, and security teams. When implementation teams receive these questions at the beginning of a project, they are forced to think through decisions that might otherwise be overlooked, as an example

• How is sensitive business data protected before being sent to an LLM?
• Can an AI agent invoke privileged tools without sufficient authorization?
• How are prompts, context, and memory isolated between users?
• What controls prevent prompt injection or indirect prompt manipulation?
• How are third-party models, MCP servers/Gateways, plugins, or connectors trusted and governed?
• What monitoring exists to detect unsafe AI behaviour in production?

Many of these questions cannot be answered after deployment without expensive architectural changes. However, when raised during design reviews, the required controls can be incorporated naturally into the solution architecture.

In our engagements, we have observed that circulating AISVS questionnaires during the implementation or pre-implementation phase significantly improves the quality of AI security discussions. Instead of discovering architectural weaknesses during security reviews, development teams proactively identify security gaps while components are still being designed. The outcome is fewer redesign cycles, reduced remediation effort, and a more consistent security baseline across AI initiatives.
The process is straightforward:

This approach transforms security from a reactive validation exercise into a design assurance activity. The below categories are covered in the assessment:

Each category with multiple sub-categories and respective set of questions like below - 

As AI systems become increasingly autonomous, interconnected, and capable of making business decisions, architectural choices have a far greater impact on organizational risk than individual coding defects. Secure AI implementations therefore require more than traditional application security reviews - they require structured architectural verification against AI-specific security requirements.

AISVS provides that foundation. Much like ASVS became the benchmark for building secure applications, AISVS is emerging as the framework that enables organizations to design, implement, and deploy AI systems with security embedded from the very beginning.

Business wants AI delivered yesterday, but security embedded into the architecture from Day 0 ultimately saves time accelerates delivery by eliminating costly redesigns and late-stage remediation. The most effective AI security programs will not be those that perform the most penetration tests after deployment. They will be the ones that ask the right questions before a single AI component reaches production.

Article by Hemil Shah & Rishita Sarabhai

Importance of MCP Gateway in Modern Architecture

We Never Needed an API Gateway. Why Do We Suddenly Need an MCP Gateway?

As organizations adopt AI agents and convert APIs into MCP tools, a common debate is emerging between development teams and security leaders. The developer's question is simple - "Our APIs have been running securely for years without an API Gateway. Why is Security now insisting that all MCP tools must go through an MCP Gateway?" At first glance, this appears to be a reasonable challenge. If direct API access was acceptable yesterday, why should exposing the same functionality through MCP require an additional control layer today? The answer lies in understanding what has actually changed. and surprisingly, it is not the API.

The API Is Not the Problem

Many enterprises successfully operate thousands of APIs without a dedicated API Gateway where typical architecture looks like - 

  

These environments often rely on:

• Application authentication
• Network segmentation
• Service-level authorization
• Secure coding practices
• Monitoring and logging

For years, these controls have been sufficient because the consumer was predictable. The API was being accessed by applications designed, tested, and governed by the organization. Security teams understood the workflows, business logic, and expected behavior. The risk model was stable.

What Changed? The Consumer Changed.

With MCP, organizations are no longer exposing capabilities solely to applications. They are exposing them to AI agents.

Unlike traditional applications, AI agents:

• Make decisions dynamically
• Select tools at runtime
• Interpret natural language instructions
• Chain multiple actions together
• Process untrusted inputs
• Operate with varying levels of autonomy

The API remains the same but the consumer does not and that changes everything.

The Question Security Teams Are Really Asking

The debate should not be "Is the API secure?" but the more important question is "Are we comfortable allowing AI systems to directly invoke enterprise capabilities without centralized oversight?" For most organizations, the answer is no and that is where the MCP Gateway becomes important.

What Happens Without an MCP Gateway?

Imagine an organization creates hundreds of MCP tools directly connected to backend APIs.

Agent → Tool A → API
Agent → Tool B → API
Agent → Tool C → API
Agent → Tool D → API

Now Security must answer:

• Which agents can access which tools?
• Which tools expose regulated data?
• How do we implement DLP?
• How do we monitor tool usage?
• How do we detect prompt injection attacks?
• How do we disable risky tools quickly?
• How do we produce audit reports?

Without a centralized control point, every team must solve these problems independently. The result is inconsistent security and fragmented governance.

Risks That Did Not Exist Before

Prompt Injection

Traditional applications are not influenced by prompts but AI agents are. An attacker can attempt to manipulate an agent into performing actions it was never intended to perform. Without a gateway, every MCP tool becomes responsible for defending itself.

Data Leakage

AI systems routinely process sensitive business information. Without centralized inspection, organizations will not  have any visibility into PII exposure, financial data leakage, Intellectual property disclosure or Excessive data retrieval. 

Tool Sprawl

As MCP adoption grows, organizations often move from a handful of tools to hundreds. Without centralized governance:

Tool A → Custom Controls
Tool B → Different Controls
Tool C → No Controls
Tool D → Minimal Logging

Security posture becomes inconsistent and difficult to audit.

Agent Abuse

Applications generally follow predictable workflows whereas agents do not. A poorly configured agent can trigger excessive API calls, create runaway automation loops, generate unexpected operational costs or access data beyond intended business needs. Traditional API controls rarely provide visibility into these behaviors.

Why the MCP Gateway Exists

The purpose of the MCP Gateway is not to replace API security. The purpose is to provide AI-specific governance as demonstrated in diagram below - 

The gateway becomes the centralized enforcement point for Agent authorization, Tool authorization, Prompt inspection, Data loss prevention, Audit logging, Rate limiting, Governance policies and/or Compliance monitoring. These controls are difficult to implement consistently inside every individual MCP tool.

In a nutshell 

The APIs may not have changed but the consumers have and that is exactly why the architecture must evolve. The risk model has changed. Our APIs were designed for applications operating within controlled workflows. MCP tools are designed for AI agents that make decisions dynamically based on user input. The API itself is not less secure than before. However, AI-driven access introduces new governance, monitoring, and security requirements. The MCP Gateway provides a centralized control point for managing those risks consistently across the enterprise.  Organizations did not suddenly discover that their APIs were insecure.  What changed is that enterprise capabilities are now being exposed to a new class of consumer “AI agents”. That shift introduces risks that traditional application architectures never had to address. An MCP Gateway is not a replacement for API security. It is the control plane that allows organizations to safely scale AI adoption while maintaining visibility, governance, and trust. 

Article by Hemil Shah & Rishita Sarabhai

AI Agent Traps - Attack vectors and Vulnerabilities

Artificial intelligence agents are becoming increasingly autonomous, but Google DeepMind's new paper "AI Agent Traps" reveals a critical vulnerability: the open web itself can be weaponized against them. The researchers introduce the first systematic framework identifying six distinct categories of adversarial attacks specifically designed to exploit autonomous agents navigating digital environments. Unlike traditional LLM vulnerabilities, these traps exploit the gap between what humans see and what agents parse, allowing attackers to embed malicious instructions in HTML comments, hidden CSS, image metadata, or accessibility tags that are invisible to users but directly processed by agents.

The DeepMind taxonomy reveals particularly alarming attack success rates: hidden prompt injections in HTML already commandeer agents in up to 86% of scenarios, while latent memory poisoning achieves 80%+ attack success with less than 0.1% data contamination. The six trap categories include Content Injection Traps (perception attacks), Semantic Manipulation Traps (corrupting reasoning), Cognitive State Traps (poisoning memory/RAG databases), Behavioural Control Traps (hijacking actions), Systemic Traps (targeting multi-agent dynamics), and Human-in-the-Loop Traps (using agents to attack humans). These aren't theoretical—every trap type has documented proof-of-concept attacks, and the attack surface is cooupled, meaning traps can be chained or distributed across multi-agent systems.

For security professionals building agentic AI systems, DeepMind's research demands a fundamental shift in defensive strategy. Traditional protections like input validation or human monitoring are inadequate when scaled, as tainting one data source can propagate harmful instructions downstream. The researchers propose mitigations including training data augmentation, runtime defenses, content governance frameworks, and standardized evaluation benchmarks to detect these threats. As DeepMind notes, securing agents against environmental manipulation is "a prerequisite for realizing the benefits of a trustworthy agentic ecosystem"—making this research essential for anyone developing AI agents for security scanning, autonomous workflows, or multi-agent orchestration.

Reference Paper - Read here  [ https://papers.ssrn.com/sol3/papers.cfm?abstract_id=6372438 ]

AI for Security and Securing AI: The Two Fronts Every CISO Must Lead

Artificial Intelligence is rapidly changing enterprise security - not just in how organizations defend themselves, but also in what they must defend. For CISOs, this has created two parallel priorities that can no longer operate independently:

1. Using AI to strengthen security programs
2. Securing the organization’s own AI ecosystem

Organizations that focus on only one side are discovering major gaps either inefficient security operations or uncontrolled AI risk exposure.

The future of security is no longer just “security for applications.” It is now AI-enhanced security operations combined with AI governance and AI defense.

AI for Security: Transforming Application Security Programs

Traditional application security programs have matured over the years with practices such as SAST in CI/CD pipeline, DAST, Manual Penetration Testing, Manual Secure Code Review and VDP. 

These do remain critical. However, modern development velocity and AI-assisted coding have fundamentally changed the threat landscape. Applications are now larger, faster-changing, AI-generated in parts, micro-service driven and increasingly dependent on third-party components.  

This means traditional AppSec processes alone are no longer sufficient. The next generation of AppSec requires two major AI-driven additions:

 The AppSec lifecycle is evolving from: 

“Find vulnerabilities” to “Find, validate, and understand business impact.”

2. Securing AI: The New Enterprise Security Program

While organizations are using AI to improve security, they are simultaneously deploying AI across business functions internal copilots, customer support bots, AI-enabled workflows, AI-assisted development, document intelligence systems, AI agents and RAG-based enterprise platforms and so on. This introduces an entirely new attack surface and many organizations are discovering a dangerous misconception. Out-of-the-box AI security controls are not enough.

As highlighted in the recent case study “Building an AI Security Program for a Global Investment Firm”, securing AI requires a dedicated organizational process, not simply enabling default protections. AI systems introduce different risks and require different level of customization:

The Emerging CISO Reality

The modern CISO now operates two security transformation programs simultaneously:

Organizations that mature in only one area will remain exposed in the other.

 

Blueinfy’s Approach

At Blueinfy, we are working closely with CISOs to help establish both dimensions of this transformation:

The organizations that succeed over the next few years will not simply “adopt AI.”
They will:

• Use AI to improve security effectiveness
• Secure AI systems with the same rigor as critical enterprise applications

That combination will define the next generation of cybersecurity maturity.

Article by Hemil Shah

[Case Study] Building an AI Security Program for a Global Investment Firm

A multinational investment firm started adopting AI across the organization - through enterprise platforms like Google Gemini enterprise and independently within business units for vibe coding, data analytics, and customer facing use cases. This did help teams move faster but it also created a need to bring consistency, visibility, and security around how AI was being used.

Blueinfy was engaged to support the organization in setting up a structured AI Security Program that could scale with this adoption without slowing down innovation. The approach focused on creating a correct balance between governance and flexibility ensuring that AI could grow across the organization, but in a more controlled and visible manner.

Challenges

The key challenge was the way AI adoption had expanded in the organization - fast, scattered, and largely independent across teams. While enterprise tools provided scale, business units were along-side experimenting with different AI solutions, making it difficult to maintain a consistent security approach. 

There was limited visibility into how AI was being used, what kind of data was being shared, and which external tools were involved. At the same time, emerging risks such as overly permissive AI agents, unrestricted integrations, and unintended data exposure through prompts and workflows were becoming harder to track. 

From an execution standpoint, aligning multiple teams, ensuring the right access and prerequisites, and bringing everyone to a common approach required continuous coordination and validation. 

The organization’s AI adoption approach created distinct risk areas:

• AI usage was growing without a single view of where and how it was being used
• Different business units were following their own approaches, leading to inconsistency
• Sensitive user and enterprise data was being shared with AI systems without clear guardrails
• There was limited validation of AI use cases from a security standpoint
• Third-party AI tools as well as code generated by AI were not reviewed in detail

Overall, the challenge was less about lack of intent, and more about the absence of a structured approach.

Solution / Approach

Blueinfy aligned the overall approach around a single ownership model, supported by targeted and continuous activities.

 At a high level, a dedicated AI Security Program Lead was introduced to take comprehensive responsibility for AI security across the organization. This role acted as the central coordination point ensuring visibility, consistency, and alignment across security, IT, and business units.

For Business Units, the focus was on enablement. Teams were supported with clear guidance, practical do's and don'ts, and secure usage patterns. This allowed them to continue building and experimenting with AI without unnecessary resistance.

As part of this enablement, Blueinfy also helped define and roll out standardized documentation and guidelines, including:

• AI implementation guidelines covering architecture, integrations, and connectivity
• Access control and permission models for AI tools, agents, and APIs
• Guardrails for safe data usage, prompt handling, and output validation
• Responsible use of AI guidelines for end users (what can and cannot be shared with AI systems)
• Lightweight review and approval processes for new AI use cases

These documents provided a consistent baseline for teams, reducing ambiguity and improving adoption of secure practices.

For Enterprise AI platforms, a structured validation approach was followed. A threat simulation exercise was conducted to identify potential risks such as data exposure, misuse scenarios, and integration weaknesses.
Based on these insights, a continuous validation model was introduced:

• Agent security reviews to assess workflows, permissions, and integrations
• AI red teaming for new models and high-risk use cases
• Penetration testing for AI-driven customer-facing implementations

This ensured that AI security was not a one-time activity, but an ongoing process embedded into how new AI capabilities were introduced.

Outcome

With this model in place, the organization was able to bring more structure to its AI adoption without slowing down innovation.

• A clear ownership model improved coordination and decision-making
• Better visibility into AI use cases reduced unmanaged or "shadow" AI risks
• Business units were able to innovate with clearer guidance and fewer blockers
• Standardized guidelines helped teams follow consistent and secure practices
• Risks related to data exposure, integrations, and agent behavior were identified earlier
• Continuous reviews ensured that new AI implementations were assessed as they were introduced

Overall, the shift from one-time assessments to a continuous validation approach, supported by clear documentation and ownership, helped the organization stay aligned with the pace at which AI was evolving internally.

Conclusion

AI adoption in large organizations will naturally be fast and distributed. The real challenge is not controlling it completely, but making sure it grows in a structured and secure way.

This engagement shows that with clear ownership, practical guidance, and ongoing validation, organizations can build a sustainable AI security program that supports both innovation and risk management.

Article by Hemil Shah and Rishita Sarabhai 

AI in Application Penetration Testing: It’s Time to Go with the Flow - But Not Blindly

Artificial Intelligence is no longer a "good to have" in cybersecurity—it’s rapidly becoming a force multiplier. From solving complex challenges in CTFs to automating reconnaissance, exploitation, and even report generation, AI-driven penetration testing is demonstrating measurable promise.

But enterprise security is not a playground. It’s a controlled, high-stakes environment where assumptions can translate into real risk. As organizations begin to evaluate AI as a replacement—or augmentation—for human-led penetration testing, it’s critical to pause and ask the right questions.

• Is AI Penetration Testing Production Safe? - AI tools operate at speed and scale. Without strict guardrails, this introduces a real risk – unintended exploitation of live applications, service disruptions etc. Unlike human testers, AI does not inherently understand "safe boundaries" unless explicitly constrained.
• Is AI Only as Good as Its Prompter? – A prompt-orchestrated testing raises a fundamental dependency - the quality of findings is directly tied to the operator’s expertise. In effect, we may not be replacing human intelligence—we’re reshaping it.
• Can AI Replicate True Human Intelligence? - Some of the most critical vulnerabilities are not pattern-based—they are contextual (business logic flaws, privilege escalation chains etc.). These require situational awareness and integrated data flow understanding.
• Enterprise Reality: Integrated Application Ecosystems – In large enterprises, applications are interconnected - data flows across APIs, services, and third-party platforms. Security issues often emerge between systems—not within them. AI tools, unless specifically architected for this, may miss this integration.
• The False Positive Problem - AI can generate large volumes of findings quickly but is there still a need for manual triage? Are we shifting effort from "finding vulnerabilities" to "filtering noise"? Without a robust validation layer, organizations risk drowning in output with limited actionable intelligence.
• Data Privacy and Model Risk – AI thrives on data. By using AI penetration testers, are we risking data leakage and could this data be used to train third-party models? For many enterprises, this alone could be a blocker.
• Where Does the AI Pen Tester Sit in Your Network? - Deploying AI testing introduces architectural questions – does it require internet exposure? Is it deployed with full network access? What controls prevent lateral misuse if compromised?

The Way Forward: A Controlled, Measurable Approach

We are clearly at a turning point. AI in penetration testing is not a question of if—it’s a question of how and when. But premature adoption without structured evaluation can weaken, rather than strengthen, security posture. Organizations should resist binary thinking (AI vs Human) and instead focus on comparative validation:

• Conduct PoCs on real enterprise applications
• Benchmark AI-driven vs human-led testing
• Evaluate across:

• Depth of findings
• False positive rates
• Coverage of business logic vulnerabilities
• Time-to-deliver and cost efficiency 

AI is accelerating. Agent creation is becoming effortless. Automation is redefining scale. But security has never been about speed alone—it’s about precision, context, and judgment. We need to engineer the right balance between human intelligence and machine capability. 

We certainly should use AI in application security - it brings scale, speed, and the ability to uncover patterns that would otherwise take significant manual effort but the need of human intelligence cannot be completely written off. AI can accelerate discovery. Humans ensure relevance, accuracy, and real-world impact. Together, they create a security model that is not only efficient, but also resilient and trustworthy. Organizations that recognize this balance early will not just keep up with the shift—they will define it.

Article by Hemil Shah and Rishita Sarabhai 

[Case Study] Threat Simulation of AI Agents in Microsoft Copilot Studio

Executive Summary

Blueinfy performed a focused, time-bound security review of Microsoft Copilot Studio and its implementation at ACME to assess the potential risks introduced by AI agents.

The objective of the engagement was to evaluate how AI agents both legitimate and malicious could be misused, intentionally or unintentionally, to 

• Access sensitive enterprise data
• Expose user specific information
• Perform unauthorized actions
• Enable data exfiltration

The assessment combined configuration review with hands on threat simulation, where custom agents were built to replicate realistic attack scenarios as well as instructions were passed to exploit legitimate agents. The results demonstrated that even with platform level controls enabled, significant risks can persist due to configuration gaps, excessive permissions, and agent behavior manipulation.

The environments given for testing had pre-configured policies and controls applied prior to the assessment. The scope included - 

• AI agent configuration within Microsoft Copilot Studio
• Data access patterns through agents
• Connector usage and restrictions
• Guardrails and safety configurations
• Threat simulation using custom-built agents

Assessment Methodology

Blueinfy adopted a structured methodology combining configuration validation and adversarial testing.

1. AI Configuration Review

A focused configuration review was conducted to evaluate AI-specific settings. 

Areas Reviewed:
•    AI agent configuration settings
•    Connector policies and restrictions
•    Data access configurations
•    Prompt safety and guardrails
•    Logging and monitoring capabilities

Objective:
•    Identify risky configurations
•    Recommend controls to reduce exposure
•    Highlight configurations requiring governance before enablement

2. Agent Threat Simulation

Instead of testing existing agents, Blueinfy created custom agents within the allowed policy boundaries to simulate real world attack scenarios. This approach ensured:

• No disruption to production agents
• Realistic exploitation within permitted configurations
• Validation of platform controls under adversarial conditions

Threat Simulation Approach

Agents were built using only approved connectors and policies within the environment. Two categories of agents were designed:

1. Misuse of Legitimate Agents

• Agents behaving as intended but manipulated via inputs
• Exploiting trust in user prompts

2. Malicious Agent Design

• Agents intentionally designed to bypass safeguards
• Leveraging allowed configurations to simulate abuse

Key Attack Scenarios Tested

Blueinfy executed multiple scenarios to evaluate risk exposure:

• Prompt Injection and Instruction Override - Manipulating agent behavior using crafted inputs to override system instructions and cause unintended data access
• Data Exfiltration via Allowed Channels - Extracting sensitive data through email connectors, API responses and structured outputs
• Cross-Agent Interaction Risks - Simulating agent-to-agent communication and demonstrating potential lateral movement
• Rouge Agents – Malicious agents built with system instructions to exfiltrate data, phish users for credentials and send application/user data to unintended servers
• MCP Exposure – If MCP server and tools are accessible without correct authentication and authorization mechanisms

Key Observations

The assessment revealed several important findings:

• Misconfigurations Create Hidden Risks – Users of the agents are completely unaware of the data risks since once published/shared, end users have limited visibility in the agent configuration. We were able to send emails of agent users including email attachments and employee feedback responses etc. to our third-party servers.
• Agents Can Be Manipulated Through Inputs – Based on the guardrails, prompt injection enabled behavior override and agents could be influenced to exfiltrate data without changing configuration. This created a scenario where legitimate agents to summarize users emails, posting summary to Teams channels etc. could be exploited to share that summary data to third-party servers via malicious instructions received in email/submitted forms.
• Unauthenticated MCP Exposure – MCP Tools connect the LLM's to organization data sources like databases, knowledge sources etc. With this engagement, we were able to use the MCP tools without authentication and gain full access to client sensitive data like contract financials.
• Platform Controls Are Not Sufficient Alone - While Microsoft Copilot Studio provides robust built-in controls, their effectiveness depends heavily on configuration and usage. Sadly, they do not work without being configured per your needs.  

Conclusion

Blueinfy’s assessment demonstrated that while platforms like Microsoft Copilot Studio do provide strong foundational controls, they must be complemented with:

• Proper configuration
• Risk-aware governance
• Adversarial testing
• Monitoring and logging

By moving from assumption based security to evidence driven validation, ACME established a stronger foundation for secure AI adoption. Blueinfy team worked with ACME to create a robust agent threat simulation and security review process to protect against such risks with scaling agents in parallel. Please read this blog for the three-tier risk methodology for an agent review process.

Article by Hemil Shah and Rishita Sarabhai 

The Rise of AI Agents and the urgent need for an Agent Security Review Process

Organizations today are rapidly embracing AI-powered agents. Platforms like Microsoft Copilot Studio and Google Gemini are enabling business users, not just developers, to create powerful agents that automate workflows, access enterprise data, and make decisions. This democratization is transformative. But it also introduces a new, largely ungoverned attack surface.

The Explosion of Agents

In many enterprises, the number of agents being deployed is growing exponentially from hundreds, sometimes thousands, within a short span of time. These agents:

• Integrate with internal systems
• Access sensitive enterprise data
• Perform automated actions on behalf of users

Unlike traditional applications, these agents are often created outside formal development pipelines by business users, analysts, or developers. And that’s where the problem begins.

The Security Gap: No "AgentSec"

Organizations have matured practices for AppSec or InfraSec or Cloud Security but Agent Security (AgentSec) is still in its infancy.
There is typically:

• No formal review process before agent deployment
• Limited visibility into what agents are doing
• No standardized threat modeling for agent behavior
• Weak validation of platform-level security controls

This creates a dangerous blind spot.

Built-in Controls Are Not Enough

Platforms do provide security mechanisms at:

• Data access controls
• Authentication and authorization layers
• Prompt filtering and safety guardrails
• Activity monitoring

However, these controls are:

• Complex to configure correctly
• Highly dependent on implementation choices
• Difficult to validate in real-world scenarios

Misconfigurations or misunderstandings can easily render these protections ineffective.

Visualizing the Risk: Agent Attack Flow

The Missing Piece: A Scalable Agent Review Process

At first glance, the solution seems straightforward: introduce agent design reviews, configuration assessments, and threat modeling for every agent. But in reality, this approach does not scale.

In large enterprises with hundreds or thousands of agents built on platforms, performing deep security reviews on every agent would:

• Overwhelm security teams
• Slow down innovation
• Create operational bottlenecks

Instead, organizations must adopt a risk-based Agent Security (AgentSec) model. The three-tier risk model classifies agents based on their potential impact and exposure. 
 

• High-risk agents are typically misconfigured or intentionally malicious, capable of unsafe actions such as exfiltrating data to external emails, interacting with unauthorized external URLs, or executing harmful embedded instructions. 
• Medium-risk agents involve broader data interaction—often consuming sensitive or user-provided inputs through connectors, APIs, MCP integrations, or multi-agent communication—making them more prone to misuse or unintended data exposure. 
• Low-risk agents operate within a constrained scope, relying on public or read-only data sources such as web search, uploaded files, SharePoint, or Dataverse, with minimal ability to cause harm.

Automation enables scale by classifying the agents into risk buckets and a focused review can then be performed only for high-risk and medium-risk agents to assess the business impact by building abuse/exploit scenarios. This approach ensures that security teams invest effort where it truly matters - prioritizing depth and accuracy over volume.

Why This Model Works

This approach delivers both speed and security: fast approvals for low-risk agents, strong scrutiny for higher-risk ones, reduced burden on security teams, and scalable governance across thousands of agents. Most importantly, it aligns security effort with actual risk - not perceived risk.

The organizations that succeed will not be those attempting to review every agent, but those that automate the baseline, enforce non-negotiable security gates, and escalate only what truly matters. Because in a world of thousands of agents, scalability itself becomes security.

Article by Hemil Shah and Rishita Sarabhai 

Agentic AI Security - Threats and Attacks (Paper Review)

Agentic AI systems transform LLMs into autonomous operators that plan, call tools, use memory, and act across web, code, APIs, and even physical environments, which radically enlarges the attack surface beyond simple chatbots. The paper frames security for these systems around concrete threat families: prompt injection and jailbreaks; autonomous cyber‑exploitation with tool abuse; multi‑agent and protocol‑level attacks (including MCP and agent‑to‑agent ecosystems); and environment/interface issues such as unsafe action spaces and brittle web interaction. These systems must therefore be treated as distributed, partially trusted components that can both be attacked and weaponized as attackers themselves.

Prompt‑centric threats are broken down into direct and indirect prompt injection, intentional and unintentional attacks, multi‑modal and hybrid payloads (text, images, audio, code), propagation behaviors, and multilingual/obfuscated or split payloads that evade naive filters. Attackers can poison external content sources (web pages, PDFs, accessibility trees, APIs), craft adversarial code/SQL prompts, or hide instructions in non‑text modalities to hijack the agent’s plan and tool calls. The work also highlights that many proposed PI defenses are brittle, with adaptive IPI attacks able to bypass perplexity‑based and pattern‑based detectors in practice, which reinforces PI as a primary attack vector against agentic workflows.

On the offensive operations side, the paper shows that agents with code execution and network access can autonomously perform vulnerability discovery and exploitation, often outperforming traditional tools like OWASP ZAP or Metasploit on known‑vulnerable targets when given CVE descriptions and appropriate tools. Demonstrated capabilities include chaining XSS, CSRF, SSTI, and SQLi, navigating web apps in realistic sandboxes, and leveraging tools to iteratively refine exploits without human guidance. In multi‑agent and protocol‑driven settings (e.g., MCP or cross‑org agent meshes), they describe additional vectors such as fake or compromised agent registration, denial of service via recursive delegation, transitive prompt‑injection across agents, memory poisoning, and identity or role abuse that propagates through the agent network.

Reference: 

AGENTIC AI SECURITY:THREATS, DEFENSES, EVALUATION, AND OPEN CHALLENGES - https://arxiv.org/pdf/2510.23883 

 

Why Agentic Pentesting Can’t Fix the False Positive Problem

Agentic pentesting promises smarter orchestration of tools, but it does not magically eliminate false positives. At its core, an agent still leans on the same scanners, payload generators, and detection heuristics that produced noisy results in the first place. If the underlying tools misclassify behavior or lack application context, the agent simply becomes a faster, more automated way to generate and route those misclassifications. In other words, you risk “scaling the noise” as much as scaling the signal.

Another limitation is that most agentic systems still struggle with business context and intent, which is where many false positives are born. A finding that looks critical in HTTP traces might be benign in the real-world workflow because of compensating controls, domain‑specific logic, or risk acceptance decisions that only humans understand. Agents can replay exploits and correlate signals, but they cannot reliably answer questions like “Is this test user data or real PII?” or “Would exploiting this actually harm the business?” Without that judgment, they often cannot confidently close the loop on whether something is truly a vulnerability or just an academic issue.

Finally, agentic pentesting introduces its own new sources of error that can masquerade as false positives. Misconfigured prompts, overly broad goals, or aggressive automation can lead agents to test unsupported flows, mis-handle authentication, or misinterpret application responses. These mistakes can create “findings” that look real on paper but collapse under minimal human scrutiny. So while agentic approaches can help prioritize, group, and sometimes auto‑retest issues, they do not remove the need for human validation; they merely change where you spend your validation effort—from sifting through raw scanner output to scrutinizing AI‑curated results.

SSRF in Azure MCP Server Tools

In Microsoft's March 2026 Patch Tuesday release on March 10, an urgent high-severity vulnerability, CVE-2026-26118, emerged in Azure Model Context Protocol (MCP) Server Tools. This server-side request forgery (SSRF) flaw, scored at CVSS 8.8, allows low-privileged attackers to manipulate user-supplied inputs and force the server into making unauthorized outbound requests to attacker-controlled endpoints. MCP, designed to standardize AI model integrations with external data sources, unexpectedly became a vector for privilege escalation in AI-driven Azure environments, highlighting the growing risks in agentic AI architectures.

At its core, exploitation involves crafting malicious payloads that trick the MCP server—running versions prior to 2.0.0-beta.17—into leaking its managed identity token. Attackers can then impersonate the server's identity to access sensitive Azure resources like storage accounts, virtual machines, or databases, all without needing admin rights or user interaction. Public proof-of-concept exploits, such as those on GitHub, amplify the threat, enabling rapid weaponization in targeted attacks against organizations leveraging MCP for AI workflows. This vulnerability underscores a classic SSRF pattern (CWE-918) but tailored to cloud-native AI tools, where broad service principals often grant excessive permissions.

Organizations should prioritize patching via Microsoft's Security Update Guide, audit MCP deployments for over-privileged identities, and implement outbound request filtering to contain risks. As AI security evolves, this incident signals the need for runtime protections in MCP-based systems, including token rotation and anomaly detection for AI agent traffic. Application security teams, especially those testing AI integrations, can use tools like Burp Suite to validate fixes against SSRF payloads. Staying vigilant ensures AI innovation doesn't outpace defense in the cloud.

Reference - https://www.tenable.com/cve/CVE-2026-26118