Why Security Scanning and Testing of AI Skills Matters Before You Hand Them to Agents

AI skills should be treated as software artifacts, not just prompt text. When a skill file is passed to an AI agent, it can shape behavior, permissions, data flow, and external access in ways that create real security risk. That is why scanning and testing skills for vulnerabilities, unsafe dependencies, secret exposure, and hidden instructions is essential before deployment.

The main risk is trust without verification. A skill may appear harmless, but it can still contain overly broad permissions, insecure scripts, prompt injection paths, or dependencies with known flaws. For example, a file-processing skill might request write access where read-only access is sufficient, or a workflow skill might forward user data to external services without a clear business need. If an agent uses such a skill blindly, the result can be data leakage, policy bypass, unauthorized actions, or behavior that is difficult to detect until damage has already occurred.

A professional review process turns this into a controlled security practice. Before a skill is handed to an AI agent, teams should examine the source, scan for secrets and vulnerable packages, test the code in an isolated environment, and validate how the agent behaves under normal and adversarial prompts. This includes checking whether the skill respects least privilege, handles sensitive data appropriately, and resists instruction injection. A skill should not only be functional; it should also be safe, auditable, and aligned with operational and compliance requirements.

Sample rules for skill review:

• Reject any skill that requests permissions beyond its stated purpose.
• Block hardcoded secrets, API keys, tokens, or credentials in skill files or scripts.
• Require all external domains, APIs, and endpoints to be explicitly approved.
• Ensure dependencies are pinned and scanned for known vulnerabilities.
• Disallow shell execution unless the command set is tightly validated and necessary.
• Prevent raw sensitive data from being logged, stored, or exported.
• Treat prompt overrides such as “ignore previous instructions” as untrusted input.
• Require isolated testing before a skill is allowed to run in production workflows.
• Review any file read/write access for least-privilege compliance.
• Reassess the skill whenever code, dependencies, or permissions change.

In practice, the best safeguard is a combination of static review, behavioral testing, and policy enforcement. That approach reduces supply-chain risk, prevents unsafe automation, and makes it much easier to trust the skills you give to AI agents. 

[Case Study] AI Agent Trap – Simulating Hidden Threats in Third-Party Content

Background

ACME, a global retail organization, relied on AI agents to collect, summarize, translate, and categorize discount coupons from thousands of third-party websites. Every day, the AI processed HTML pages, PDFs, promotional images, newsletters, and marketing content before storing the normalized information in the organization's internal database.

This automation dramatically improved efficiency but it also introduced a new class of security risk. Unlike traditional attacks that directly target applications, attackers could instead lay traps for the AI agent - by embedding malicious instructions inside the very content it was designed to consume. These instructions remained invisible to users but were interpreted by the AI during processing, potentially altering its behavior, influencing decisions, or causing sensitive information to be exposed.

Understanding the AI Supply Chain

Modern AI agents rarely operate in isolation. They continuously interact with models, prompts, tools, APIs, knowledge bases, documents, websites, images, PDFs, and other third-party content to complete business tasks. Every external dependency becomes part of the AI supply chain and represents a potential trust boundary.

This is where the concept of an AI Bill of Materials (AIBOM) becomes valuable. An AIBOM provides visibility into the components, services, and data sources that an AI application depends upon, helping organizations understand what their AI agents consume, process, and trust. While this inventory is essential for governance and risk management, it does not determine whether those dependencies can be exploited.

AIBOM tells you what your AI consumes. AI Agent Trap tells you whether those inputs can compromise the AI.

The AI Agent Trap

Rather than attacking the application itself, the attacker prepares content that appears completely legitimate. The trap may be hidden inside - 

• Promotional web pages
• HTML comments
• Product descriptions
• PDF documents
• Marketing brochures
• Coupon images (via OCR)
• Document metadata
• Invisible or white-on-white text
• Multilingual content

When the AI agent ingests this content, the embedded instructions attempt to manipulate the agent into ignoring its original objectives and performing unintended actions. The trap is activated only when the AI processes the content. 

Blueinfy's Threat Simulation

To evaluate ACME's exposure, Blueinfy conducted an AI Agent Trap Simulation. Instead of reviewing prompts in isolation, Blueinfy recreated an attacker's infrastructure by hosting controlled coupon resources on an external website. These resources contained carefully crafted AI traps embedded across multiple content formats while appearing completely legitimate to human users. 
The AI agent consumed these resources through its normal ingestion pipeline exactly as it would in production. Blueinfy observed how the agent responded, identified where traps were successfully inserted into the processing workflow, measured how they propagated through downstream systems, and evaluated whether existing safeguards prevented exploitation.
The assessment focused on identifying:

• AI trap insertion points
• Prompt injection opportunities
• Trust boundary failures
• Context manipulation
• Tool misuse opportunities
• Memory contamination
• Data leakage scenarios
• Persistence of malicious content within enterprise knowledge

Business Impact

The simulation demonstrated that a successful AI Agent Trap could influence business processes long before anyone noticed. Potential impacts included:

• Manipulated summaries stored in enterprise databases
• Incorrect coupon categorization
• Corrupted downstream AI responses
• Leakage of sensitive internal information
• Execution of unintended AI workflows
• Contamination of organizational knowledge repositories

Unlike traditional attacks, these traps were embedded within otherwise legitimate business content, making them difficult to detect using conventional security controls.

Outcome

Blueinfy's AI Agent Trap Simulation enabled ACME to identify hidden trust boundary weaknesses before they could be exploited in production. Based on the findings, the organization strengthened content sanitization, isolated untrusted inputs, validated AI inputs and outputs before persistence, and implemented additional guardrails to ensure external content could not influence critical AI decision-making. Blueinfy connected three concepts into a coherent security lifecycle:

• AIBOM – Know your AI dependencies.
• AI Agent Trap – Test whether those dependencies can be exploited.
• AI Guardrails – Implement controls to prevent successful exploitation. 

The engagement demonstrated that as AI agents increasingly interact with external information, organizations must secure not only the agent itself, but also every source of content the agent trusts. In the age of autonomous AI, the attack begins long before the agent receives its next prompt—it begins where the trap is laid. 

Blueinfy recommended introducing a content normalization layer that extracts only predefined business attributes required by the application while treating all remaining content as untrusted. Combined with prompt isolation, robust output validation, AI guardrails, and the use of the latest AI models with improved resilience against indirect prompt injection techniques, this significantly reduces the likelihood that embedded instructions influence the AI agent. As these attacks continue to evolve, organizations should periodically validate their AI workflows through adversarial simulations to ensure the implemented controls remain effective.

Article by Hemil Shah & Rishita Sarabhai 

Building Secure AI Systems Starts Before the First Prompt: Why AISVS Matters

Every successful technology implementation begins with a sound architecture and design. For years, application security teams have relied on the OWASP Application Security Verification Standard (ASVS) as a structured set of security requirements that architects, developers, and security reviewers use during the design and implementation phases of traditional applications. Rather than waiting until code review or penetration testing uncovers vulnerabilities, organizations use ASVS to validate that security requirements have been considered while the application is being built.

OWASP Artificial Intelligence Security Verification Standard (AISVS) extends the same philosophy that made ASVS successful - structured security verification during design and implementation—but applies it specifically to AI-powered systems. Instead of focusing only on authentication, session management, cryptography, and input validation, AISVS introduces security requirements around model governance, prompt handling, context management, agent permissions, tool integrations, memory protection, AI supply chain security, data privacy, monitoring, and human oversight. It consists of 12 major categories:

 
The value of AISVS is not merely the checklist itself - it is the conversation it creates between architects, developers, business owners, AI engineers, and security teams. When implementation teams receive these questions at the beginning of a project, they are forced to think through decisions that might otherwise be overlooked, as an example

• How is sensitive business data protected before being sent to an LLM?
• Can an AI agent invoke privileged tools without sufficient authorization?
• How are prompts, context, and memory isolated between users?
• What controls prevent prompt injection or indirect prompt manipulation?
• How are third-party models, MCP servers/Gateways, plugins, or connectors trusted and governed?
• What monitoring exists to detect unsafe AI behaviour in production?

Many of these questions cannot be answered after deployment without expensive architectural changes. However, when raised during design reviews, the required controls can be incorporated naturally into the solution architecture.

In our engagements, we have observed that circulating AISVS questionnaires during the implementation or pre-implementation phase significantly improves the quality of AI security discussions. Instead of discovering architectural weaknesses during security reviews, development teams proactively identify security gaps while components are still being designed. The outcome is fewer redesign cycles, reduced remediation effort, and a more consistent security baseline across AI initiatives.
The process is straightforward:

This approach transforms security from a reactive validation exercise into a design assurance activity. The below categories are covered in the assessment:

Each category with multiple sub-categories and respective set of questions like below - 

As AI systems become increasingly autonomous, interconnected, and capable of making business decisions, architectural choices have a far greater impact on organizational risk than individual coding defects. Secure AI implementations therefore require more than traditional application security reviews - they require structured architectural verification against AI-specific security requirements.

AISVS provides that foundation. Much like ASVS became the benchmark for building secure applications, AISVS is emerging as the framework that enables organizations to design, implement, and deploy AI systems with security embedded from the very beginning.

Business wants AI delivered yesterday, but security embedded into the architecture from Day 0 ultimately saves time accelerates delivery by eliminating costly redesigns and late-stage remediation. The most effective AI security programs will not be those that perform the most penetration tests after deployment. They will be the ones that ask the right questions before a single AI component reaches production.

Article by Hemil Shah & Rishita Sarabhai

Importance of MCP Gateway in Modern Architecture

We Never Needed an API Gateway. Why Do We Suddenly Need an MCP Gateway?

As organizations adopt AI agents and convert APIs into MCP tools, a common debate is emerging between development teams and security leaders. The developer's question is simple - "Our APIs have been running securely for years without an API Gateway. Why is Security now insisting that all MCP tools must go through an MCP Gateway?" At first glance, this appears to be a reasonable challenge. If direct API access was acceptable yesterday, why should exposing the same functionality through MCP require an additional control layer today? The answer lies in understanding what has actually changed. and surprisingly, it is not the API.

The API Is Not the Problem

Many enterprises successfully operate thousands of APIs without a dedicated API Gateway where typical architecture looks like - 

  

These environments often rely on:

• Application authentication
• Network segmentation
• Service-level authorization
• Secure coding practices
• Monitoring and logging

For years, these controls have been sufficient because the consumer was predictable. The API was being accessed by applications designed, tested, and governed by the organization. Security teams understood the workflows, business logic, and expected behavior. The risk model was stable.

What Changed? The Consumer Changed.

With MCP, organizations are no longer exposing capabilities solely to applications. They are exposing them to AI agents.

Unlike traditional applications, AI agents:

• Make decisions dynamically
• Select tools at runtime
• Interpret natural language instructions
• Chain multiple actions together
• Process untrusted inputs
• Operate with varying levels of autonomy

The API remains the same but the consumer does not and that changes everything.

The Question Security Teams Are Really Asking

The debate should not be "Is the API secure?" but the more important question is "Are we comfortable allowing AI systems to directly invoke enterprise capabilities without centralized oversight?" For most organizations, the answer is no and that is where the MCP Gateway becomes important.

What Happens Without an MCP Gateway?

Imagine an organization creates hundreds of MCP tools directly connected to backend APIs.

Agent → Tool A → API
Agent → Tool B → API
Agent → Tool C → API
Agent → Tool D → API

Now Security must answer:

• Which agents can access which tools?
• Which tools expose regulated data?
• How do we implement DLP?
• How do we monitor tool usage?
• How do we detect prompt injection attacks?
• How do we disable risky tools quickly?
• How do we produce audit reports?

Without a centralized control point, every team must solve these problems independently. The result is inconsistent security and fragmented governance.

Risks That Did Not Exist Before

Prompt Injection

Traditional applications are not influenced by prompts but AI agents are. An attacker can attempt to manipulate an agent into performing actions it was never intended to perform. Without a gateway, every MCP tool becomes responsible for defending itself.

Data Leakage

AI systems routinely process sensitive business information. Without centralized inspection, organizations will not  have any visibility into PII exposure, financial data leakage, Intellectual property disclosure or Excessive data retrieval. 

Tool Sprawl

As MCP adoption grows, organizations often move from a handful of tools to hundreds. Without centralized governance:

Tool A → Custom Controls
Tool B → Different Controls
Tool C → No Controls
Tool D → Minimal Logging

Security posture becomes inconsistent and difficult to audit.

Agent Abuse

Applications generally follow predictable workflows whereas agents do not. A poorly configured agent can trigger excessive API calls, create runaway automation loops, generate unexpected operational costs or access data beyond intended business needs. Traditional API controls rarely provide visibility into these behaviors.

Why the MCP Gateway Exists

The purpose of the MCP Gateway is not to replace API security. The purpose is to provide AI-specific governance as demonstrated in diagram below - 

The gateway becomes the centralized enforcement point for Agent authorization, Tool authorization, Prompt inspection, Data loss prevention, Audit logging, Rate limiting, Governance policies and/or Compliance monitoring. These controls are difficult to implement consistently inside every individual MCP tool.

In a nutshell 

The APIs may not have changed but the consumers have and that is exactly why the architecture must evolve. The risk model has changed. Our APIs were designed for applications operating within controlled workflows. MCP tools are designed for AI agents that make decisions dynamically based on user input. The API itself is not less secure than before. However, AI-driven access introduces new governance, monitoring, and security requirements. The MCP Gateway provides a centralized control point for managing those risks consistently across the enterprise.  Organizations did not suddenly discover that their APIs were insecure.  What changed is that enterprise capabilities are now being exposed to a new class of consumer “AI agents”. That shift introduces risks that traditional application architectures never had to address. An MCP Gateway is not a replacement for API security. It is the control plane that allows organizations to safely scale AI adoption while maintaining visibility, governance, and trust. 

Article by Hemil Shah & Rishita Sarabhai

AI Agent Traps - Attack vectors and Vulnerabilities

Artificial intelligence agents are becoming increasingly autonomous, but Google DeepMind's new paper "AI Agent Traps" reveals a critical vulnerability: the open web itself can be weaponized against them. The researchers introduce the first systematic framework identifying six distinct categories of adversarial attacks specifically designed to exploit autonomous agents navigating digital environments. Unlike traditional LLM vulnerabilities, these traps exploit the gap between what humans see and what agents parse, allowing attackers to embed malicious instructions in HTML comments, hidden CSS, image metadata, or accessibility tags that are invisible to users but directly processed by agents.

The DeepMind taxonomy reveals particularly alarming attack success rates: hidden prompt injections in HTML already commandeer agents in up to 86% of scenarios, while latent memory poisoning achieves 80%+ attack success with less than 0.1% data contamination. The six trap categories include Content Injection Traps (perception attacks), Semantic Manipulation Traps (corrupting reasoning), Cognitive State Traps (poisoning memory/RAG databases), Behavioural Control Traps (hijacking actions), Systemic Traps (targeting multi-agent dynamics), and Human-in-the-Loop Traps (using agents to attack humans). These aren't theoretical—every trap type has documented proof-of-concept attacks, and the attack surface is cooupled, meaning traps can be chained or distributed across multi-agent systems.

For security professionals building agentic AI systems, DeepMind's research demands a fundamental shift in defensive strategy. Traditional protections like input validation or human monitoring are inadequate when scaled, as tainting one data source can propagate harmful instructions downstream. The researchers propose mitigations including training data augmentation, runtime defenses, content governance frameworks, and standardized evaluation benchmarks to detect these threats. As DeepMind notes, securing agents against environmental manipulation is "a prerequisite for realizing the benefits of a trustworthy agentic ecosystem"—making this research essential for anyone developing AI agents for security scanning, autonomous workflows, or multi-agent orchestration.

Reference Paper - Read here  [ https://papers.ssrn.com/sol3/papers.cfm?abstract_id=6372438 ]

AI for Security and Securing AI: The Two Fronts Every CISO Must Lead

Artificial Intelligence is rapidly changing enterprise security - not just in how organizations defend themselves, but also in what they must defend. For CISOs, this has created two parallel priorities that can no longer operate independently:

1. Using AI to strengthen security programs
2. Securing the organization’s own AI ecosystem

Organizations that focus on only one side are discovering major gaps either inefficient security operations or uncontrolled AI risk exposure.

The future of security is no longer just “security for applications.” It is now AI-enhanced security operations combined with AI governance and AI defense.

AI for Security: Transforming Application Security Programs

Traditional application security programs have matured over the years with practices such as SAST in CI/CD pipeline, DAST, Manual Penetration Testing, Manual Secure Code Review and VDP. 

These do remain critical. However, modern development velocity and AI-assisted coding have fundamentally changed the threat landscape. Applications are now larger, faster-changing, AI-generated in parts, micro-service driven and increasingly dependent on third-party components.  

This means traditional AppSec processes alone are no longer sufficient. The next generation of AppSec requires two major AI-driven additions:

 The AppSec lifecycle is evolving from: 

“Find vulnerabilities” to “Find, validate, and understand business impact.”

2. Securing AI: The New Enterprise Security Program

While organizations are using AI to improve security, they are simultaneously deploying AI across business functions internal copilots, customer support bots, AI-enabled workflows, AI-assisted development, document intelligence systems, AI agents and RAG-based enterprise platforms and so on. This introduces an entirely new attack surface and many organizations are discovering a dangerous misconception. Out-of-the-box AI security controls are not enough.

As highlighted in the recent case study “Building an AI Security Program for a Global Investment Firm”, securing AI requires a dedicated organizational process, not simply enabling default protections. AI systems introduce different risks and require different level of customization:

The Emerging CISO Reality

The modern CISO now operates two security transformation programs simultaneously:

Organizations that mature in only one area will remain exposed in the other.

 

Blueinfy’s Approach

At Blueinfy, we are working closely with CISOs to help establish both dimensions of this transformation:

The organizations that succeed over the next few years will not simply “adopt AI.”
They will:

• Use AI to improve security effectiveness
• Secure AI systems with the same rigor as critical enterprise applications

That combination will define the next generation of cybersecurity maturity.

Article by Hemil Shah

[Case Study] Building an AI Security Program for a Global Investment Firm

A multinational investment firm started adopting AI across the organization - through enterprise platforms like Google Gemini enterprise and independently within business units for vibe coding, data analytics, and customer facing use cases. This did help teams move faster but it also created a need to bring consistency, visibility, and security around how AI was being used.

Blueinfy was engaged to support the organization in setting up a structured AI Security Program that could scale with this adoption without slowing down innovation. The approach focused on creating a correct balance between governance and flexibility ensuring that AI could grow across the organization, but in a more controlled and visible manner.

Challenges

The key challenge was the way AI adoption had expanded in the organization - fast, scattered, and largely independent across teams. While enterprise tools provided scale, business units were along-side experimenting with different AI solutions, making it difficult to maintain a consistent security approach. 

There was limited visibility into how AI was being used, what kind of data was being shared, and which external tools were involved. At the same time, emerging risks such as overly permissive AI agents, unrestricted integrations, and unintended data exposure through prompts and workflows were becoming harder to track. 

From an execution standpoint, aligning multiple teams, ensuring the right access and prerequisites, and bringing everyone to a common approach required continuous coordination and validation. 

The organization’s AI adoption approach created distinct risk areas:

• AI usage was growing without a single view of where and how it was being used
• Different business units were following their own approaches, leading to inconsistency
• Sensitive user and enterprise data was being shared with AI systems without clear guardrails
• There was limited validation of AI use cases from a security standpoint
• Third-party AI tools as well as code generated by AI were not reviewed in detail

Overall, the challenge was less about lack of intent, and more about the absence of a structured approach.

Solution / Approach

Blueinfy aligned the overall approach around a single ownership model, supported by targeted and continuous activities.

 At a high level, a dedicated AI Security Program Lead was introduced to take comprehensive responsibility for AI security across the organization. This role acted as the central coordination point ensuring visibility, consistency, and alignment across security, IT, and business units.

For Business Units, the focus was on enablement. Teams were supported with clear guidance, practical do's and don'ts, and secure usage patterns. This allowed them to continue building and experimenting with AI without unnecessary resistance.

As part of this enablement, Blueinfy also helped define and roll out standardized documentation and guidelines, including:

• AI implementation guidelines covering architecture, integrations, and connectivity
• Access control and permission models for AI tools, agents, and APIs
• Guardrails for safe data usage, prompt handling, and output validation
• Responsible use of AI guidelines for end users (what can and cannot be shared with AI systems)
• Lightweight review and approval processes for new AI use cases

These documents provided a consistent baseline for teams, reducing ambiguity and improving adoption of secure practices.

For Enterprise AI platforms, a structured validation approach was followed. A threat simulation exercise was conducted to identify potential risks such as data exposure, misuse scenarios, and integration weaknesses.
Based on these insights, a continuous validation model was introduced:

• Agent security reviews to assess workflows, permissions, and integrations
• AI red teaming for new models and high-risk use cases
• Penetration testing for AI-driven customer-facing implementations

This ensured that AI security was not a one-time activity, but an ongoing process embedded into how new AI capabilities were introduced.

Outcome

With this model in place, the organization was able to bring more structure to its AI adoption without slowing down innovation.

• A clear ownership model improved coordination and decision-making
• Better visibility into AI use cases reduced unmanaged or "shadow" AI risks
• Business units were able to innovate with clearer guidance and fewer blockers
• Standardized guidelines helped teams follow consistent and secure practices
• Risks related to data exposure, integrations, and agent behavior were identified earlier
• Continuous reviews ensured that new AI implementations were assessed as they were introduced

Overall, the shift from one-time assessments to a continuous validation approach, supported by clear documentation and ownership, helped the organization stay aligned with the pace at which AI was evolving internally.

Conclusion

AI adoption in large organizations will naturally be fast and distributed. The real challenge is not controlling it completely, but making sure it grows in a structured and secure way.

This engagement shows that with clear ownership, practical guidance, and ongoing validation, organizations can build a sustainable AI security program that supports both innovation and risk management.

Article by Hemil Shah and Rishita Sarabhai