New

The executive guide to generative AI

Read more
About usPartnersSupport|Login
IMPORTANT: No additional bug fixes or documentation updates will be released for this version. For the latest information, see the current release documentation.
« Azure Network Watcher Deletion Azure Privilege Identity Management Role Modified »
Elastic Docs ›Elastic Security Solution [8.16] ›Detections and alerts ›Prebuilt rule reference

Azure OpenAI Insecure Output Handling

edit

Azure OpenAI Insecure Output Handling

edit

Detects when Azure OpenAI requests result in zero response length, potentially indicating issues in output handling that might lead to security exploits such as data leaks or code execution. This can occur in cases where the API fails to handle outputs correctly under certain input conditions.

Rule type: esql

Rule indices: None

Severity: low

Risk score: 21

Runs every: 10m

Searches indices from: now-60m (Date Math format, see also Additional look-back time)

Maximum alerts per execution: 100

References:

Tags:

  • Domain: LLM
  • Data Source: Azure OpenAI
  • Data Source: Azure Event Hubs
  • Use Case: Insecure Output Handling
  • Resources: Investigation Guide

Version: 2

Rule authors:

  • Elastic

Rule license: Elastic License v2

Investigation guide

edit

Triage and analysis

Disclaimer: This investigation guide was created using generative AI technology and has been reviewed to improve its accuracy and relevance. While every effort has been made to ensure its quality, we recommend validating the content and adapting it to suit your specific environment and operational needs.

Investigating Azure OpenAI Insecure Output Handling

Azure OpenAI integrates AI capabilities into applications, enabling natural language processing tasks. However, improper output handling can lead to vulnerabilities, such as data leaks or unauthorized code execution. Adversaries might exploit these by crafting inputs that cause the API to mishandle responses. The detection rule identifies anomalies by flagging instances where API responses are unexpectedly empty, suggesting potential misuse or misconfiguration, especially when such events occur frequently.

Possible investigation steps

  • Review the logs for the specific Azure resource name flagged in the alert to understand the context and frequency of zero-length responses.
  • Examine the request lengths associated with the zero-length responses to identify any patterns or anomalies in the input data that might be causing the issue.
  • Check the cloud account ID associated with the alert to determine if there are any known issues or recent changes in configuration that could affect output handling.
  • Investigate the operation name "ChatCompletions_Create" to ensure that the API is being used as intended and that there are no unauthorized or unexpected uses.
  • Assess the overall environment for any recent updates or changes in the Azure OpenAI configuration that might have impacted output handling.

False positive analysis

  • Frequent legitimate requests with zero response length can occur during testing or development phases. To manage this, exclude known test environments or accounts from the detection rule by adding exceptions for specific cloud.account.id or azure.resource.name values.
  • Some applications may intentionally send requests that do not require a response, resulting in zero response length. Identify these applications and adjust the rule to exclude their specific azure.resource.name.
  • Network issues or temporary service disruptions can lead to zero-length responses. Monitor for patterns of such occurrences and consider excluding specific time frames or network segments if they are known to cause false positives.
  • Automated scripts or bots that interact with the API might generate zero-length responses as part of their normal operation. Identify these scripts and exclude their associated identifiers from the rule to prevent false alerts.

Response and remediation

  • Immediately isolate the affected Azure OpenAI resource to prevent further exploitation. This can be done by temporarily disabling the API or restricting access to it.
  • Review and validate the input handling mechanisms of the affected API to ensure they are robust against malformed or malicious inputs that could lead to insecure output handling.
  • Conduct a thorough audit of recent API requests and responses to identify any unauthorized access or data leaks. Pay special attention to requests with zero response length.
  • Implement additional logging and monitoring for the affected API to capture detailed information about requests and responses, which can help in identifying patterns or repeated attempts of exploitation.
  • Notify the security team and relevant stakeholders about the incident, providing them with detailed findings and any potential impact on data security.
  • If unauthorized access or data leakage is confirmed, follow the organization’s incident response plan to notify affected parties and comply with any regulatory requirements.
  • Enhance detection capabilities by integrating anomaly detection tools that can identify unusual patterns in API usage, such as frequent zero-length responses, to prevent similar threats in the future.

Setup

edit

Setup

For more information on streaming events, see the Azure OpenAI documentation:

https://learn.microsoft.com/en-us/azure/azure-monitor/essentials/stream-monitoring-data-event-hubs

Rule query

edit
from logs-azure_openai.logs-*
| where azure.open_ai.properties.response_length == 0 and azure.open_ai.result_signature == "200" and azure.open_ai.operation_name == "ChatCompletions_Create"
| keep azure.open_ai.properties.request_length, azure.open_ai.result_signature, cloud.account.id, azure.resource.name
| stats count = count() by azure.resource.name
| where count >= 10
| sort count desc
« Azure Network Watcher Deletion Azure Privilege Identity Management Role Modified »
Was this helpful?
Feedback