What are Microsoft Sentinel Data Connectors?

In today’s article, I’d like to introduce Microsoft Sentinel Data Connectors. They are a critical component of the broader Sentinel ecosystem - effectively the "lifeblood" of the entire analytics engine. Why? Because without a steady, reliable stream of data, even the most sophisticated detection rules and AI-driven insights remain idle.

The Challenge of Data Sprawl

Before we dive into the architecture, let's look at the core challenge facing modern Security Administrators: data sprawl. For decades, security logs have been the foundational source of truth in cybersecurity. However, while the necessity of logs is a constant, their origins have shifted dramatically.

Today, telemetry is scattered across a fragmented landscape: multi-cloud environments, legacy on-prem data centers, edge IoT devices, and a sprawling web of IaaS, PaaS, and SaaS ecosystems. This presents several challenges for security teams:

• Ingestion: How do we reliably collect data from such disparate sources into a unified plane like Microsoft Sentinel?
• Normalization: How do we process that data so it actually makes sense to the system and is ready for analysis?

That leads us to... (drumroll please...) Microsoft Sentinel Data Connectors! Think of these as the essential pipes or conduits that connect a variety of log sources on one side to their ultimate home within a Log Analytics Workspace. Their sole purpose is to connect Point A to Point B and make sense of the log data traversing that path so that Sentinel can put it to work.

Data Connector Anatomy

So, now that we know that a Data Connector is basically just a data ingestion pipeline, what exactly is it made of? That depends on how the data is being transported. Typically, it will be via one of five techniques:

• API-Based Connectors (Azure Functions/Logic Apps): This technique uses serverless code to poll external APIs on a recurring schedule. Upon receiving the log data, the function pushes it to the Azure Log Analytics Ingestion API.
  • Examples: Zoom, Cisco Duo, Slack.
• Agent-Based Connectors: This involves deploying a utility (typically the Azure Monitor Agent) on VMs or servers. These agents collect logs locally and stream the data via HTTPS straight to the Log Analytics workspace.
  • Examples: Windows Security Events, Linux system logs.
• Service-to-Service Connectors: These are native Azure resources handing off logs. Because data flows directly within the Azure environment, no intermediate compute is required.
  • Examples: Microsoft 365, Microsoft Defender for Cloud, Azure Activity Logs.
• Syslog/CEF Forwarders: This involves a Linux VM acting as a relay. The system receives Syslog or Common Event Format (CEF) messages from devices that use traditional logging (like network infrastructure) and uploads them via an agent.
  • Examples: Firewalls, Routers, legacy network hardware.
• Custom Connectors: These use direct REST API calls to the Data Ingestion API. They provide maximum flexibility and full control over data transformation and ingestion.

The "Big Four" of Architecture

Phew! That’s a lot of possibilities. With me thus far? The core thing to keep in mind is that modern Sentinel ingestion relies on four main pillars:

1. Data Collection Endpoint (DCE): The entry point (URL) where the data first arrives.
2. Data Collection Rule (DCR): The critical logic that determines how to filter, transform, and route the data.
3. Log Analytics Workspace: The storage "bucket" where data lives in indexed tables.
4. KQL Tables: The specific schema or structure that is eventually queried.

In essence, the "connector" is just a configuration wrapper around these underlying mechanisms.

Conclusion

There is a lot more I could write about Microsoft Sentinel logging, but I think this is a good stopping point for today. In future articles, we'll explore related concepts, including making use of the data and applying effective principles of threat hunting.

In the meantime, if you're curious and want to learn more, check out these additional resources:

Microsoft Sentinel data connectors

Learn about supported data connectors, like Microsoft 365 and Office 365, and Microsoft Entra ID to Microsoft Sentinel.

Azure DocsWenJason

Resources for creating Microsoft Sentinel custom connectors

Learn about available resources for creating custom connectors for Microsoft Sentinel. Methods include the Log Analytics API, Logstash, Logic Apps, PowerShell, and Azure Functions.

Azure DocsWenJason

Best practices for data collection in Microsoft Sentinel

Learn about best practices to employ when connecting data sources to Microsoft Sentinel.

Azure DocsWenJason