Microsoft has announced the public preview of Azure Monitor Pipeline Transformations, a powerful new capability that allows organizations to transform telemetry data before ingestion into Azure Monitor. This feature is designed to reduce ingestion costs, improve data quality, and simplify analytics by shaping and optimizing data at the pipeline level.
What This Means
The Azure Monitor pipeline now enables:
- At-scale telemetry collection (100K+ EPS)
- Edge and multi-cloud data collection
- Local caching during connectivity loss
- Smart routing before sending data to Azure Monitor
With pipeline transformations, organizations can filter, aggregate, and standardize logs before they reach Log Analytics helping control costs and improve query performance.
Key Benefits
Lower Ingestion Costs
Filter, aggregate, and drop unnecessary data before ingestion to reduce volume and storage expenses.
Improved Analytics
Standardized schemas and cleaner datasets result in faster queries and better dashboards.
Future-Ready Governance
Built-in schema validation prevents deployment issues caused by incompatible transformations.
Key Features in Public Preview
- Schema Change Detection (Syslog & CEF)
Built-in validation detects schema-breaking transformations and provides corrective guidance.
- Pre-built KQL Templates
Ready-to-use transformation templates reduce errors and speed up implementation.
- Automatic Schema Standardization
Syslog and CEF data automatically aligned to standard tables.
- Advanced Filtering
Drop events based on attributes like Facility, SeverityLevel, DeviceVendor, or DestinationPort.
- Log Aggregation
Summarize high-volume logs into time-based intervals for better performance and insights.
- Field Reduction
Remove unnecessary columns to streamline storage.
Supported KQL capabilities include filtering, aggregation, schematization, variable creation, string manipulation, and data type conversions.
Why It Matters
High ingestion costs and noisy telemetry are common challenges in modern monitoring environments. Azure Monitor pipeline transformations address this by ensuring data is clean, structured, and optimized before it reaches your Log Analytics Workspace.
This update strengthens Azure Monitor’s role in cost-efficient, scalable, and governance-aligned observability strategies.