Internals

Deep Dive into Spark SQL Metrics (Part 5): Gluten Metrics Internals

Part 5 of the SQL Metrics deep dive. How Gluten maps Substrait plan nodes to Velox operators, aggregates metrics across pipelines, walks the MetricsUpdaterTree, and handles aggregation sub-phases and shuffle metrics.

Deep Dive into Spark SQL Metrics (Part 2): Internals and How AQE Uses Them

Part 2 of the SQL Metrics deep dive. How metrics flow from tasks to driver, and how Adaptive Query Execution uses shuffle statistics to rewrite plans at runtime.