Skip to content

feat(perf): add per-plan hardware performance counters to EXPLAIN PERF#19493

Merged
zhang2014 merged 5 commits intodatabendlabs:mainfrom
dqhl76:perf-event
Mar 9, 2026
Merged

feat(perf): add per-plan hardware performance counters to EXPLAIN PERF#19493
zhang2014 merged 5 commits intodatabendlabs:mainfrom
dqhl76:perf-event

Conversation

@dqhl76
Copy link
Copy Markdown
Collaborator

@dqhl76 dqhl76 commented Feb 28, 2026

I hereby agree to the terms of the CLA available at: https://docs.databend.com/dev/policies/cla/

Summary

closes #19488

This PR integrates Linux perf_event hardware performance counters into EXPLAIN PERF, providing CPU microarchitecture-level insights per query plan node alongside
the existing flamegraph output.

Extended SQL syntax

The EXPLAIN PERF syntax has been extended to support both default and custom hardware event profiling. We introduced a grouping syntax (+) to pair related events, which is necessary for calculating derived metrics like IPC (Instructions Per Cycle) or cache miss rates.

default:

  -- default events (cycles+instructions, branch-misses, cache-misses+cache-refs)
  EXPLAIN PERF SELECT ...;

group cycles+instructions to get IPC to get cache miss rate:

explain perf (events='cycles+instructions,branch-misses') select avg(number) from numbers(10000);

not group:

  EXPLAIN PERF (events = 'page-faults, cycles') SELECT ...;
image

Tests

  • Unit Test
  • Logic Test
  • Benchmark Test
  • No Test - Explain why

Type of change

  • Bug Fix (non-breaking change which fixes an issue)
  • New Feature (non-breaking change which adds functionality)
  • Breaking Change (fix or feature that could cause existing functionality not to work as expected)
  • Documentation Update
  • Refactoring
  • Performance Improvement
  • Other (please describe):

This change is Reviewable

@github-actions github-actions bot added the pr-feature this PR introduces a new feature to the codebase label Feb 28, 2026
@dqhl76 dqhl76 added the ci-cloud Build docker image for cloud test label Mar 4, 2026
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Mar 4, 2026

Docker Image for PR

  • tag: pr-19493-70627fe-1772627191

note: this image tag is only available for internal use.

dqhl76 and others added 2 commits March 5, 2026 08:11
- Add PerfEvent hw counters (cycles, instructions, branch misses, cache refs/misses)
- Support grouped and standalone perf counter modes with per-event multiplexing info
- Propagate PerfConfig through ExecutorSettings and QueryCtxShared
- Add DataPacket::QueryPerfCounters flight packet for distributed hw counter collection
- Fix distributed EXPLAIN PERF: enable perf on remote nodes, propagate executor to ctx
- Fix double profiler start on coordinator node via is_request_server guard
- Render per-node hw counter tables in EXPLAIN PERF HTML output

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@dqhl76 dqhl76 added ci-cloud Build docker image for cloud test and removed ci-cloud Build docker image for cloud test labels Mar 5, 2026
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Mar 5, 2026

Docker Image for PR

  • tag: pr-19493-3e72a17-1772702834

note: this image tag is only available for internal use.

Add missing Software variants (CpuClock, AlignmentFaults, EmulationFaults,
BpfOutput, CgroupSwitches) and Cache variant (NodeReadMisses, NodeReadAccesses).
Reorder Hardware and Software sections to match perf_event2 declaration order.
Add sqllogictest covering all event names.
@dqhl76 dqhl76 marked this pull request as ready for review March 6, 2026 01:18
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6282a11ef8

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

@dqhl76 dqhl76 requested a review from zhang2014 March 6, 2026 01:48
@zhang2014 zhang2014 merged commit a430499 into databendlabs:main Mar 9, 2026
101 checks passed
@dqhl76 dqhl76 deleted the perf-event branch March 10, 2026 11:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci-cloud Build docker image for cloud test pr-feature this PR introduces a new feature to the codebase

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature: add per-plan hardware performance counters to EXPLAIN PERF

2 participants