kernel.git/tools/perf/util/bpf_counter_cgroup.c, branch master

perf auxtrace: Remove errno.h from auxtrace.h and fix transitive dependencies

2025-11-14T07:03:11Z

errno.h isn't used in auxtrace.h so remove it and fix build failures caused by transitive dependencies through auxtrace.h on errno.h. Signed-off-by: Ian Rogers Reviewed-by: James Clark Signed-off-by: Namhyung Kim

perf stat: Add/fix bperf cgroup max events workarounds

2025-10-25T23:44:21Z

Commit b8308511f6e0 bumped the max events to 1024 but this results in BPF verifier issues if the number of command line events is too large. Workaround this by: 1) moving the constants to a header file to share between BPF and perf C code, 2) testing that the maximum number of events doesn't cause BPF verifier issues in debug builds, 3) lower the max events from 1024 to 128, 4) in perf stat, if there are more events than the BPF counters can support then disable BPF counter usage. The rodata setup is factored into its own function to avoid duplicating it in the testing code. Signed-off-by: Ian Rogers Fixes: b8308511f6e0 ("perf stat bperf cgroup: Increase MAX_EVENTS from 32 to 1024") Signed-off-by: Namhyung Kim

perf bpf_counter: Fix handling of cpumap fixing hybrid

2025-10-01T18:27:27Z

Don't open evsels on all CPUs, open them just on the CPUs they support. This avoids opening say an e-core event on a p-core and getting a failure - achieve this by getting rid of the "all_cpu_map". In install_pe functions don't use the cpu_map_idx as a CPU number, translate the cpu_map_idx, which is a dense index into the cpu_map skipping holes at the beginning, to a proper CPU number. Before: ``` $ perf stat --bpf-counters -a -e cycles,instructions -- sleep 1 Performance counter stats for 'system wide': cpu_atom/cycles/ 566,270,672 cpu_core/cycles/ cpu_atom/instructions/ 572,792,836 cpu_core/instructions/ # 1.01 insn per cycle 1.001595384 seconds time elapsed ``` After: ``` $ perf stat --bpf-counters -a -e cycles,instructions -- sleep 1 Performance counter stats for 'system wide': 443,299,201 cpu_atom/cycles/ 1,233,919,737 cpu_core/cycles/ 213,634,112 cpu_atom/instructions/ # 0.48 insn per cycle 2,758,965,527 cpu_core/instructions/ # 2.24 insn per cycle 1.001699485 seconds time elapsed ``` Fixes: 7fac83aaf2eecc9e ("perf stat: Introduce 'bperf' to share hardware PMCs with BPF") Signed-off-by: Ian Rogers Cc: Adrian Hunter Cc: Alexander Shishkin Cc: Athira Rajeev Cc: bpf@vger.kernel.org Cc: Gabriele Monaco Cc: Howard Chu Cc: Ingo Molnar Cc: James Clark Cc: Jiri Olsa Cc: Namhyung Kim Cc: Peter Zijlstra Cc: Song Liu Cc: Tengda Wu Signed-off-by: Arnaldo Carvalho de Melo

perf bpf_counter: Move header declarations into C code

2025-10-01T18:27:02Z

Reduce the API surface that is in bpf_counter.h, this helps compiler analysis like unused static function, makes it easier to set a breakpoint and just makes it easier to see the code is self contained. When code is shared between BPF C code, put it inside HAVE_BPF_SKEL. Move transitively found #includes into appropriate C files. No functional change. Signed-off-by: Ian Rogers Cc: Adrian Hunter Cc: Alexander Shishkin Cc: Athira Rajeev Cc: Gabriele Monaco Cc: Howard Chu Cc: Ingo Molnar Cc: James Clark Cc: Jiri Olsa Cc: Namhyung Kim Cc: Peter Zijlstra Cc: Song Liu Cc: Tengda Wu Signed-off-by: Arnaldo Carvalho de Melo

perf stat: Constify control data for BPF

2024-09-03T14:43:16Z

The control knobs set before loading BPF programs should be declared as 'const volatile' so that it can be optimized by the BPF core. Committer testing: root@x1:~# perf stat --bpf-counters -e cpu_core/cycles/,cpu_core/instructions/ sleep 1 Performance counter stats for 'sleep 1': 2,442,583 cpu_core/cycles/ 2,494,425 cpu_core/instructions/ 1.002687372 seconds time elapsed 0.001126000 seconds user 0.001166000 seconds sys root@x1:~# perf trace -e bpf --max-events 10 perf stat --bpf-counters -e cpu_core/cycles/,cpu_core/instructions/ sleep 1 0.000 ( 0.019 ms): perf/2944119 bpf(cmd: OBJ_GET, uattr: 0x7fffdf5cdd40, size: 20) = 5 0.021 ( 0.002 ms): perf/2944119 bpf(cmd: OBJ_GET_INFO_BY_FD, uattr: 0x7fffdf5cdcd0, size: 16) = 0 0.030 ( 0.005 ms): perf/2944119 bpf(cmd: MAP_LOOKUP_ELEM, uattr: 0x7fffdf5ceda0, size: 32) = 0 0.037 ( 0.004 ms): perf/2944119 bpf(cmd: LINK_GET_FD_BY_ID, uattr: 0x7fffdf5ced80, size: 12) = -1 ENOENT (No such file or directory) 0.189 ( 0.004 ms): perf/2944119 bpf(cmd: 36, uattr: 0x7fffdf5cec10, size: 8) = -1 EOPNOTSUPP (Operation not supported) 0.201 ( 0.095 ms): perf/2944119 bpf(cmd: PROG_LOAD, uattr: 0x7fffdf5ce940, size: 148) = 10 0.305 ( 0.026 ms): perf/2944119 bpf(cmd: PROG_LOAD, uattr: 0x7fffdf5cea00, size: 148) = 10 0.347 ( 0.012 ms): perf/2944119 bpf(cmd: BTF_LOAD, uattr: 0x7fffdf5ce8e0, size: 40) = 10 0.364 ( 0.004 ms): perf/2944119 bpf(cmd: BTF_LOAD, uattr: 0x7fffdf5ce950, size: 40) = 10 0.376 ( 0.006 ms): perf/2944119 bpf(cmd: BTF_LOAD, uattr: 0x7fffdf5ce730, size: 40) = 10 root@x1:~# Performance counter stats for 'sleep 1': 271,221 cpu_core/cycles/ 139,150 cpu_core/instructions/ 1.002881677 seconds time elapsed 0.001318000 seconds user 0.001314000 seconds sys root@x1:~# Signed-off-by: Namhyung Kim Tested-by: Arnaldo Carvalho de Melo Cc: Adrian Hunter Cc: Ian Rogers Cc: Ingo Molnar Cc: Jiri Olsa Cc: Kan Liang Cc: Peter Zijlstra Cc: Song Liu Link: https://lore.kernel.org/r/20240902200515.2103769-2-namhyung@kernel.org Signed-off-by: Arnaldo Carvalho de Melo

perf tools: Ignore deleted cgroups

2024-05-10T13:52:46Z

On large systems, cgroups can be created and deleted often. That means there's a race between perf tools and cgroups when it gets the cgroup name and opens the cgroup. I got a report that 'perf stat' with many cgroups failed quite often due to the missing cgroups on such a large machine. I think we can ignore such cgroups when expanding events and use id 0 if it fails to read the cgroup id. IIUC 0 is not a vaild cgroup id so it won't update event counts for the failed cgroups. Signed-off-by: Namhyung Kim Cc: Adrian Hunter Cc: Ian Rogers Cc: Ingo Molnar Cc: Jiri Olsa Cc: Kan Liang Cc: Peter Zijlstra Link: https://lore.kernel.org/r/20240509182235.2319599-1-namhyung@kernel.org Signed-off-by: Arnaldo Carvalho de Melo

perf stat: Fix handling of unsupported cgroup events when using BPF counters

2023-01-04T13:52:07Z

When --for-each-cgroup option is used, it fails when any of events is not supported and exits immediately. This is not how 'perf stat' handles unsupported events. Let's ignore the failure and proceed with others so that the output is similar to when BPF counters are not used: Before: $ sudo ./perf stat -a --bpf-counters -e L1-icache-loads,L1-dcache-loads --for-each-cgroup system.slice,user.slice sleep 1 Failed to open first cgroup events $ After it shows output similat to when --bpf-counters isn't specified: $ sudo ./perf stat -a --bpf-counters -e L1-icache-loads,L1-dcache-loads --for-each-cgroup system.slice,user.slice sleep 1 Performance counter stats for 'system wide': L1-icache-loads system.slice 29,892,418 L1-dcache-loads system.slice L1-icache-loads user.slice 52,497,220 L1-dcache-loads user.slice $ Fixes: 944138f048f7d759 ("perf stat: Enable BPF counter with --for-each-cgroup") Signed-off-by: Namhyung Kim Tested-by: Arnaldo Carvalho de Melo Cc: Adrian Hunter Cc: Ian Rogers Cc: Ingo Molnar Cc: Jiri Olsa Cc: Peter Zijlstra Cc: Song Liu Link: https://lore.kernel.org/r/20230104064402.1551516-4-namhyung@kernel.org Signed-off-by: Arnaldo Carvalho de Melo

perf stat: Use evsel->core.cpus to iterate cpus in BPF cgroup counters

2022-09-21T13:30:55Z

If it mixes core and uncore events, each evsel would have different cpu map. But it assumed they are same with evlist's all_cpus and accessed by the same index. This resulted in a crash like below. $ perf stat -a --bpf-counters --for-each_cgroup ^. -e cycles,imc/cas_count_read/ sleep 1 Segmentation fault While it's not recommended to use uncore events for cgroup aggregation, it should not crash. Signed-off-by: Namhyung Kim Cc: Adrian Hunter Cc: Ian Rogers Cc: Ingo Molnar Cc: Jiri Olsa Cc: Peter Zijlstra Cc: Song Liu Cc: bpf@vger.kernel.org Link: https://lore.kernel.org/r/20220916184132.1161506-4-namhyung@kernel.org Signed-off-by: Arnaldo Carvalho de Melo

perf stat: Fix cpu map index in bperf cgroup code

2022-09-21T13:30:55Z

The previous cpu map introduced a bug in the bperf cgroup counter. This results in a failure when user gives a partial cpu map starting from non-zero. $ sudo ./perf stat -C 1-2 --bpf-counters --for-each-cgroup ^. sleep 1 libbpf: prog 'on_cgrp_switch': failed to create BPF link for perf_event FD 0: -9 (Bad file descriptor) Failed to attach cgroup program To get the FD of an evsel, it should use a map index not the CPU number. Fixes: 0255571a16059c8e ("perf cpumap: Switch to using perf_cpu_map API") Signed-off-by: Namhyung Kim Cc: Adrian Hunter Cc: bpf@vger.kernel.org Cc: Ian Rogers Cc: Ingo Molnar Cc: Jiri Olsa Cc: Peter Zijlstra Cc: Song Liu Link: https://lore.kernel.org/r/20220916184132.1161506-3-namhyung@kernel.org Signed-off-by: Arnaldo Carvalho de Melo

perf cpumap: Switch to using perf_cpu_map API

2022-05-05T17:07:27Z

Switch some raw accesses to the cpu map to using the library API. This can help with reference count checking. Some BPF cases switch from index to CPU for consistency, this shouldn't matter as the CPU map is full. Signed-off-by: Ian Rogers Acked-by: Namhyung Kim Cc: Adrian Hunter Cc: Alexander Antonov Cc: Alexander Shishkin Cc: Alexei Starovoitov Cc: Alexey Bayduraev Cc: Andi Kleen Cc: Andrii Nakryiko Cc: Daniel Borkmann Cc: German Gomez Cc: James Clark Cc: Jiri Olsa Cc: John Fastabend Cc: John Garry Cc: KP Singh Cc: Kajol Jain Cc: Leo Yan Cc: Mark Rutland Cc: Martin KaFai Lau Cc: Mathieu Poirier Cc: Mike Leach Cc: Peter Zijlstra Cc: Riccardo Mancini Cc: Song Liu Cc: Stephane Eranian Cc: Suzuki Poulouse Cc: Will Deacon Cc: Yonghong Song Link: http://lore.kernel.org/lkml/20220503041757.2365696-2-irogers@google.com Signed-off-by: Arnaldo Carvalho de Melo