kernel.git/tools/perf/util/bpf_counter.c, branch master

perf bpf_counter: Fix opening of "any"(-1) CPU events

2025-10-13T08:58:51Z

The bperf BPF counter code doesn't handle "any"(-1) CPU events, always wanting to aggregate a count against a CPU, which avoids the need for atomics so let's not change that. Force evsels used for BPF counters to require a CPU when not in system-wide mode so that the "any"(-1) value isn't used during map propagation and evsel's CPU map matches that of the PMU. Fixes: b91917c0c6fa ("perf bpf_counter: Fix handling of cpumap fixing hybrid") Signed-off-by: Ian Rogers Signed-off-by: Namhyung Kim

perf bpf_counter: Fix handling of cpumap fixing hybrid

2025-10-01T18:27:27Z

Don't open evsels on all CPUs, open them just on the CPUs they support. This avoids opening say an e-core event on a p-core and getting a failure - achieve this by getting rid of the "all_cpu_map". In install_pe functions don't use the cpu_map_idx as a CPU number, translate the cpu_map_idx, which is a dense index into the cpu_map skipping holes at the beginning, to a proper CPU number. Before: ``` $ perf stat --bpf-counters -a -e cycles,instructions -- sleep 1 Performance counter stats for 'system wide': cpu_atom/cycles/ 566,270,672 cpu_core/cycles/ cpu_atom/instructions/ 572,792,836 cpu_core/instructions/ # 1.01 insn per cycle 1.001595384 seconds time elapsed ``` After: ``` $ perf stat --bpf-counters -a -e cycles,instructions -- sleep 1 Performance counter stats for 'system wide': 443,299,201 cpu_atom/cycles/ 1,233,919,737 cpu_core/cycles/ 213,634,112 cpu_atom/instructions/ # 0.48 insn per cycle 2,758,965,527 cpu_core/instructions/ # 2.24 insn per cycle 1.001699485 seconds time elapsed ``` Fixes: 7fac83aaf2eecc9e ("perf stat: Introduce 'bperf' to share hardware PMCs with BPF") Signed-off-by: Ian Rogers Cc: Adrian Hunter Cc: Alexander Shishkin Cc: Athira Rajeev Cc: bpf@vger.kernel.org Cc: Gabriele Monaco Cc: Howard Chu Cc: Ingo Molnar Cc: James Clark Cc: Jiri Olsa Cc: Namhyung Kim Cc: Peter Zijlstra Cc: Song Liu Cc: Tengda Wu Signed-off-by: Arnaldo Carvalho de Melo

perf bpf_counter: Move header declarations into C code

2025-10-01T18:27:02Z

Reduce the API surface that is in bpf_counter.h, this helps compiler analysis like unused static function, makes it easier to set a breakpoint and just makes it easier to see the code is self contained. When code is shared between BPF C code, put it inside HAVE_BPF_SKEL. Move transitively found #includes into appropriate C files. No functional change. Signed-off-by: Ian Rogers Cc: Adrian Hunter Cc: Alexander Shishkin Cc: Athira Rajeev Cc: Gabriele Monaco Cc: Howard Chu Cc: Ingo Molnar Cc: James Clark Cc: Jiri Olsa Cc: Namhyung Kim Cc: Peter Zijlstra Cc: Song Liu Cc: Tengda Wu Signed-off-by: Arnaldo Carvalho de Melo

perf stat: Support inherit events during fork() for bperf

2024-11-02T06:31:08Z

bperf has a nice ability to share PMUs, but it still does not support inherit events during fork(), resulting in some deviations in its stat results compared with perf. perf stat result: $ ./perf stat -e cycles,instructions -- ./perf test -w sqrtloop Performance counter stats for './perf test -w sqrtloop': 2,316,038,116 cycles 2,859,350,725 instructions 1.009603637 seconds time elapsed 1.004196000 seconds user 0.003950000 seconds sys bperf stat result: $ ./perf stat --bpf-counters -e cycles,instructions -- \ ./perf test -w sqrtloop Performance counter stats for './perf test -w sqrtloop': 18,762,093 cycles 23,487,766 instructions 1.008913769 seconds time elapsed 1.003248000 seconds user 0.004069000 seconds sys In order to support event inheritance, two new bpf programs are added to monitor the fork and exit of tasks respectively. When a task is created, add it to the filter map to enable counting, and reuse the `accum_key` of its parent task to count together with the parent task. When a task exits, remove it from the filter map to disable counting. After support: $ ./perf stat --bpf-counters -e cycles,instructions -- \ ./perf test -w sqrtloop Performance counter stats for './perf test -w sqrtloop': 2,316,252,189 cycles 2,859,946,547 instructions 1.009422314 seconds time elapsed 1.003597000 seconds user 0.004270000 seconds sys Signed-off-by: Tengda Wu Cc: song@kernel.org Cc: bpf@vger.kernel.org Link: https://lore.kernel.org/r/20241021110201.325617-2-wutengda@huaweicloud.com Signed-off-by: Namhyung Kim

libperf cpumap: Replace usage of perf_cpu_map__new(NULL) with perf_cpu_map__new_online_cpus()

2023-12-12T17:55:13Z

Passing NULL to perf_cpu_map__new() performs perf_cpu_map__new_online_cpus(), just directly call perf_cpu_map__new_online_cpus() to be more intention revealing. Reviewed-by: James Clark Signed-off-by: Ian Rogers Cc: Adrian Hunter Cc: Alexander Shishkin Cc: Alexandre Ghiti Cc: Andrew Jones Cc: André Almeida Cc: Athira Jajeev Cc: Atish Patra Cc: Changbin Du Cc: Darren Hart Cc: Davidlohr Bueso Cc: Huacai Chen Cc: Ingo Molnar Cc: Jiri Olsa Cc: John Garry Cc: K Prateek Nayak Cc: Kajol Jain Cc: Kan Liang Cc: Leo Yan Cc: Mark Rutland Cc: Mike Leach Cc: Namhyung Kim Cc: Nick Desaulniers Cc: Paolo Bonzini Cc: Paran Lee Cc: Peter Zijlstra Cc: Ravi Bangoria Cc: Sandipan Das Cc: Sean Christopherson Cc: Steinar H. Gunderson Cc: Suzuki Poulouse Cc: Thomas Gleixner Cc: Will Deacon Cc: Yang Jihong Cc: Yang Li Cc: Yanteng Si Cc: bpf@vger.kernel.org Cc: coresight@lists.linaro.org Cc: linux-arm-kernel@lists.infradead.org Link: https://lore.kernel.org/r/20231129060211.1890454-5-irogers@google.com Signed-off-by: Arnaldo Carvalho de Melo

perf bpf_counter: Fix a few memory leaks

2023-10-12T17:01:57Z

Memory leaks were detected by clang-tidy. Signed-off-by: Ian Rogers Acked-by: Namhyung Kim Cc: Ravi Bangoria Cc: Nick Desaulniers Cc: Yang Jihong Cc: Huacai Chen Cc: Nathan Chancellor Cc: Kan Liang Cc: llvm@lists.linux.dev Cc: Ming Wang Cc: Tom Rix Cc: bpf@vger.kernel.org Link: https://lore.kernel.org/r/20231009183920.200859-20-irogers@google.com Signed-off-by: Namhyung Kim

perf bperf: Avoid use after free via unrelated 'struct evsel' anonymous union field

2023-04-11T12:22:25Z

If bperf (perf tools that use BPF skels) sets evsel->leader_skel or evsel->follower_skel then it appears that evsel->bpf_skel is set and can trigger the following use-after-free: ==13575==ERROR: AddressSanitizer: heap-use-after-free on address 0x60c000014080 at pc 0x55684b939880 bp 0x7ffdfcf30d70 sp 0x7ffdfcf30d68 READ of size 8 at 0x60c000014080 thread T0 #0 0x55684b93987f in sample_filter_bpf__destroy tools/perf/bpf_skel/sample_filter.skel.h:44:11 #1 0x55684b93987f in perf_bpf_filter__destroy tools/perf/util/bpf-filter.c:155:2 #2 0x55684b98f71e in evsel__exit tools/perf/util/evsel.c:1521:2 #3 0x55684b98a352 in evsel__delete tools/perf/util/evsel.c:1547:2 #4 0x55684b981918 in evlist__purge tools/perf/util/evlist.c:148:3 #5 0x55684b981918 in evlist__delete tools/perf/util/evlist.c:169:2 #6 0x55684b887d60 in cmd_stat tools/perf/builtin-stat.c:2598:2 .. 0x60c000014080 is located 0 bytes inside of 128-byte region [0x60c000014080,0x60c000014100) freed by thread T0 here: #0 0x55684b780e86 in free compiler-rt/lib/asan/asan_malloc_linux.cpp:52:3 #1 0x55684b9462da in bperf_cgroup_bpf__destroy tools/perf/bpf_skel/bperf_cgroup.skel.h:61:2 #2 0x55684b9462da in bperf_cgrp__destroy tools/perf/util/bpf_counter_cgroup.c:282:2 #3 0x55684b944c75 in bpf_counter__destroy tools/perf/util/bpf_counter.c:819:2 #4 0x55684b98f716 in evsel__exit tools/perf/util/evsel.c:1520:2 #5 0x55684b98a352 in evsel__delete tools/perf/util/evsel.c:1547:2 #6 0x55684b981918 in evlist__purge tools/perf/util/evlist.c:148:3 #7 0x55684b981918 in evlist__delete tools/perf/util/evlist.c:169:2 #8 0x55684b887d60 in cmd_stat tools/perf/builtin-stat.c:2598:2 ... previously allocated by thread T0 here: #0 0x55684b781338 in calloc compiler-rt/lib/asan/asan_malloc_linux.cpp:77:3 #1 0x55684b944e25 in bperf_cgroup_bpf__open_opts tools/perf/bpf_skel/bperf_cgroup.skel.h:73:35 #2 0x55684b944e25 in bperf_cgroup_bpf__open tools/perf/bpf_skel/bperf_cgroup.skel.h:97:9 #3 0x55684b944e25 in bperf_load_program tools/perf/util/bpf_counter_cgroup.c:55:9 #4 0x55684b944e25 in bperf_cgrp__load tools/perf/util/bpf_counter_cgroup.c:178:23 #5 0x55684b889289 in __run_perf_stat tools/perf/builtin-stat.c:713:7 #6 0x55684b889289 in run_perf_stat tools/perf/builtin-stat.c:949:8 #7 0x55684b888029 in cmd_stat tools/perf/builtin-stat.c:2537:12 Resolve by clearing 'evsel->bpf_skel' as part of bpf_counter__destroy(). Suggested-by: Namhyung Kim Signed-off-by: Ian Rogers Cc: Adrian Hunter Cc: Alexander Shishkin Cc: Jiri Olsa Cc: Mark Rutland Cc: Namhyung Kim Cc: Peter Zijlstra Cc: bpf@vger.kernel.org Link: http://lore.kernel.org/lkml/20230411051718.267228-1-irogers@google.com Signed-off-by: Arnaldo Carvalho de Melo

perf bpf_counter: Use public cpumap accessors

2023-03-20T15:50:31Z

Avoid the use of internal apis via the cpumap accessor functions. Signed-off-by: Ian Rogers Cc: Adrian Hunter Cc: Alexander Shishkin Cc: Alexey Bayduraev Cc: Andi Kleen Cc: Andrew Morton Cc: André Almeida Cc: Andy Shevchenko Cc: Darren Hart Cc: Davidlohr Bueso Cc: Dmitriy Vyukov Cc: Eric Dumazet Cc: German Gomez Cc: Hao Luo Cc: Ingo Molnar Cc: James Clark Cc: Jiri Olsa Cc: John Garry Cc: Kajol Jain Cc: Kan Liang Cc: Leo Yan Cc: Madhavan Srinivasan Cc: Mark Rutland Cc: Masami Hiramatsu Cc: Miaoqian Lin Cc: Namhyung Kim Cc: Peter Zijlstra Cc: Riccardo Mancini Cc: Shunsuke Nakamura Cc: Song Liu Cc: Stephane Eranian Cc: Stephen Brennan Cc: Steven Rostedt (VMware) Cc: Thomas Gleixner Cc: Thomas Richter Cc: Yury Norov Link: https://lore.kernel.org/r/20230320033810.980165-2-irogers@google.com Signed-off-by: Arnaldo Carvalho de Melo

perf record: Add BPF event filter support

2023-03-15T14:08:34Z

Use --filter option to set BPF filter for generic events other than the tracepoints or Intel PT. The BPF program will check the sample data and filter according to the expression. For example, the below is the typical perf record for frequency mode. The sample period started from 1 and increased gradually. $ sudo ./perf record -e cycles true $ sudo ./perf script perf-exec 2272336 546683.916875: 1 cycles: ffffffff828499b8 perf_event_exec+0x298 ([kernel.kallsyms]) perf-exec 2272336 546683.916892: 1 cycles: ffffffff828499b8 perf_event_exec+0x298 ([kernel.kallsyms]) perf-exec 2272336 546683.916899: 3 cycles: ffffffff828499b8 perf_event_exec+0x298 ([kernel.kallsyms]) perf-exec 2272336 546683.916905: 17 cycles: ffffffff828499b8 perf_event_exec+0x298 ([kernel.kallsyms]) perf-exec 2272336 546683.916911: 100 cycles: ffffffff828499b8 perf_event_exec+0x298 ([kernel.kallsyms]) perf-exec 2272336 546683.916917: 589 cycles: ffffffff828499b8 perf_event_exec+0x298 ([kernel.kallsyms]) perf-exec 2272336 546683.916924: 3470 cycles: ffffffff828499b8 perf_event_exec+0x298 ([kernel.kallsyms]) perf-exec 2272336 546683.916930: 20465 cycles: ffffffff828499b8 perf_event_exec+0x298 ([kernel.kallsyms]) true 2272336 546683.916940: 119873 cycles: ffffffff8283afdd perf_iterate_ctx+0x2d ([kernel.kallsyms]) true 2272336 546683.917003: 461349 cycles: ffffffff82892517 vma_interval_tree_insert+0x37 ([kernel.kallsyms]) true 2272336 546683.917237: 635778 cycles: ffffffff82a11400 security_mmap_file+0x20 ([kernel.kallsyms]) When you add a BPF filter to get samples having periods greater than 1000, the output would look like below: $ sudo ./perf record -e cycles --filter 'period > 1000' true $ sudo ./perf script perf-exec 2273949 546850.708501: 5029 cycles: ffffffff826f9e25 finish_wait+0x5 ([kernel.kallsyms]) perf-exec 2273949 546850.708508: 32409 cycles: ffffffff826f9e25 finish_wait+0x5 ([kernel.kallsyms]) perf-exec 2273949 546850.708526: 143369 cycles: ffffffff82b4cdbf xas_start+0x5f ([kernel.kallsyms]) perf-exec 2273949 546850.708600: 372650 cycles: ffffffff8286b8f7 __pagevec_lru_add+0x117 ([kernel.kallsyms]) perf-exec 2273949 546850.708791: 482953 cycles: ffffffff829190de __mod_memcg_lruvec_state+0x4e ([kernel.kallsyms]) true 2273949 546850.709036: 501985 cycles: ffffffff828add7c tlb_gather_mmu+0x4c ([kernel.kallsyms]) true 2273949 546850.709292: 503065 cycles: 7f2446d97c03 _dl_map_object_deps+0x973 (/usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2) Committer notes: Add stubs for perf_bpf_filter__prepare() and perf_bpf_filter__destroy() to tools/perf/util/python.c to keep it building. Signed-off-by: Namhyung Kim Acked-by: Jiri Olsa Cc: Adrian Hunter Cc: Andi Kleen Cc: Hao Luo Cc: Ian Rogers Cc: Ingo Molnar Cc: James Clark Cc: Kan Liang Cc: Leo Yan Cc: Peter Zijlstra Cc: Ravi Bangoria Cc: Song Liu Cc: Stephane Eranian Cc: bpf@vger.kernel.org Link: https://lore.kernel.org/r/20230314234237.3008956-2-namhyung@kernel.org Signed-off-by: Arnaldo Carvalho de Melo

perf bpf: Remove pre libbpf 1.0 conditional logic

2023-03-14T11:29:46Z

Tests are no longer applicable as libbpf 1.0 can be assumed. Signed-off-by: Ian Rogers Acked/Tested-by: Jiri Olsa Acked/Tested-by: Jiri Olsa Cc: Adrian Hunter Cc: Alexander Shishkin Cc: Andres Freund Cc: Andrii Nakryiko Cc: Christy Lee Cc: Ingo Molnar Cc: Mark Rutland Cc: Namhyung Kim Cc: Peter Zijlstra Cc: Quentin Monnet Cc: Roberto Sassu Cc: bpf@vger.kernel.org Link: https://lore.kernel.org/r/20230116010115.490713-2-irogers@google.com Signed-off-by: Arnaldo Carvalho de Melo