kernel.git/drivers/cpuidle/cpuidle.c, branch linux-6.4.y

cpuidle: move to use bus_get_dev_root()

2023-03-22T19:25:23Z

Direct access to the struct bus_type dev_root pointer is going away soon so replace that with a call to bus_get_dev_root() instead, which is what it is there for. This allows us to clean up the cpuidle_add_interface() call a bit as it was only called in one place, with the same argument so just put that into the function itself. Note that cpuidle_remove_interface() should also probably be removed in the future as there are no callers of it for some reason. Cc: Daniel Lezcano Cc: linux-pm@vger.kernel.org Acked-by: Rafael J. Wysocki Link: https://lore.kernel.org/r/20230322090557.2943479-1-gregkh@linuxfoundation.org Signed-off-by: Greg Kroah-Hartman

cpuidle: Fix poll_idle() noinstr annotation

2023-01-31T14:01:47Z

The instrumentation_begin()/end() annotations in poll_idle() were complete nonsense. Specifically they caused tracing to happen in the middle of noinstr code, resulting in RCU splats. Now that local_clock() is noinstr, mark up the rest and let it rip. Fixes: 00717eb8c955 ("cpuidle: Annotate poll_idle()") Reported-by: kernel test robot Signed-off-by: Peter Zijlstra (Intel) Signed-off-by: Ingo Molnar Acked-by: Rafael J. Wysocki Link: https://lore.kernel.org/oe-lkp/202301192148.58ece903-oliver.sang@intel.com Link: https://lore.kernel.org/r/20230126151323.819534689@infradead.org

cpuidle: Add comments about noinstr/__cpuidle usage

2023-01-13T10:48:18Z

Add a few words on noinstr / __cpuidle usage. Signed-off-by: Peter Zijlstra (Intel) Signed-off-by: Ingo Molnar Link: https://lore.kernel.org/r/20230112195542.397238052@infradead.org

cpuidle: Ensure ct_cpuidle_enter() is always called from noinstr/__cpuidle

2023-01-13T10:48:17Z

Tracing (kprobes included) and other compiler instrumentation relies on a normal kernel runtime. Therefore all functions that disable RCU should be noinstr, as should all functions that are called while RCU is disabled. Signed-off-by: Peter Zijlstra (Intel) Signed-off-by: Ingo Molnar Link: https://lore.kernel.org/r/20230112195542.212914195@infradead.org

cpuidle: Fix ct_idle_*() usage

2023-01-13T10:48:15Z

The whole disable-RCU, enable-IRQS dance is very intricate since changing IRQ state is traced, which depends on RCU. Add two helpers for the cpuidle case that mirror the entry code: ct_cpuidle_enter() ct_cpuidle_exit() And fix all the cases where the enter/exit dance was buggy. Signed-off-by: Peter Zijlstra (Intel) Signed-off-by: Ingo Molnar Tested-by: Tony Lindgren Tested-by: Ulf Hansson Acked-by: Rafael J. Wysocki Acked-by: Frederic Weisbecker Link: https://lore.kernel.org/r/20230112195540.130014793@infradead.org

cpuidle: Move IRQ state validation

2023-01-13T10:03:22Z

Make cpuidle_enter_state() consistent with the s2idle variant and verify ->enter() always returns with interrupts disabled. Signed-off-by: Peter Zijlstra (Intel) Signed-off-by: Ingo Molnar Tested-by: Tony Lindgren Tested-by: Ulf Hansson Acked-by: Rafael J. Wysocki Acked-by: Frederic Weisbecker Link: https://lore.kernel.org/r/20230112195539.576412812@infradead.org

Merge tag 'pm-5.20-rc1-2' of git://git.kernel.org/pub/scm/linux/kernel/git/rafael/linux-pm

2022-08-08T21:29:00Z

Pull more power management updates from Rafael Wysocki: "These are ARM cpufreq updates and operating performance points (OPP) updates plus one cpuidle update adding a new trace point. Specifics: - Fix return error code in mtk_cpu_dvfs_info_init (Yang Yingliang). - Minor cleanups and support for new boards for Qcom cpufreq drivers (Bryan O'Donoghue, Konrad Dybcio, Pierre Gondois, and Yicong Yang). - Fix sparse warnings for Tegra cpufreq driver (Viresh Kumar). - Make dev_pm_opp_set_regulators() accept NULL terminated list (Viresh Kumar). - Add dev_pm_opp_set_config() and friends and migrate other users and helpers to using them (Viresh Kumar). - Add support for multiple clocks for a device (Viresh Kumar and Krzysztof Kozlowski). - Configure resources before adding OPP table for Venus (Stanimir Varbanov). - Keep reference count up for opp->np and opp_table->np while they are still in use (Liang He). - Minor OPP cleanups (Viresh Kumar and Yang Li). - Add a trace event for cpuidle to track missed (too deep or too shallow) wakeups (Kajetan Puchalski)" * tag 'pm-5.20-rc1-2' of git://git.kernel.org/pub/scm/linux/kernel/git/rafael/linux-pm: (55 commits) cpuidle: Add cpu_idle_miss trace event venus: pm_helpers: Fix warning in OPP during probe OPP: Don't drop opp->np reference while it is still in use OPP: Don't drop opp_table->np reference while it is still in use cpufreq: tegra194: Staticize struct tegra_cpufreq_soc instances dt-bindings: cpufreq: cpufreq-qcom-hw: Add SM6375 compatible dt-bindings: opp: Add msm8939 to the compatible list dt-bindings: opp: Add missing compat devices dt-bindings: opp: opp-v2-kryo-cpu: Fix example binding checks cpufreq: Change order of online() CB and policy->cpus modification cpufreq: qcom-hw: Remove deprecated irq_set_affinity_hint() call cpufreq: qcom-hw: Disable LMH irq when disabling policy cpufreq: qcom-hw: Reset cancel_throttle when policy is re-enabled cpufreq: qcom-cpufreq-hw: use HZ_PER_KHZ macro in units.h cpufreq: mediatek: fix error return code in mtk_cpu_dvfs_info_init() OPP: Remove dev{m}_pm_opp_of_add_table_noclk() PM / devfreq: tegra30: Register config_clks helper OPP: Allow config_clks helper for single clk case OPP: Provide a simple implementation to configure multiple clocks OPP: Assert clk_count == 1 for single clk helpers ...

cpuidle: Add cpu_idle_miss trace event

2022-08-03T15:50:58Z

Add a trace event for cpuidle to track missed (too deep or too shallow) wakeups. After each wakeup, CPUIdle already computes whether the entered state was optimal, above or below the desired one and updates the relevant counters. This patch makes it possible to trace those events in addition to just reading the counters. The patterns of types and percentages of misses across different workloads appear to be very consistent. This makes the trace event very useful for comparing the relative correctness of different CPUIdle governors for different types of workloads, or for finding the optimal governor for a given device. Signed-off-by: Kajetan Puchalski Reviewed-by: Steven Rostedt (Google) Signed-off-by: Rafael J. Wysocki

context_tracking: Take idle eqs entrypoints over RCU

2022-07-05T20:32:16Z

The RCU dynticks counter is going to be merged into the context tracking subsystem. Start with moving the idle extended quiescent states entrypoints to context tracking. For now those are dumb redirections to existing RCU calls. [ paulmck: Apply kernel test robot feedback. ] Signed-off-by: Frederic Weisbecker Cc: Peter Zijlstra Cc: Thomas Gleixner Cc: Neeraj Upadhyay Cc: Uladzislau Rezki Cc: Joel Fernandes Cc: Boqun Feng Cc: Nicolas Saenz Julienne Cc: Marcelo Tosatti Cc: Xiongfeng Wang Cc: Yu Liao Cc: Phil Auld Cc: Paul Gortmaker Cc: Alex Belits Signed-off-by: Paul E. McKenney Reviewed-by: Nicolas Saenz Julienne Tested-by: Nicolas Saenz Julienne

cpuidle: Select polling interval based on a c-state with a longer target residency

2020-12-01T16:59:18Z

It was noted that a few workloads that idle rapidly regressed when commit 36fcb4292473 ("cpuidle: use first valid target residency as poll time") was merged. The workloads in question were heavy communicators that idle rapidly and were impacted by the c-state exit latency as the active CPUs were not polling at the time of wakeup. As they were not particularly realistic workloads, it was not considered to be a major problem. Unfortunately, a bug was reported for a real workload in a production environment that relied on large numbers of threads operating in a worker pool pattern. These threads would idle for periods of time longer than the C1 target residency and so incurred the c-state exit latency penalty. The application is very sensitive to wakeup latency and indirectly relying on behaviour prior to commit on a37b969a61c1 ("cpuidle: poll_state: Add time limit to poll_idle()") to poll for long enough to avoid the exit latency cost. The target residency of C1 is typically very short. On some x86 machines, it can be as low as 2 microseconds. In poll_idle(), the clock is checked every POLL_IDLE_RELAX_COUNT interations of cpu_relax() and even one iteration of that loop can be over 1 microsecond so the polling interval is very close to the granularity of what poll_idle() can detect. Furthermore, a basic ping pong workload like perf bench pipe has a longer round-trip time than the 2 microseconds meaning that the CPU will almost certainly not be polling when the ping-pong completes. This patch selects a polling interval based on an enabled c-state that has an target residency longer than 10usec. If there is no enabled-cstate then polling will be up to a TICK_NSEC/16 similar to what it was up until kernel 4.20. Polling for a full tick is unlikely (rescheduling event) and is much longer than the existing target residencies for a deep c-state. As an example, consider a CPU with the following c-state information from an Intel CPU; residency exit_latency C1 2 2 C1E 20 10 C3 100 33 C6 400 133 The polling interval selected is 20usec. If booted with intel_idle.max_cstate=1 then the polling interval is 250usec as the deeper c-states were not available. On an AMD EPYC machine, the c-state information is more limited and looks like residency exit_latency C1 2 1 C2 800 400 The polling interval selected is 250usec. While C2 was considered, the polling interval was clamped by CPUIDLE_POLL_MAX. Note that it is not expected that polling will be a universal win. As well as potentially trading power for performance, the performance is not guaranteed if the extra polling prevented a turbo state being reached. Making it a tunable was considered but it's driver-specific, may be overridden by a governor and is not a guaranteed polling interval making it difficult to describe without knowledge of the implementation. tbench4 vanilla polling Hmean 1 497.89 ( 0.00%) 543.15 * 9.09%* Hmean 2 975.88 ( 0.00%) 1059.73 * 8.59%* Hmean 4 1953.97 ( 0.00%) 2081.37 * 6.52%* Hmean 8 3645.76 ( 0.00%) 4052.95 * 11.17%* Hmean 16 6882.21 ( 0.00%) 6995.93 * 1.65%* Hmean 32 10752.20 ( 0.00%) 10731.53 * -0.19%* Hmean 64 12875.08 ( 0.00%) 12478.13 * -3.08%* Hmean 128 21500.54 ( 0.00%) 21098.60 * -1.87%* Hmean 256 21253.70 ( 0.00%) 21027.18 * -1.07%* Hmean 320 20813.50 ( 0.00%) 20580.64 * -1.12%* Signed-off-by: Mel Gorman Signed-off-by: Rafael J. Wysocki