new parameter hardwaretuning includes max and min frequency for reserved cpus

[sabbir-47]

What?

Configuring CPU frequency for reserved and isolated cpus through PerformanceProfile.

Why?

• For FlexRAN (and FlexRAN-like applications), the hardware vendors require that the default cpu frequency be set to a specific frequency for running workloads which is actually lower than the default running frequency.
• Users constantly ask for reducing the platform core footprint. By setting higher frequencies for reserved cpus can translates the effort into reducing the number of cpu cores reserved for running Openshift components.

Additional resources:

• EPIC link: Reserved Core Frequency Tuning
• Design discussion

Characteristics

• CPU frequency setting is optional
• Requires intel_pstate to be active
• This low level parameters should only be used based on the vendors recommendation on frequencies.
• Feature is hardware agnostic, but must follow frequency guideline depending on hw generation.

How?

By introducing a hardwareTuning parameter in the spec of PerformanceProfile. The onus is on user to provide correct frequencies.

hardwareTuning:
  isolatedCpuFreq: 2500000
  reservedCpuFreq: 2800000

It will update the assets/performanceprofile/tuned/openshift-node-performance and cpu plugin will apply the frequencies using sysfs directive

Changes

• Extended CRD definition
• Modified static openshift-node-performance profile
• intel_pstate=active is changed as default
• When realTime workloadHints is set to true, previously it would disable intel_pstate and would inactive cpufreq driver. With this proposal, intel_pstate=active is set for realTime=true
• Extended performance profile creator by introducing a new flag i.e.; --enable-hardware-tuning. When passed, commented lines get added at the end with an example on how to set the frequency values. For example:

apiVersion: performance.openshift.io/v2
kind: PerformanceProfile
metadata:
  name: performance
spec:
  cpu:
    isolated: 4-15
    reserved: 0-3
    .......................
    .......................
  workloadHints:
    highPowerConsumption: false
    perPodPowerManagement: false
    realTime: true
#HardwareTuning is an advanced feature, and only intended to be used if 
#user is aware of the vendor recommendation on maximum cpu frequency.
#The structure must follow
#
# hardwareTuning:
#   isolatedCpuFreq: <Maximum frequency for applications running on isolated cpus>
#   reservedCpuFreq: <Maximum frequency for platform software running on reserved cpus>

When it will not work?

• For RAN cases, we recommend below workloadHints:

    highPowerConsumption: false
    perPodPowerManagement: false
    realTime: true

if perPodPowerManagement is set to true, the default intel_pstate is set to passive. For this feature to take in effect, P-state needs to be active. Therefore, cpu frequencies will not be set if perPodPowerManagement is true

• Providing incorrect frequencies will not take in effect, and it is strongly recommended to follow hardware vendors guideline.

Can it be done other ways?

Changing cpu frequencies can be done through the tuned performance-patch, using the sysfs plugin. This would look something like this (using 2.5 GHz as the default frequency and 2.8 GHz for reserved CPUs on a system with 64 CPUs and CPUS 0,1,32,33 reserved for the platform):

[sysfs]
/sys/devices/system/cpu/cpu0/cpufreq/scaling_max_freq=2800000
/sys/devices/system/cpu/cpu1/cpufreq/scaling_max_freq=2800000
/sys/devices/system/cpu/cpu2/cpufreq/scaling_max_freq=2500000
/sys/devices/system/cpu/cpu3/cpufreq/scaling_max_freq=2500000
<...>
/sys/devices/system/cpu/cpu31/cpufreq/scaling_max_freq=2500000
/sys/devices/system/cpu/cpu32/cpufreq/scaling_max_freq=2800000
/sys/devices/system/cpu/cpu33/cpufreq/scaling_max_freq=2800000
/sys/devices/system/cpu/cpu34/cpufreq/scaling_max_freq=2500000
/sys/devices/system/cpu/cpu35/cpufreq/scaling_max_freq=2500000
<...>
/sys/devices/system/cpu/cpu63/cpufreq/scaling_max_freq=2500000

This requires 64 lines of configuration (one per logical CPU). The proposed approach reduces the complexity of this configuration, makes it lesser prone to mistake from users side.