tests: pm: Adding tests based on new approach to PM testing. #80692

gbarkadiusz · 2024-10-31T14:30:53Z

Introduces two scripts abstract_power_monitor.py and pwsh_stm32l562.py.
Adds three new power management pytests based on a new approach.

SETUP

TESTS

pm.states: Verifies the transition to all available power states on stm32l562e_dk.

During the test, pytest detects each step and calculates the RMS (Root Mean Square) current for every detected state. The test then compares the calculated RMS values against the expected values, which have been determined manually through experimental observation

pm.residency_time: Tests the residency time of each power state.
This test evaluates the residency time of different power states on the stm32l562e_dk board.

pm.wakeup_timer: Tests the RTC alarm’s ability to wake the CPU from the lowest power state.

Each test calculates RMS current values for each detected state and compares them to expected values, which were manually set based on experimental data.

To Reproduce

Configure the stm32l562e_dk board according to the instructions provided above.
Connect both USB ports of the board to your host system. The board will appear as two separate targets:
Power Monitor: e.g., /dev/ttyACM0 or /dev/serial/by-id/usb-STMicroelectronics_PowerShield__Virtual_ComPort_in_FS_Mode__FFFFFFFEFFFF-if00
Target: e.g., /dev/ttyACM1 or /dev/serial/by-id/usb-STMicroelectronics_STLINK-V3_004##############39-if02
Copy the path of the Power Monitor and update the <path_to_power_monitor> in the tests/subsys/pm/power_states/testcase.yaml file under the pytest_args section as follows:
pytest_args: [--powershield=<path_to_power_monitor>]
Run the following command to execute the test:
twister --device-testing --device-serial <path_to_target> -p stm32l562e_dk -s <test_name> -x=CONFIG_BOOT_DELAY=500

hakehuang · 2024-11-25T07:35:36Z

tests/subsys/pm/power_states/pytest/test_power_states.py

+    data = current_measurement_output
+    # Empirical RMS values in mA with descriptive keys
+    rms_expected = {
+        "k_cpu_idle":57.0,  # K_cpu_idle


can we move those expected data to a yaml file for different platform?

Of course we can, the good hint. Thank you.

PS: Btw, Please take a look. #80989. Feel free to share ideas or warries. :)

hakehuang · 2024-11-25T07:40:33Z

tests/subsys/pm/power_states/src/main.c

+	while (1) {
+		printk("\nGoing to k_cpu_idle.\n");
+		k_msleep(400);
+		printk("\nWake Up.\n");


can we use this tests\subsys\pm\power_mgmt_soc\src\power_mgmt.c, instead of the sleep time?

I don't know yet, let me see.

Yep, get the information from the device tree is the way to go imho, otherwise it will not be portable and the previous comment is useless.

@ceolin I added the use of DTS, is this what you meant?

hakehuang · 2024-11-25T08:30:50Z

tests/subsys/pm/power_residency_time/pytest/current_analyzer.py

@@ -0,0 +1,30 @@
+# Copyright (c) 2024 Intel Corporation.


would this be better to move this function to scripts/pm/pm_helper.py, and rms is not the only implement, e.g. we can have average mean

gbarkadiusz · 2025-01-10T13:50:31Z

@hakehuang I've moved the hardcoded expected value to the appropriate yaml file. Please take a look.

Introduces scripts for powerShiled on stm32l562e_dk board. Adds three new power management pytests based on a new approach. Signed-off-by: Arkadiusz Cholewinski <[email protected]>

hakehuang

looks good

erwango · 2025-01-16T14:34:49Z

tests/subsys/pm/power_residency_time/testcase.yaml

+      pytest_args: [--powershield=<path_to>/STMicroelectronics_PowerShield__Virtual_ComPort,
+                    --expected-values=<path_to>/stm32l562e_dk.yaml]


IIUC these need to be set manually before testing, right ?

Any way we can spare the need for manual update ?
At least, <path_to>/stm32l562e_dk.yaml is rather static, so it looks reachable.
For <path_to>/STMicroelectronics_PowerShield__Virtual_ComPort maybe a dedicated twister configuration could be used or added ?

bjarki-andreasen · 2025-01-24T14:53:58Z

I believe the approach in this PR is too highly coupled to pm power states, a specific power monitor, and a specific test case.

Measuring current/power or anything else is "generic", its measured in SI units, and can be performed by an "endless" variation of probes. I compare measuring power with "any probe" to flashing a board with "any runner", be it jlink, openocd, pyocd, or whatever.

I believe it can and should be as simple to define a testcase requiring power measurements as this
testcase.yaml:

tests:
  pm.power_states:
    harness: probe
    harness_config:
      measurements:
        - name: k_cpu_idle:
          rms: 56.0
        - name: state_0:
          rms: 56.0

The measurements could be anything, a mode, some peripheral being on, like a comparator in low power mode:

        - name: comp_low_power:
          rms: 56.0
        - name: comp_high_power:
          rms: 71.0

and that specific probes, like runners, should be defined like --probe stm_powershield and --probe-port /dev/ttyACM0 if required, to specify which probe to use for the test. See https://docs.zephyrproject.org/latest/develop/test/twister.html#running-tests-on-hardware for example. Any testsuite can be run for any board, with whatever flasher is available and supported.

This is important, as the test case should not depend on any specific probe, if one developer has a JouleScope, another has a Nordic PPK2, and a third has an stm_powershield, they should all be able to run the same testcase. We are measuring the same "generic" properties after all.

This PR keeps a bunch of this local, using pytest, but I don't think that is a nice, scalable approach to power measurements.

gbarkadiusz added the DNM This PR should not be merged (Do Not Merge) label Oct 31, 2024

gbarkadiusz changed the title ~~tests: pm: Adding tests based on a new approach to power management~~ tests: pm: Adding tests based on new approach to PM testing. Oct 31, 2024

gbarkadiusz force-pushed the topic/rfc/power_management branch 8 times, most recently from bb6e11e to cc760a2 Compare November 5, 2024 11:26

gbarkadiusz mentioned this pull request Nov 6, 2024

RFC: New Testing Approach for Power Management Subsystem. #80989

Open

gbarkadiusz linked an issue Nov 6, 2024 that may be closed by this pull request

RFC: New Testing Approach for Power Management Subsystem. #80989

Open

nashif requested review from ceolin, erwango and bjarki-andreasen November 13, 2024 10:07

gbarkadiusz marked this pull request as ready for review November 14, 2024 09:38

zephyrbot added the area: Power Management label Nov 14, 2024

zephyrbot requested review from JordanYates, nashif, teburd and tmleman November 14, 2024 09:39

zephyrbot assigned ceolin Nov 14, 2024

hakehuang reviewed Nov 25, 2024

View reviewed changes

gbarkadiusz marked this pull request as draft November 25, 2024 14:49

gbarkadiusz force-pushed the topic/rfc/power_management branch 3 times, most recently from f959bb2 to 980a943 Compare December 10, 2024 14:35

gbarkadiusz force-pushed the topic/rfc/power_management branch 15 times, most recently from 1f00e48 to 69e3a4f Compare December 13, 2024 11:57

gbarkadiusz force-pushed the topic/rfc/power_management branch 6 times, most recently from 2525c2f to dc18404 Compare January 10, 2025 13:43

gbarkadiusz requested a review from hakehuang January 10, 2025 13:49

gbarkadiusz marked this pull request as ready for review January 10, 2025 13:50

gbarkadiusz force-pushed the topic/rfc/power_management branch from dc18404 to cbc99c3 Compare January 15, 2025 11:56

gbarkadiusz removed the DNM This PR should not be merged (Do Not Merge) label Jan 15, 2025

tests: pm: Adding tests based on new approach to PM testing.

cbc99c3

Introduces scripts for powerShiled on stm32l562e_dk board. Adds three new power management pytests based on a new approach. Signed-off-by: Arkadiusz Cholewinski <[email protected]>

hakehuang approved these changes Jan 16, 2025

View reviewed changes

erwango reviewed Jan 16, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

tests: pm: Adding tests based on new approach to PM testing. #80692

tests: pm: Adding tests based on new approach to PM testing. #80692

gbarkadiusz commented Oct 31, 2024 •

edited

Loading

hakehuang Nov 25, 2024 •

edited

Loading

gbarkadiusz Nov 25, 2024

gbarkadiusz Nov 25, 2024

hakehuang Nov 25, 2024

gbarkadiusz Nov 25, 2024

ceolin Dec 2, 2024

gbarkadiusz Dec 5, 2024

hakehuang Nov 25, 2024

gbarkadiusz commented Jan 10, 2025

hakehuang left a comment

erwango Jan 16, 2025

bjarki-andreasen commented Jan 24, 2025 •

edited

Loading

		pytest_args: [--powershield=<path_to>/STMicroelectronics_PowerShield__Virtual_ComPort,
		--expected-values=<path_to>/stm32l562e_dk.yaml]

tests: pm: Adding tests based on new approach to PM testing. #80692

Are you sure you want to change the base?

tests: pm: Adding tests based on new approach to PM testing. #80692

Conversation

gbarkadiusz commented Oct 31, 2024 • edited Loading

SETUP

TESTS

hakehuang Nov 25, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gbarkadiusz commented Jan 10, 2025

hakehuang left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bjarki-andreasen commented Jan 24, 2025 • edited Loading

gbarkadiusz commented Oct 31, 2024 •

edited

Loading

hakehuang Nov 25, 2024 •

edited

Loading

bjarki-andreasen commented Jan 24, 2025 •

edited

Loading