ASoC: SOF: Intel: Add support for hardware mic privacy change reporting #5312

ujfalusi · 2025-01-29T12:33:05Z

Hi,

ACE3 (Panther Lake) introduced support for microphone privacy feature which
can - in hardware - mute incoming audio data based on a state of a physical
switch.
The change in the privacy state is delivered through interface IP blocks
and can only be handled by the link owner.
In Intel platforms Soundwire is for example host owned, so the interrupt
can only be handled by the host.

Since the input stream is going to be muted by hardware, the host needs to
send a message to firmware about the change in privacy so it can execute a
fade out/in to enhance user experience.

The support for microphone privacy can be queried from the HW_CONFIG data
under the INTEL_MIC_PRIVACY_CAP tuple. This is Intel specific data, the
core will pass it to platform code if the intel_configure_mic_privacy()
callback is provided.

Platform code can call sof_ipc4_mic_privacy_state_change() to send the IPC
message to the firmware on state change.

This series will change how we set up dsp_ops for ACE versions to make it cleaner what needs to be handled differently in each iteration of the architecture and adds the needed definitions and code to handle the mic privacy via vendor specific SHIM register.

bardliao · 2025-02-03T13:19:05Z

include/sound/hda-mlink.h

+void hdac_bus_eml_set_mic_privacy_mask(struct hdac_bus *bus, bool alt, int elid,
+				       unsigned long mask);
+bool hdac_bus_eml_check_mic_privacy(struct hdac_bus *bus, bool alt, int elid);
+bool hdac_bus_eml_get_mic_privacy(struct hdac_bus *bus, bool alt, int elid);


What is the difference between check and get?

Ok, I got it after reading all the commits. Check if it is a mic privacy irq and get the mic privacy status.

Had to go all the way the register spec to get that, but then I did understand that too. A short comment somewhere would not hurt thou.

OK, let me do some renaming and commenting.

sound/soc/sof/intel/ptl.c

jsarha

Only couple of not so important suggestions. LGTM.

jsarha · 2025-02-06T09:42:53Z

sound/soc/sof/intel/lnl.c

 {
 	struct hdac_bus *bus = sof_to_bus(sdev);

 	return hdac_bus_eml_check_interrupt(bus, true,  AZX_REG_ML_LEPTR_ID_SDW);
 }
+EXPORT_SYMBOL_NS(lnl_dsp_check_sdw_irq, "SND_SOC_SOF_INTEL_LNL");


Would it be worth mentioning in the commit message that it also unexports the few lnl specific ops and turns them into static?

It only changes the lnl functions that are needed for ptl from static to be exported, which is a must to create the separate ptl.c

jsarha · 2025-02-06T17:15:00Z

include/sound/hda-mlink.h

+void hdac_bus_eml_set_mic_privacy_mask(struct hdac_bus *bus, bool alt, int elid,
+				       unsigned long mask);
+bool hdac_bus_eml_check_mic_privacy(struct hdac_bus *bus, bool alt, int elid);
+bool hdac_bus_eml_get_mic_privacy(struct hdac_bus *bus, bool alt, int elid);


Had to go all the way the register spec to get that, but then I did understand that too. A short comment somewhere would not hurt thou.

ujfalusi · 2025-02-07T09:24:02Z

Changes since v1:

rename hdac_bus_eml_check_mic_privacy() to hdac_bus_eml_is_mic_privacy_changed()
rename hdac_bus_eml_get_mic_privacy() to hdac_bus_eml_get_mic_privacy_state()

kv2019i

On question inline, but no blockers.

sound/soc/sof/intel/ptl.c

ujfalusi · 2025-02-07T11:34:07Z

SOFCI TEST

plbossart · 2025-02-11T19:10:17Z

include/sound/sof/ipc4/header.h

@@ -396,6 +396,7 @@ enum sof_ipc4_base_fw_params {
 	SOF_IPC4_FW_PARAM_MODULES_INFO_GET,
 	SOF_IPC4_FW_PARAM_LIBRARIES_INFO_GET = 16,
 	SOF_IPC4_FW_PARAM_SYSTEM_TIME = 20,
+	SOF_IPC4_FW_PARAM_MIC_PRIVACY_STATE_CHANGE = 35,


This is very controversial.
In the case of DMICs, the mute can be completely handled by firmware since it also processes the GPIO that toggles the privacy.
In the case of SoundWire, the mute is expected to be handled in the external codec.

In other words, this IPC does not seem to have a purpose other than academic - and it's mind-blowing that such a split implementation for a 'secure' solution relies on an IPC from the host. Recompile your kernel and the privacy is crippled... That does not sound good, does it.

@plbossart, this is not the codec implemented privacy (SPE), I think SDCA spec also have similar described for the Function agent (iow, DSP).
The muting is controlled by hardware completely: it follows the mic privacy GPIO state, it is muting the incoming stream regardless what the firmware or Linux does.
But this muting would be 'rude'. We can request interrupt via audio IPs to their owner: DMIC is owned by firmware, so the interrupt goes there, SDW is owned by Linux, so the interrupt goes to Linux.
To provide better user experience, the FW is expected to do a fade on MUTE/UNMUTE (but regardless of this, the input stream is going to be muted).
FW can handle the DMIC case, but it needs a notification from Linux when SDW is used to be able to perform the fade.
If we don't send the notification to firmware the only thing that happens is that the mute is going to be immediate which is not too nice.

plbossart · 2025-02-11T19:13:01Z

include/sound/hda-mlink.h

+				       unsigned long mask);
+bool hdac_bus_eml_is_mic_privacy_changed(struct hdac_bus *bus, bool alt, int elid);
+bool hdac_bus_eml_get_mic_privacy_state(struct hdac_bus *bus, bool alt, int elid);
+


I think the commit message should be clearer that the information is from hardware to host only. This is different to the previous patch in the sense that the host can select to receive information on the privacy changes, but it is not involved in the privacy flows.

The interrupt is to the owner of the IP, if we would have DMIC handling in kernel side then the interrupt would be coming to host.
I made the API generic to avoid churn (with sdw/dmic/ssp/hda/this/that/ variants) if ever we will have to deal with this on different IPs in kernel.

Let me think how I can improve the commit message.

ujfalusi · 2025-02-13T13:36:56Z

Changes since v2:

the commit message for the hda-mlink patch rewritten

ujfalusi · 2025-02-26T08:45:28Z

Changes since v3:

dropped the hack/debugging patch

kv2019i

A few trivial things, but code looks good and commit messages are decriptive.

kv2019i · 2025-02-26T10:47:26Z

sound/soc/sof/intel/ptl.c

@@ -0,0 +1,52 @@
+// SPDX-License-Identifier: (GPL-2.0-only OR BSD-3-Clause)
+//
+// Copyright(c) 2023 Intel Corporation


sound/soc/sof/intel/hda.c

ujfalusi · 2025-02-26T12:14:37Z

Changes since v4:

Date correct for ptl.c
grammar fix in commit message

ranj063 · 2025-02-26T17:00:58Z

include/sound/hda-mlink.h

+static inline bool
+hdac_bus_eml_is_mic_privacy_changed(struct hdac_bus *bus, bool alt, int elid)
+{
+	return false


ranj063 · 2025-02-26T17:03:42Z

sound/soc/sof/intel/hda.c

 		hda_dsp_sdw_thread(irq, hdev->sdw);
+
+		if (hda_dsp_sdw_check_mic_privacy_irq(sdev)) {


Does this have to inside the if (hda_dsp_check_sdw_irq(sdev)) @ujfalusi or is it independent of the sdw_irq like the wakeen irq?

The mic privacy interrupt in PTL is piggy-backing on the SDW interrupt for host (and on DMIC interrupt in DSP), if the SDW is not powered up we will not receive interrupt about the change. Even if we have SDW active, but the link which is under HW managed mic privacy mode is not active we will not receive interrupt (we only enable the interrupt via the managed link).

ujfalusi · 2025-02-27T08:47:27Z

Changes since v5:

missed ; is added to hda-mlink.h

lgirdwood

Just one question

include/sound/hda-mlink.h

ujfalusi · 2025-02-27T11:34:30Z

Changes since v6:

moved the ; fix to the correct patch (one up)

Move the sof_mtl_ops and sof_mtl_ops_init() to pci-mtl.c as local static and add a 'generic' sof_mtl_set_ops() function as replacement exported function to fill the dsp_ops structure. Signed-off-by: Peter Ujfalusi <[email protected]>

LunarLake is a next generation in ACE architecture and most of the dsp_ops are the same as it is in previous generation. Use the sof_mtl_set_ops() to get the ops used for mtl and update the ones that needs different functions for LNL. Update pci-ptl at the same time to use the LNL dsp_ops as before. Signed-off-by: Peter Ujfalusi <[email protected]>

There is no need to export individual dsp_ops functions anymore as the callbacks are filled now by sof_mtl_set_ops() Signed-off-by: Peter Ujfalusi <[email protected]>

Create a minimal placeholder to make it possible to add code to handle the new features of Panther Lake compared to MTL/LNL. Signed-off-by: Peter Ujfalusi <[email protected]>

ACE3 (Panther Lake) introduced support for microphone privacy feature which can - in hardware - mute incoming audio data based on a state of a physical switch. The change in the privacy state is delivered through interface IP blocks and can only be handled by the link owner. In Intel platforms Soundwire is for example host owned, so the interrupt can only be handled by the host. Since the input stream is going to be muted by hardware, the host needs to send a message to firmware about the change in privacy so it can execute a fade out/in to enhance user experience. The support for microphone privacy can be queried from the HW_CONFIG data under the INTEL_MIC_PRIVACY_CAP tuple. This is Intel specific data, the core will pass it to platform code if the intel_configure_mic_privacy() callback is provided. Platform code can call sof_ipc4_mic_privacy_state_change() to send the IPC message to the firmware on state change. Signed-off-by: Peter Ujfalusi <[email protected]>

…egisters New register has been introduced with PTL in the vendor specific SHIM registers, outside of the IPs itself for microphone privacy status handling. Via the PVCCS register the current microphone privacy status can be checked and the interrupt generation on status change can be enabled/disabled. The status change interrupt is routed to the owner of the interface (DSP/host). The PVCCS is provided for each sublink under the IP to make it possible to control the interrupt generation per sublink. On status change the MDSTSCHG bit needs to be cleared for all sublink of the interface to be able to detect future changes in privacy. The status bit (MDSTS) is volatile in all PVCCS register, it reflects the current state of the GPIO signal. Microphone privacy is a hardware feature (if enabled and configured that way), the host has only passive, monitoring role. The added functions are generic to be future proof if the mic privacy support is extended beyond Soundwire and DMIC links. Signed-off-by: Peter Ujfalusi <[email protected]>

Add generic callback definitions for checking the mic privacy interrupt and status. Implement wrappers for mic privacy reported via the Soundwire interrupt and its vendor specific SHIM registers. Signed-off-by: Peter Ujfalusi <[email protected]>

Implement the three callbacks that is needed to enable support for reporting the mic privacy change via soundwire. In PTL the mic privacy reporting is supported via soundwire and DMIC and the soundwire is owned by the host, it's interrupt is routed there. To enable the interrupt, the sublink mask needs to be passed to the multilink layer, the check_mic_privacy_irq/process_mic_privacy callbacks needs to be implemented to check and report the mic privacy change. Signed-off-by: Peter Ujfalusi <[email protected]>

ujfalusi requested review from jsarha, abonislawski and mbukowsk January 29, 2025 12:33

ujfalusi requested review from bardliao, lgirdwood, plbossart, ranj063, kv2019i and dbaluta as code owners January 29, 2025 12:33

bardliao reviewed Feb 3, 2025

View reviewed changes

sound/soc/sof/intel/ptl.c Show resolved Hide resolved

jsarha approved these changes Feb 6, 2025

View reviewed changes

ujfalusi changed the title ~~ASoC: SOF: Intel: Add support for hardware mic privacy change reproting~~ ASoC: SOF: Intel: Add support for hardware mic privacy change reporting Feb 7, 2025

ujfalusi force-pushed the peter/sof/pr/ptl_sdw_mic_privacy_handling_01 branch from 3ff7eef to 85814f7 Compare February 7, 2025 09:22

ujfalusi requested a review from lyakh as a code owner February 7, 2025 09:22

kv2019i previously approved these changes Feb 7, 2025

View reviewed changes

sound/soc/sof/intel/ptl.c Show resolved Hide resolved

bardliao previously approved these changes Feb 7, 2025

View reviewed changes

plbossart reviewed Feb 11, 2025

View reviewed changes

ujfalusi dismissed stale reviews from bardliao and kv2019i via a6d41fb February 13, 2025 13:35

ujfalusi force-pushed the peter/sof/pr/ptl_sdw_mic_privacy_handling_01 branch from 85814f7 to a6d41fb Compare February 13, 2025 13:35

lgirdwood previously approved these changes Feb 13, 2025

View reviewed changes

bardliao previously approved these changes Feb 26, 2025

View reviewed changes

ujfalusi dismissed stale reviews from bardliao and lgirdwood via 94818a3 February 26, 2025 08:45

ujfalusi force-pushed the peter/sof/pr/ptl_sdw_mic_privacy_handling_01 branch from a6d41fb to 94818a3 Compare February 26, 2025 08:45

bardliao previously approved these changes Feb 26, 2025

View reviewed changes

kv2019i previously approved these changes Feb 26, 2025

View reviewed changes

ujfalusi dismissed stale reviews from kv2019i and bardliao via 869b728 February 26, 2025 12:13

ujfalusi force-pushed the peter/sof/pr/ptl_sdw_mic_privacy_handling_01 branch from 94818a3 to 869b728 Compare February 26, 2025 12:13

bardliao previously approved these changes Feb 26, 2025

View reviewed changes

ranj063 reviewed Feb 26, 2025

View reviewed changes

ujfalusi dismissed bardliao’s stale review via 728d727 February 27, 2025 08:46

ujfalusi force-pushed the peter/sof/pr/ptl_sdw_mic_privacy_handling_01 branch from 869b728 to 728d727 Compare February 27, 2025 08:46

lgirdwood reviewed Feb 27, 2025

View reviewed changes

include/sound/hda-mlink.h Outdated Show resolved Hide resolved

ujfalusi force-pushed the peter/sof/pr/ptl_sdw_mic_privacy_handling_01 branch from 728d727 to cf0443d Compare February 27, 2025 11:34

ujfalusi added 8 commits February 27, 2025 13:34

ASoC: SOF: Intel: mtl: Split up dsp_ops setup code

6178620

Move the sof_mtl_ops and sof_mtl_ops_init() to pci-mtl.c as local static and add a 'generic' sof_mtl_set_ops() function as replacement exported function to fill the dsp_ops structure. Signed-off-by: Peter Ujfalusi <[email protected]>

ASoC: SOF: Intel: mtl: Stop exporting dsp_ops callback functions

3d5ff5b

There is no need to export individual dsp_ops functions anymore as the callbacks are filled now by sof_mtl_set_ops() Signed-off-by: Peter Ujfalusi <[email protected]>

ASoC: SOF: Intel: Create ptl.c as placeholder for Panther Lake features

54f085a

Create a minimal placeholder to make it possible to add code to handle the new features of Panther Lake compared to MTL/LNL. Signed-off-by: Peter Ujfalusi <[email protected]>

ranj063 approved these changes Feb 27, 2025

View reviewed changes

lgirdwood approved these changes Feb 27, 2025

View reviewed changes

kv2019i approved these changes Feb 28, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ASoC: SOF: Intel: Add support for hardware mic privacy change reporting #5312

ASoC: SOF: Intel: Add support for hardware mic privacy change reporting #5312

ujfalusi commented Jan 29, 2025

bardliao Feb 3, 2025

bardliao Feb 3, 2025

jsarha Feb 6, 2025

ujfalusi Feb 7, 2025

jsarha left a comment

jsarha Feb 6, 2025

ujfalusi Feb 7, 2025

jsarha Feb 6, 2025

ujfalusi commented Feb 7, 2025

kv2019i left a comment

ujfalusi commented Feb 7, 2025

plbossart Feb 11, 2025

ujfalusi Feb 13, 2025

plbossart Feb 11, 2025

ujfalusi Feb 13, 2025

ujfalusi commented Feb 13, 2025

ujfalusi commented Feb 26, 2025

kv2019i left a comment

kv2019i Feb 26, 2025

ujfalusi commented Feb 26, 2025

ranj063 Feb 26, 2025

ujfalusi Feb 27, 2025

ranj063 Feb 26, 2025

ujfalusi Feb 27, 2025

ujfalusi commented Feb 27, 2025

lgirdwood left a comment

ujfalusi commented Feb 27, 2025

		hda_dsp_sdw_thread(irq, hdev->sdw);

		if (hda_dsp_sdw_check_mic_privacy_irq(sdev)) {

ASoC: SOF: Intel: Add support for hardware mic privacy change reporting #5312

Are you sure you want to change the base?

ASoC: SOF: Intel: Add support for hardware mic privacy change reporting #5312

Conversation

ujfalusi commented Jan 29, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jsarha left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ujfalusi commented Feb 7, 2025

kv2019i left a comment

Choose a reason for hiding this comment

ujfalusi commented Feb 7, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ujfalusi commented Feb 13, 2025

ujfalusi commented Feb 26, 2025

kv2019i left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ujfalusi commented Feb 26, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ujfalusi commented Feb 27, 2025

lgirdwood left a comment

Choose a reason for hiding this comment

ujfalusi commented Feb 27, 2025