[Bug]: Sudden peaks appearing in the measured data on Solax X3 G4

[shaarkys]

Describe the bug

Hi,

sudden anomalous peaks are appearing in the Solax integration data, most probably when inverted stops responding for a while. These spikes are not observed in other platforms like Homey (same sampling rate) or the Solax Cloud. The setup uses Modbus TCP via a Multiplexer and had been stable for months prior to the upgrades.
I thought it was caused by V3 Wifi+LAN dongle upgrade that was done remotely, which caused inverter to lock several times during the day, but after reverting it, the Lock issues are gone but the peaks remains.

While I have to clarify why modbus disconnects (visible on Homey logs) suddenly for a moment, is there some possibility to avoid such peaks ? Apologies if this would be some kind of HW failure on my side but trying to rule out everything.

Integration Version

2025.01.8 (testing now 2025.01.6 to see any difference)

Homeassistant core version

2025.1.4

Inverter brand

Solax X3 G4

Plugin used

plugin_solax.py

Serial prefix

H34A10

Inverter firmware versions

DSP v1.46 ARM v1.44

Connection Method

Solax Wifi+LAN adapter V3 with fw via LAN

Dongle firmware

1.003.11 (reverted as troubleshooting steps from 1.005.04 )

Detailed Error Log

This error originated from a custom integration.

Logger: pymodbus.logging
Source: custom_components/solax_modbus/__init__.py:771
integration: SolaX Inverter Modbus (documentation, issues)
First occurred: January 27, 2025 at 20:04:19 (21864 occurrences)
Last logged: 11:25:39

BinaryPayloadDecoder is deprecated and will be removed in v3.9.0 ! Please use "client.convert_from_registers()" or "client.convert_to_registers" See documentation: "https://pymodbus.readthedocs.io/en/latest/source/client.html#pymodbus.client.mixin.ModbusClientMixin.convert_from_registers"

This error originated from a custom integration.

Logger: custom_components.solax_modbus
Source: custom_components/solax_modbus/__init__.py:670
integration: SolaX Inverter Modbus (documentation, issues)
First occurred: January 27, 2025 at 21:51:40 (6 occurrences)
Last logged: 10:59:20

Something went wrong reading from modbus
Traceback (most recent call last):
  File "/config/custom_components/solax_modbus/__init__.py", line 670, in async_read_modbus_data
    res = await self.async_read_modbus_registers_all(group)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/solax_modbus/__init__.py", line 848, in async_read_modbus_registers_all
    data[descr.key] = descr.value_function(0, descr, data)
                      ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^
  File "/config/custom_components/solax_modbus/plugin_solax.py", line 300, in value_function_battery_voltage_cell_difference
    return datadict.get("cell_voltage_high", 0) - datadict.get("cell_voltage_low", 0)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
TypeError: unsupported operand type(s) for -: 'NoneType' and 'NoneType'

This error originated from a custom integration.

Logger: custom_components.solax_modbus
Source: custom_components/solax_modbus/__init__.py:716
integration: SolaX Inverter Modbus (documentation, issues)
First occurred: January 27, 2025 at 21:51:39 (248 occurrences)
Last logged: 10:59:20

Solax X3 G4: read failed at 0xbd: cell_voltage_low
Solax X3 G4: read failed at 0xbf: battery_soh
Solax X3 G4: read failed at 0xc8: grid_frequency
Solax X3 G4: read failed at 0xc9: grid_voltage
Solax X3 G4: read failed at 0xca: grid_voltage_l1

This error originated from a custom integration.

Logger: custom_components.solax_modbus
Source: custom_components/solax_modbus/__init__.py:670
integration: SolaX Inverter Modbus (documentation, issues)
First occurred: 08:50:12 (4 occurrences)
Last logged: 08:56:03

Something went wrong reading from modbus
Traceback (most recent call last):
  File "/config/custom_components/solax_modbus/__init__.py", line 658, in async_write_registers_multi
    resp = await self._client.write_registers(address=address, values=payload, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.13/site-packages/pymodbus/transaction/transaction.py", line 155, in execute
    raise ModbusIOException(txt)
pymodbus.exceptions.ModbusIOException: Modbus Error: [Input/Output] No response received after 6 retries, continue with next request

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/config/custom_components/solax_modbus/__init__.py", line 670, in async_read_modbus_data
    res = await self.async_read_modbus_registers_all(group)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/solax_modbus/__init__.py", line 874, in async_read_modbus_registers_all
    await self.async_write_registers_multi(
    ...<3 lines>...
    )
  File "/config/custom_components/solax_modbus/__init__.py", line 661, in async_write_registers_multi
    raise HomeAssistantError(f"Error writing multiple Modbus registers: {original_message}") from e
homeassistant.exceptions.HomeAssistantError: Error writing multiple Modbus registers: Modbus Error: [Input/Output] No response received after 6 retries, continue with next request

Additional context

Note - around 9AM I upgraded firmware to DSP v1.46 ARM v1.44 as another troubleshooting step.

[shaarkys]

Seems that 2025.01.6 is not showing such sudden peaks, despite of short/random disconnections (at least based on this short test).

Just wonder, is it due to the recent Python related changes and will this change maybe address it ?
048ac75

[Shortykiller]

Hi

Just want to state that I to have the same problem with random spikes. Im new to the integration, so have no history to cross check with.

My Solar System is Growatt, running Modbus/TCP custom esp32, directly on the modbus com port.

[thkn1973]

I should check github more often as soon as errors occur.
I have also had these spikes since then.

[matejkaf]

Anyone tried latest release 2025.01.9? Spike Problem solved? At the moment I am stuck at 2025.01.6

[Shortykiller]

Anyone tried latest release 2025.01.9? Spike Problem solved? At the moment I am stuck at 2025.01.6

No, not yet.. I downgraded to 2025.01.6 yesterdays to check if that solved it temporary..

[shaarkys]

Anyone tried latest release 2025.01.9

I did, running it almost full day, it looks much better... but still some smaller peaks appears

But finally no crazy reading in energy ... !

[shaarkys]

Hmm, so reverting back ;-(

[shaarkys]

[Zwer2k]

Anyone tried latest release 2025.01.9? Spike Problem solved? At the moment I am stuck at 2025.01.6

No. Complety destroys my Statistics. Back to 2025.01.6 . Inverter Sofar HYD10-KTL-3ph

[shaarkys]

Just use /developer-tools/statistics and remove faulty values - it's better to do it the same day it happens, it's faster then. ;-)

[Zwer2k]

Thank you. I know the method. Unfortunately it is a bit time-consuming.

[shaarkys]

So far 2025.01.6 is really working fine for me.

@wills106 , any thoughts on that please ?

[wills106]

Integration 2025.01.6 is using pyModbus 3.7.4
Integration 2025.01.8 and above is using pyModbus 3.8.3 to match HA.

The developer of pyModbus is striving to make pyModbus more Modbus compliant, which many Data Loggers such as the SolaX PocketWiFi / PocketWiFi + LAN and the Sofar LSE-3 aren't.

With the Sofar LSE-3 the latest firmware ME_0D_270A_1.11 made the Data Logger even more uncompliant, people are recommended to downgrade to ME_0D_270A_1.09. But even then they are still getting issues.

I will be updating the documentation to recommend people to use a USB-RS485 or an Ethernet-RS485 device such as the Waveshare as these are less problematic, instead of these Data Loggers.

It is possible that improvements could be made to the Integration using pyModbus 3.8.3 to improve things, but I own a single Inverter with built in Modbus over TCP. So I can't actively improve support for data loggers that aren't fully Modbus compliant, as I don't own one nor can I use one on my Inverter.

[shaarkys]

Thank you @wills106 , I thought it was related to pyModbus ...however in my case, I use Wifi V3 LAN+Wifi dongle, accessing it via TCP Multiplexer on Solax X3 G4, worked very much fine last year. So not sure what do you mean with Data Loggers ?
Also I'm quite sure those peaks just occurs when dongle temporarily refuse to provide data (yet still being accessible).

In my case, I can't use WaveShare RS485, haven been this route (even created some tickets some time ago) and never worked for me reliably, not sure why, but also was getting peaks. One of the reason could be that Wifi + RS485 basically creates two devices accessing the Modbus, not sure

[matomatusov]

Hi @wills106 ,
don't you know why my pymodbus version in HA changes every now and then?

Thanks

[Zwer2k]

I use WaveShare RS485 and have the peaks that did not exist until version 2025.01.6. In my case, the data is read with two end devices (Solax Modbus + EVCC) via Modbus proxy. I have seen that there have been no changes to the integration that could cause these spikes, it must actually be due to the new pyModbus version.
This is annoying, especially since I've been struggling with pyModbus compatibility issues for two months now. Solax forces new versions of pyModbus all the time, which in turn leads to problems with other integrations that also use pyModbus. In my case Huawei-Solar. I keep having to switch to older Ha-Core, Huawei and Solax versions.
I don't think it's a good idea to use versions of libraries that are not yet included in HA-Core.

[wills106]

pyModbus 3.8.3 is in HA core, it's been in there for about a month now.

home-assistant/core#134809

I try and keep the Integration matched with HA, as some users also use the HA Modbus client directly for Air source Heat Pumps etc.

[shaarkys]

...however in my case, I use Wifi V3 LAN+Wifi dongle, accessing it via TCP Multiplexer on Solax X3 G4, worked very much fine last year. So not sure what do you mean with Data Loggers ?
Also I'm quite sure those peaks just occurs when dongle temporarily refuse to provide data (yet still being accessible).

@wills106 and please your response to peaks on Solax / with Wifi Dongle V3 LAN+Wifi ? Those peaks were not there so it must be something with the library itself. I checked changelog, haven't seen anything obvious and also no "bug" planned to be fixed in the next version, but I'm not expert ;-( I hope it will not result into "dongle unsupported / not recommended" anymore ;-)

[Zwer2k]

Yesterday I connected the integration directly to the WaveShare RS485 adapter (without Modbus proxy) and since then I have had no more spikes. EVCC continues to run via Modbus proxy.
The problem only seems to occur in association with Modbus proxy. Version of Modbus proxy is still the same as before the problem, I have not changed anything

[shaarkys]

Integration 2025.01.6 is using pyModbus 3.7.4
Integration 2025.01.8 and above is using pyModbus 3.8.3 to match HA.

Hi @wills106 , FYI - I can confirm that combination of pyModbus 3.8.3 with Solax X3 G4 connected via WLAN V3 dongle LAN+Wifi do not work well. Since I switched over to 2025.01.6 (back to pyModbus 3.7.4 ), the integration works perfectly like whole previous time. Since we know that the cause is pyModbus 3.8.3, is there anything else I can do to help troubleshoot this ?

Perfect "curve" on 2025.01.6 (pyModbus 3.7.4)

The changelog is pretty extensive since 3.7.4 but what could be causing this behavior ?
https://pymodbus.readthedocs.io/en/v3.8.3/source/changelog.html

And why there are not yet multiple reports like this one everywhere ? ...rhetorical question I'm asking myself.

Asking "AI advisor", this is the view (I don't know if it's reasonable or not)

The most likely suspect is the change introduced in 3.8.1 that causes the async client to raise an exception on no response (see #2502). This change means that if the client stops responding momentarily, it now throws an exception rather than failing silently. That could disrupt the normal flow—triggering reconnection or fallback handling—and lead to those measurement peaks you’re seeing.

Additionally, changes like the re-instantiation of Futures on reconnect (#2501) might contribute to delays or unexpected behavior if the error handling isn’t adjusted accordingly. However, the explicit raising of exceptions on no response is the primary change to check.

[matomatusov]

Hi,
It happens to me that some registries fall out. In the past, everything was OK.

[darkrain-nl]

Just want to share I see peaks as well with plugin_sofar...

[steps56]

Me too a few rare times at SolaX.

[HA-Supi]

I can also confirm this behavior with Qcell's Hyp-G3 (Solax X3 G4).
Data retrieval via Modbus TCP / Stick 3.0 FW 19.01 / Inverter FW 48 08 45

Please take a look at the following information on troubleshooting > @wills106 :
I switched from the 2024.05.3 integration to 2025.01.2 in January 2025. With old Integration I used a tile with FW info, the removal of the entity bothered me. ... but because of HA Updates... I switched to a recent Addon Version.
Previously, the integration worked for 6 months without any problems.

I suspect that there are data errors during retrieval that are not recognized. (no data checksum?)

I thought it might be due to the change of operating mode (inverter check mode, last picture), but why would the battery data be going so crazy at that moment?

Here are a couple of screenshots:
They do not show all entities with errors... only a selection.

Edit: Debug Log attached:
2025-02-09 074229.024 debug_log.txt

[steps56]

Yes, I got same Errors at my system.
Newest Modbus Integration, newest HA core and supervisor and Qcells/SolaX brandnew Firmware 1.50/1.47 from this week and latest Lan/Wlan Dongle too.

The spikes must come from Integrantion blackouts, as Qcells/SolaX App History doesn't show any spikes ever.

After each new start of HA, the pyModbus depricates error shows up once too !

Using Backup Mode during winter avoids inverter idle... and no heating if ideling.
Setting FVRT to on (instead off by default) gives more Grid tollerance to grid up and downs, avoiding even more errors at Qcells/SolaX System also.

[shaarkys]

@wills106 , you probably know already but seems that there are some changes in PyModbus 3.8.4, 3.8.5 and 3.8.6, which could help

https://pymodbus.readthedocs.io/en/latest/source/changelog.html

[godfreym29]

This issue is also causing issues with the inverter charge and discharge time on Solis.
Most of the times the entity values are to high to do anything, but sometimes they are within the 0-23 for hours or 0-59 for minutes and can cause the inverter to discharge the battery, on occasions emptying the battery.

[steps56]

I went back to 2025.01.6
No spikes... and hopefully wills106 may find the reason why newest Version pruduces those spikes.... and may find a solution.
Hope for solution update soon by 2025.02....

[godfreym29]

Just to confirm
Downgrading to 2025.01.6 has resolved the issue.
6 hours without any peaks or issues with the inverter charge and discharge time on Solis.

[steps56]

@wills106
Any news from your side about working at the spikes problem and to find the reason and a solution ?
Downgrading and to stuck at 2025.01.6 don't look like a permanent solution to me.
I love your work/integration, but communikation at github generally lacks ... its always guessing about "work in progress"... or sadly may be ignored at all 🤷🏼‍♂️.
Bye for now and please answer about this, thanks in advance.

[shaarkys]

@steps56 I'm sure wills106 will provide some solution, when / if it will be ready, Probably currently busy with different priorities but asking repeatedly for updates is not helping anyway. Let's be patient here.

[steps56]

@steps56 I'm sure wills106 will provide some solution, when / if it will be ready, Probably currently busy with different priorities but asking repeatedly for updates is not helping anyway. Let's be patient here.

Asking just for any Info by communication, just to know about... and not always only asume, guess... to know just nothing about "work in progress".
(Thats what I hate most concerning Github)
Communication is always not any miracle, but AVOIDING any (un)necessary asking.
Not that hard to understand, isn't it ?

[shaarkys]

Yep, not also hard to understand nobody was asking for comment nor reply ;-) As all those who commented here are also automatically receiving notifications, it's something which is not welcome either.

But don't worry, just avoiding developer to be, unintentionally, mad. It's quite hard for any developer to read any "demands" when the developer is working on it during his free time or almost for free.

That's last reply from my side, we have enough proof it's caused by the library used, also proved temporary workaround, so currently there is nothing to be hurried about.

[wills106]

I have noticed in this comment

There is mention about increasing the timeout to 9.
Could someone who's having this issue try and see if 9 or 10 makes an improvement?

homeassistant-solax-modbus/custom_components/solax_modbus/__init__.py

Lines 261 to 262 in c889aa7

	if tcp_type == "rtu":
	self._client = AsyncModbusTcpClient(host=host, port=port, timeout=5, framer=FramerType.RTU, retries=6)

I can't find any reference to what message_wait_milliseconds: does in the HA Integration as it doesn't seem to link to anything in the pyModbus Library.

[darkrain-nl]

I have set it to 9 and restarted HA, let's wait and see :)

[steps56]

I now installed last Integration version .9 again and set the timout to 10 at init.py line 262 as mentioned by wills106 before.
Hope it will avoid the spikes... I watch it and will tell here anyway.
TANKS A LOT for investigating this problem to @wills106 and all others involved. 👌

I also set in the past already config settings like shown at the screen here:

[steps56]

Update/Edit:
As I'm using Modbus Tcpip (SolaX newest Lan/Wlan Dongle using Wlan connection), I set the 2 other timeout settings also at now all, to 15 Sec. timeout.

I assume, @wills106 hint war "just" for the "waveshare" users and not Tcpip users as myself.
Tell me please, if I'm wrong about this and/or about the 15 Sec. setting for safety.

See screen about my edited lines concerning timeout:

[wills106]

I thought it was mostly the PocketWiFi that was playing up?
But it's worth trying a change or 10 or whatever to see if it improves it or not for anyone getting these spikes.

I'm not getting any of these spikes, so there isn't much I can do locally.

[steps56]

I thought it was mostly the PocketWiFi that was playing up? But it's worth trying a change or 10 or whatever to see if it improves it or not for anyone getting these spikes.

I'm not getting any of these spikes, so there isn't much I can do locally.

Yes, its the PocketWifi about the spikes.

Your change about timeout line 262 looks as "just" concerning the rtu / Waveshare connection only.

Modbus via tcpip = Pocketwifi Dongle may be connected to the next lines 265 and 266, if I'm not wrong.
So I assumed to change timeout there too.
Wrong or right ???🤷🏼‍♂️.

Thanks about any infos and again for your really important work 👍.

At the moment no errors and no spikes.
Hopefully the next days too, I will tell about it anyway. 🙋🏼‍♂️

[Shortykiller]

I thought it was mostly the PocketWiFi that was playing up? But it's worth trying a change or 10 or whatever to see if it improves it or not for anyone getting these spikes.

I'm not getting any of these spikes, so there isn't much I can do locally.

Hi

I just upgraded again now, and will try and change this update timeout to 10 from the 5 it has standard. I will report back here later (some days later if nothing, or as soon as I still se a spike)

Best regartz Shorty

[Thrasher2020]

Upgraded and modified to timeout=10

Will feed back after a few days (unless it spikes soon!)

[steps56]

Just my "2 Cents" additional:
If anybody is using the "Modbus Proxy Addon" (and I assume, many will necessarily do like me), I set timeout to 15 Sec at -init_.py to avoid collisions (?) to Proxy's 10 Sek. intervall (screen shows my settings):