Monitoring Bluetti Systems

[MrGPS]

I am running the Home Assistant natively on an Intel NUC which has got a Bluetooth built in.
Is there a way to run the Bluetti_MQTT also natively on that Intel NUC to have all in one Box ?
If not on which hardware/software environment are you running it ? If you use an USB-Stick for Bluetooth, what model ?
BTW: i have a Bluetti EP500 Pro which has 2 DC Inputs, DC1 and DC2 but the Bluetti App only displays the summary as PV-Input.
Only on the LCD Display I can view the 2 different DC Inputs.
Since I have 2 different PV Strings attached to the DC inputs it would be interesting to have a separate history.
Did you see in your traces also Values from multiple DC inputs ? DC1 and DC2 ?

 

[MrGPS]

I just updated my Raspberry PI to the latest software and then tried to install the package, but if I understand it right it does not support my python 3.7.3 ?
Or is something else missing on the PI ?

pi@raspberrypi:~ $ python3 --version
Python 3.7.3
pi@raspberrypi:~ $ pip install bluetti_mqtt
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Collecting bluetti_mqtt
Downloading https://files.pythonhosted.org/pack...3d4663b95e59c6aa0572c/bluetti_mqtt-0.1.tar.gz
Collecting bleak (from bluetti_mqtt)
Downloading https://files.pythonhosted.org/pack...e618d8dfa40/bleak-0.15.1-py2.py3-none-any.whl (123kB)
100% |████████████████████████████████| 133kB 1.1MB/s
Collecting crcmod (from bluetti_mqtt)
Downloading https://files.pythonhosted.org/pack...18e0b7f6826a043ca1245dd4e5b/crcmod-1.7.tar.gz (89kB)
100% |████████████████████████████████| 92kB 1.1MB/s
Collecting typing-extensions>=4.2.0 (from bleak->bluetti_mqtt)
Could not find a version that satisfies the requirement typing-extensions>=4.2.0 (from bleak->bluetti_mqtt) (from versions: 3.6.2, 3.6.2.1, 3.6.5, 3.6.6, 3.7.2, 3.7.4, 3.7.4.1, 3.7.4.2, 3.7.4.3, 3.10.0.0, 3.10.0.1, 3.10.0.2)
No matching distribution found for typing-extensions>=4.2.0 (from bleak->bluetti_mqtt)

 

[chromedshark]

I am currently trying to receive the incoming watts from the DC input via ESP32

@Tinkerpete Normally with a BLE device you can just read a specific characteristic to get the data you want, but the Bluetti is unfortunately more complicated than that.

There are two characteristics I've been using to communicate with the device. Commands are written to the write characteristic "0000ff02-0000-1000-8000-00805f9b34fb", and the response is received as a notification on the notify characteristic '0000ff01-0000-1000-8000-00805f9b34fb' (both of these can be found in client.py).

The basic algorithm for collecting the information you want is to register for notifications on the notify characteristic, build the command payload to request the data you want, write that to the write characteristic, and then wait until you receive the data you asked for (or a myriad things go wrong and you have to retry sending the command). If all you want is the "dc_input_power", you should be able to use QueryRangeCommand(0x00, 0x24, 1) to get what you want, and because this payload response should fit in a single packet you wouldn't have to deal with re-assembly.

 

[chromedshark]

if I understand it right it does not support my python 3.7.3 ?
Or is something else missing on the PI

I'm still a bit of a novice with Python, but I think if you run pip install 'bleak==0.14.3' it'll install a version of Bleak that is compatible with Python 3.7. Once that's done you should be able to run pip install bluetti_mqtt?

 

[chromedshark]

Since I have 2 different PV Strings attached to the DC inputs it would be interesting to have a separate history.
Did you see in your traces also Values from multiple DC inputs ? DC1 and DC2 ?

As far as I can tell only DC1 details are available over bluetooth. In theory there might be some way to switch it from reporting DC1 to DC2 to get both, but I haven't been able to figure out how to do that.

 

[MrGPS]

I'm still a bit of a novice with Python, but I think if you run pip install 'bleak==0.14.3' it'll install a version of Bleak that is compatible with Python 3.7. Once that's done you should be able to run pip install bluetti_mqtt?

Did not work:
pip install 'bleak==0.14.3'
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Collecting bleak==0.14.3
Downloading https://files.pythonhosted.org/pack...31bfa9447f5/bleak-0.14.3-py2.py3-none-any.whl (113kB)
100% |████████████████████████████████| 122kB 1.2MB/s
Collecting dbus-next; platform_system == "Linux" (from bleak==0.14.3)
Could not find a version that satisfies the requirement dbus-next; platform_system == "Linux" (from bleak==0.14.3) (from versions: )
No matching distribution found for dbus-next; platform_system == "Linux" (from bleak==0.14.3)

But I think the error-message is regarding "typing-extensions", so I tried to install the Version from the error message (If I don't specify a version, then typing-extensions-3.10.0.2 is installed and that still doesn't allow to install the Bluetti Matt)
pip install 'typing-extensions==4.2.0'
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Collecting typing-extensions==4.2.0
Could not find a version that satisfies the requirement typing-extensions==4.2.0 (from versions: 3.6.2, 3.6.2.1, 3.6.5, 3.6.6, 3.7.2, 3.7.4, 3.7.4.1, 3.7.4.2, 3.7.4.3, 3.10.0.0, 3.10.0.1, 3.10.0.2)
No matching distribution found for typing-extensions==4.2.0

 

[Tinkerpete]

There are two characteristics I've been using to communicate with the device. Commands are written to the write characteristic "0000ff02-0000-1000-8000-00805f9b34fb", and the response is received as a notification on the notify characteristic '0000ff01-0000-1000-8000-00805f9b34fb' (both of these can be found in client.py).

The basic algorithm for collecting the information you want is to register for notifications on the notify characteristic, build the command payload to request the data you want, write that to the write characteristic, and then wait until you receive the data you asked for (or a myriad things go wrong and you have to retry sending the command).

@chromedshark: Thank you for the quick reply! The two characteristics you mentioned help me a bit. Anyway, now I get a complete connect done!

But I'm afraid, I have to deal with the procedure in more detail, since I'm programming in C++. Unfortunately I forgot to mention this in my original request.

Since I don't know Python that well, any details you can give me would be greatly appreciated.

 

[Tinkerpete]

build the command payload to request the data you want

Can you tell me how i must do this? Or can you send me an example string for the DC-input query?

 

[Tinkerpete]

Did not work:

I had success with:

C:\Users\Thomas\AppData\Local\Programs\Python\Python310>python.exe -m pip install --upgrade pip
C:\Users\Thomas\AppData\Local\Programs\Python\Python310\Scripts>pip install 'bleak'
C:\Users\Thomas\AppData\Local\Programs\Python\Python310\Scripts>pip install bluetti_mqtt

 

[MrGPS]

I had success with:

C:\Users\Thomas\AppData\Local\Programs\Python\Python310>python.exe -m pip install --upgrade pip
C:\Users\Thomas\AppData\Local\Programs\Python\Python310\Scripts>pip install 'bleak'
C:\Users\Thomas\AppData\Local\Programs\Python\Python310\Scripts>pip install bluetti_mqtt

I tried the same on my Raspberry, no luck. Still the same error message:

pi@raspberrypi:~ $ pip install --upgrade pip
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Collecting pip
Downloading https://files.pythonhosted.org/pack...d2d02789accf7/pip-20.3.4-py2.py3-none-any.whl (1.5MB)
100% |████████████████████████████████| 1.5MB 250kB/s
Installing collected packages: pip
Successfully installed pip-20.3.4


pi@raspberrypi:~ $ pip install 'bleak'
WARNING: pip is being invoked by an old script wrapper. This will fail in a future version of pip.
Please see https://github.com/pypa/pip/issues/5599 for advice on fixing the underlying issue.
To avoid this problem you can invoke Python with '-m pip' instead of running pip directly.
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support pip 21.0 will remove support for this functionality.
Defaulting to user installation because normal site-packages is not writeable
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Collecting bleak
Downloading bleak-0.16.0-py2.py3-none-any.whl (125 kB)
|████████████████████████████████| 125 kB 1.3 MB/s
ERROR: Could not find a version that satisfies the requirement typing-extensions>=4.2.0 (from bleak) (from versions: 3.6.2, 3.6.2.1, 3.6.5, 3.6.6, 3.7.2, 3.7.4, 3.7.4.1, 3.7.4.2, 3.7.4.3, 3.10.0.0, 3.10.0.1, 3.10.0.2)
ERROR: No matching distribution found for typing-extensions>=4.2.0 (from bleak)
pi@raspberrypi:~ $
pi@raspberrypi:~ $ pip install bluetti_mqtt
WARNING: pip is being invoked by an old script wrapper. This will fail in a future version of pip.
Please see https://github.com/pypa/pip/issues/5599 for advice on fixing the underlying issue.
To avoid this problem you can invoke Python with '-m pip' instead of running pip directly.
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support pip 21.0 will remove support for this functionality.
Defaulting to user installation because normal site-packages is not writeable
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Collecting bluetti_mqtt
Using cached bluetti_mqtt-0.1.tar.gz (5.8 kB)
Collecting bleak
Using cached bleak-0.16.0-py2.py3-none-any.whl (125 kB)
Collecting crcmod
Using cached crcmod-1.7.tar.gz (89 kB)
ERROR: Could not find a version that satisfies the requirement typing-extensions>=4.2.0 (from bleak->bluetti_mqtt) (from versions: 3.6.2, 3.6.2.1, 3.6.5, 3.6.6, 3.7.2, 3.7.4, 3.7.4.1, 3.7.4.2, 3.7.4.3, 3.10.0.0, 3.10.0.1, 3.10.0.2)
ERROR: No matching distribution found for typing-extensions>=4.2.0 (from bleak->bluetti_mqtt)

 

[Tinkerpete]

I tried the same on my Raspberry, no luck. Still the same error message:

Ok, i had make it on windows 10.

 

[chromedshark]

@MrGPS Hmmmmmm. That warning that it's using Python 2.7 for pip seems like a problem. I went ahead and installed Python 3.7 just to test this and it works perfectly to install without any dependency issues. You might want to try python3 -m pip install bluetti_mqtt to force it to use Python 3.7 instead of Python 2.7 on your system. It's also possible that pip3 install bluetti_mqtt would work too - you might not have a pip3 command though.

 

[chromedshark]

Can you tell me how i must do this? Or can you send me an example string for the DC-input query?

@Tinkerpete Yeah, the string you need to send for the DC Input Power is "\x01\x03\x00\x24\x00\x01\xc4\x01". I did a detailed breakdown of how this is all built earlier in the thread so I'm not sure if you read that and there was something I said that was still unclear or you just missed that.

For the last two bytes you're going to need to find a C++ library that can do MODBUS CRC16 if you you actually want to construct arbitrary commands yourself in C++, but another option would be to just use the Python code for now for that. It sounds like you managed to install the Python code, and so you should be able to use that like this:

$ python
Python 3.7.3 (default, May 22 2022, 10:21:14)
[GCC 10.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from bluetti_mqtt.core import QueryRangeCommand
>>> bytes(QueryRangeCommand(0x00, 0x24, 1))
b'\x01\x03\x00$\x00\x01\xc4\x01'

Edit: Feel free to DM me or send me an email at the address on my Github profile if you have more questions. I might not respond immediately, but I'll definitely get back to you as soon as I can. I'm happy to help anyone who wants to work on this.

 

[Tinkerpete]

@chromedshark: Many thanks for your help!

@Tinkerpete Yeah, the string you need to send for the DC Input Power is "\x01\x03\x00\x24\x00\x01\xc4\x01".

In the moment this is my biggest problem. I know this is hex code. I think i can read this as: 0103-0024-0001-c401. But I did not understand, where i must insert this in a notify characteristic string like this: "0000ff01-0000-1000-8000-00805f9b34fb"?

I did a detailed breakdown of how this is all built earlier in the thread so I'm not sure if you read that and there was something I said that was still unclear or you just missed that.

I have see this, but not analyzed! I will do it.

For the last two bytes you're going to need to find a C++ library that can do MODBUS CRC16 if you you actually want to construct arbitrary commands yourself in C++, but another option would be to just use the Python code for now for that.

For MODBUS i found that:

Calculating Modbus RTU CRC 16

I'm implementing a software where I read and write data in Modbus RTU protocolo via serial. For that, I need to calculate the two CRC byte at the end of the string of bytes, but I'm being incapable...

stackoverflow.com

I will test it later.

It sounds like you managed to install the Python code, and so you should be able to use that like this:

Yes, i can run your python code on my PC. But it did not find the bluetti.

>>> bytes(QueryRangeCommand(0x00, 0x24, 1))
b'\x01\x03\x00$\x00\x01\xc4\x01'[/CODE]

This works!

Edit: Feel free to DM me or send me an email at the address on my Github profile if you have more questions. I might not respond immediately, but I'll definitely get back to you as soon as I can. I'm happy to help anyone who wants to work on this.

Are you not comfortable with us continuing this in this thread?

 

[Tinkerpete]

@chromedshark:

I have test this function:

unsigned int CRC16_2(unsigned char *buf, int len) {
  unsigned int crc = 0xFFFF;
  for (int pos = 0; pos < len; pos++)  {
    crc ^= (unsigned int)buf[pos];    // XOR byte into least sig. byte of crc
    for (int i = 8; i != 0; i--) {    // Loop over each bit
      if ((crc & 0x0001) != 0) {      // If the LSB is set
        crc >>= 1;                    // Shift right and XOR 0xA001
        crc ^= 0xA001;
      }
      else                            // Else LSB is not set
        crc >>= 1;                    // Just shift right
    }
  }
  return crc;
}

unsigned char arr[6] = {'\x01', '\x03', '\x00', '\x0a', '\x00', '\x37'};
Serial.println (CRC16_2(arr , 6));

and get: 7716 = Hex 1E24 !

Is this correct, because the bytes are changed in order?

If so, i can it made right with:

unsigned short rightRotate(unsigned short x, short d) {
  return (x >> d) | (x << (SHORT_SIZE - d));
}

Write Command - FF02 - Value: 0103 000A 0037 241E

1. The final two bytes, 0x241E, are the checksum. I can calculate this checksum with the code I already have, which is a good sign that I should be able to use it for any devices they have. When I add "01" to the beginning and checksum everything except the last two bytes, I also get 0x241E (which suggests that I'm probably running the checksum on the wrong stuff if changing the payload does not change the resulting checksum).

 

[chromedshark]

In the moment this is my biggest problem. I know this is hex code. I think i can read this as: 0103-0024-0001-c401. But I did not understand, where i must insert this in a notify characteristic string like this

It's not a UUID - it's just a string of raw bytes. Once you have the write characteristic (the BLERemoteCharacteristic*) you'll notice it has a writeValue method for either a string or raw uint8_t array. You'll just call it with any command string. The default for setting response to false here works fine. If you did it correctly, then shortly afterwards you should receive a notification on the notify characteristic with a response payload (which you can then decode).

Is this correct, because the bytes are changed in order?

Yes, that's the same value I get from the Python function I'm using. Most of the binary protocol seems to use network byte order (big-endian), but the CRC appears to be little-endian, so when you use it to build your string the byte order is reversed.

$ python
>>> import struct
>>> from bluetti_mqtt.core.utils import modbus_crc
>>> modbus_crc(b'\x01\x03\x00\x0a\x00\x37')
7716
>>> struct.pack('<H', 7716).hex()
'241e'

Are you not comfortable with us continuing this in this thread?

If it's working for you then that's all I'm looking for.

 

[MrGPS]

@MrGPS Hmmmmmm. That warning that it's using Python 2.7 for pip seems like a problem. I went ahead and installed Python 3.7 just to test this and it works perfectly to install without any dependency issues. You might want to try python3 -m pip install bluetti_mqtt to force it to use Python 3.7 instead of Python 2.7 on your system. It's also possible that pip3 install bluetti_mqtt would work too - you might not have a pip3 command though.

Thanks for the Tip, with the force function is working now by using the !!! "python3 -m pip install bluetti_mqtt"
You have really done agreat work. Thankyou very much.
FYI, i got the following warning:
"The scripts bluetti-logger and bluetti-mqtt are installed in '/home/pi/.local/bin' which is not on PATH."
By changing your command to "python3 /home/pi/.local/bin/bluetti-mqtt" scan, broker and debug are working.

There is only a minor problem left, exactly every 30 minutes I get 0W on "DC Input"in Home Assistant.
Does anybody else with an EP500Pro have the same problem ?

UPDATE: I watched the LCD display on the EP500PRO and it also shows that every 30 minutes both PV Inputs go down to 0 for a few seconds.
I guess it does some recalibration. I'll ask the Bluetti Support.
Anyway, the Python Script works perfect. Once more Thank you very much.

 

[ApolloSK]

@slowdiver:
How do you log data via iobroker? I've connected my Raspberry PI with iobroker and MQTT-Broker. But there are only states from the beginning of the connection but no actual data.

 

[Tinkerpete]

@chromedshark: I am going away for 2 - 3 weeks. After that I will be in touch again.

If it's working for you then that's all I'm looking for.

 

[slowdiver]

@slowdiver:
How do you log data via iobroker? I've connected my Raspberry PI with iobroker and MQTT-Broker. But there are only states from the beginning of the connection but no actual data.
View attachment 110586

@ApolloSK
Hi ApolloSK,
sorry but I moved the AC300 from my working room now to the final destination (basement). There is currently now Raspi there, so I can't test.
But if it was active, I saw changing values all the time. Values were only NOT changing if it was the same VALUE as before. I'm running only bluetti_mqtt on the Raspi. iobroker is running on a Linux box. There's MQTT Broker/Client running. Please check also If bluetti_mqtt is still running (check the logfiles), because serveral times the process was up, but no news in the logfile. I restartet the process and the updates in iobroker was visible again.