Urgent - JK BMS went Offline in the middle of the night - SOC Stuck but still discharging ???

[silverstone]

I just had a nasty surprise when waking up this morning.

TLDR is my setup consists of the following batteries in parallel (after their own independent fuse+breaker):
- Battery 01 / jk-bms-bat-01: 16s 280Ah cells (LF280K) - I think this is HW v11
- Battery 02 / jk-bms-bat-02: 16s 280Ah cells (LF280K) - I think this is HW v10

The system worked for a few months, but only recently started charging/discharging extensively (max 100% SOC, min 20%-25% SOC).
At 100% SOC the BMS "calibrates", so that's nice. Otherwise it can indicate 100% SOC at say 52V, if you leave it like that for a few days/months.

Anyways, up to 2 days ago the 2 plots followed each other perfectly.
Yesterday, the SOC were already quite different (jk-bms-bat-01 would report approx 5%-10% MORE than jk-bms-bat-02). But still jk-bms-bat-01 would keep charging.

And funnily enough, jk-bms-bat-01 is the battery with the LOWEST cell voltage, compared to jk-bms-bat-02, where one cell will go into OVP, if I charge the pack to 56V. I boost/absorb at 55V and float at 54.2V, so even with the unbalance within jk-bms-bat-02, it should be pretty safe.

So to recap:
- jk-bms-bat-01 reports the HIGHER SOC and LOWER VOLTAGE
- jk-bms-bat-02 reports the LOWER SOC and HIGHER VOLTAGE

This could possibly be due in differences in JK BMS between HW v10 and HW v11 (and their associated firmwares - voltage tolerances ???), or it could be due that one cell in Bat02 is relatively high while one cell in Bat02 is relatively low.

I tried charging to 56V and almost reached OVP on Bat02, but stopped short of that. Clearly the delta voltage is very high on bat02 at the end of charge (which to some extend can be considered as normal-ish):


This morning, for whatever reason, jk-bms-bat-02 reports: "errors: Offline". But it's still allowing charging/discharging. What does it mean ???



This is the screenshot of the error




Not the best debugging I know. I wanted to do a MQTT to InfluxDB Bridge so that I could visualize the data in Grafana but you know ... plans and priorities change .

BMS is reporting "Offline" but still allows discharge (and probably also charge). SOC is reportedly stuck and so are cell voltages etc.

Any idea ?? I'm quite nervous right now that no protection is enabled. But why is this failing in a "non-safe failure mode" manner ?

 

[silverstone]

If I try to pull the data from MQTT using my own Python Script this is what I get. Worrying and nothing like I saw before:

                                                            ID Battery_Description Battery_Number Scope           Type Physical              Name  Unit Value Cell_Number
cell_voltage_1                                  cell_voltage_1        jk-bms-bat02              2  Cell         sensor  Voltage  Cell #01 Voltage   VDC   nan           1
cell_voltage_2                                  cell_voltage_2        jk-bms-bat02              2  Cell         sensor  Voltage  Cell #02 Voltage   VDC   nan           2
cell_voltage_3                                  cell_voltage_3        jk-bms-bat02              2  Cell         sensor  Voltage  Cell #03 Voltage   VDC   nan           3
cell_voltage_4                                  cell_voltage_4        jk-bms-bat02              2  Cell         sensor  Voltage  Cell #04 Voltage   VDC   nan           4
cell_voltage_5                                  cell_voltage_5        jk-bms-bat02              2  Cell         sensor  Voltage  Cell #05 Voltage   VDC   nan           5
cell_voltage_6                                  cell_voltage_6        jk-bms-bat02              2  Cell         sensor  Voltage  Cell #06 Voltage   VDC   nan           6
cell_voltage_7                                  cell_voltage_7        jk-bms-bat02              2  Cell         sensor  Voltage  Cell #07 Voltage   VDC   nan           7
cell_voltage_8                                  cell_voltage_8        jk-bms-bat02              2  Cell         sensor  Voltage  Cell #08 Voltage   VDC   nan           8
cell_voltage_9                                  cell_voltage_9        jk-bms-bat02              2  Cell         sensor  Voltage  Cell #09 Voltage   VDC   nan           9
cell_voltage_10                                cell_voltage_10        jk-bms-bat02              2  Cell         sensor  Voltage  Cell #10 Voltage   VDC   nan          10
cell_voltage_11                                cell_voltage_11        jk-bms-bat02              2  Cell         sensor  Voltage  Cell #11 Voltage   VDC   nan          11
cell_voltage_12                                cell_voltage_12        jk-bms-bat02              2  Cell         sensor  Voltage  Cell #12 Voltage   VDC   nan          12
cell_voltage_13                                cell_voltage_13        jk-bms-bat02              2  Cell         sensor  Voltage  Cell #13 Voltage   VDC   nan          13
cell_voltage_14                                cell_voltage_14        jk-bms-bat02              2  Cell         sensor  Voltage  Cell #14 Voltage   VDC   nan          14
cell_voltage_15                                cell_voltage_15        jk-bms-bat02              2  Cell         sensor  Voltage  Cell #15 Voltage   VDC   nan          15
cell_voltage_16                                cell_voltage_16        jk-bms-bat02              2  Cell         sensor  Voltage  Cell #16 Voltage   VDC   nan          16
charging                                              charging        jk-bms-bat02              2        binary_sensor                             none    ON           0
discharging                                        discharging        jk-bms-bat02              2        binary_sensor                             none    ON           0
balancer                                              balancer        jk-bms-bat02              2               switch                             none    ON           0
enable_bluetooth_connection        enable_bluetooth_connection        jk-bms-bat02              2               switch                             none    ON           0
online_status                                    online_status        jk-bms-bat02              2        binary_sensor                             none   OFF           0
balancing                                            balancing        jk-bms-bat02              2        binary_sensor                             none   OFF           0
min_cell_voltage                              min_cell_voltage        jk-bms-bat02              2               sensor                              VDC   nan           0
max_cell_voltage                              max_cell_voltage        jk-bms-bat02              2               sensor                              VDC   nan           0
min_voltage_cell                              min_voltage_cell        jk-bms-bat02              2               sensor                              VDC   nan           0
max_voltage_cell                              max_voltage_cell        jk-bms-bat02              2               sensor                              VDC   nan           0
delta_cell_voltage                          delta_cell_voltage        jk-bms-bat02              2               sensor                              VDC   nan           0
average_cell_voltage                      average_cell_voltage        jk-bms-bat02              2               sensor                              VDC   nan           0
total_voltage                                    total_voltage        jk-bms-bat02              2               sensor                              VDC   nan           0
current                                                current        jk-bms-bat02              2               sensor                              ADC   nan           0
power                                                    power        jk-bms-bat02              2               sensor                                W   nan           0
charging_power                                  charging_power        jk-bms-bat02              2               sensor                                W   nan           0
discharging_power                            discharging_power        jk-bms-bat02              2               sensor                                W   nan           0
temperature_sensor_1                      temperature_sensor_1        jk-bms-bat02              2               sensor                               °C   nan           0
temperature_sensor_2                      temperature_sensor_2        jk-bms-bat02              2               sensor                               °C   nan           0
power_tube_temperature                  power_tube_temperature        jk-bms-bat02              2               sensor                                W   nan           0
capacity_remaining                          capacity_remaining        jk-bms-bat02              2               sensor                               Ah   nan           0
total_battery_capacity_setting  total_battery_capacity_setting        jk-bms-bat02              2               sensor                               Ah   nan           0
charging_cycles                                charging_cycles        jk-bms-bat02              2               sensor                             none   nan           0
total_charging_cycle_capacity    total_charging_cycle_capacity        jk-bms-bat02              2               sensor                               Ah   nan           0
total_runtime                                    total_runtime        jk-bms-bat02              2               sensor                                s   nan           0
balancing_current                            balancing_current        jk-bms-bat02              2               sensor                              ADC   nan           0
errors_bitmask                                  errors_bitmask        jk-bms-bat02              2               sensor                             none   nan           0

 

[Watts Happening]

Did you try turning it off and back on?

 

[silverstone]

Did you try turning it off and back on?

How would I do that ?

I do NOT have the switch+display thing.

And applying say 5V extra (when I open the breaker of course, otherwise the other battery would just take whatever I throw at it) I don't think it will work ... After all, if it's allowing discharge, maybe it's also allowing charging.

And not sure how the esphome jk-bms reports that I can still switch on/off:


By the way, I can tell it's still discharging, because the power supplied by jk-bms-bat01 is only half of my base load consumption .

 

[silverstone]

Not sure what is happening now ... I tried scanning from my JK BMS App on Smartphone from outside the garage, no BMS could be detected (probably because jk-bms-bat01 is connected correctly and jk-bms-bat02 is offline). Then I went for a walk with my dog.

Now I came back and this is what is happening









So it seems to have come back ????? Looking at SOC/Min-Max Voltage/Errors it looks like that ... Still, offline for nearly 6 hours !

Coincidence that the error resets when I scan from smartphone using JK BMS App ?

Or maybe just because right now the sun is starting to rise ?

 

[silverstone]

And now it went offline again. What's going on ???

 

[DIYrich]

How would I do that ?

I do NOT have the switch+display thing.

My first thought is that one of the BMS is failing.
Turn off the batteries to the Inverter.
Go into the BMS app, and turn off discharge and charge.
Open the fuse/breaker
Disconnect the BMS from the battery. (power cycle the BMS).

At this point, re-tighten all your battery connections.

Reverse the process.

If the problems persist, swap the BMS's.
If the problem follows a BMS, then you have a bad BMS.
Otherwise, you might have a bad cell (s), and/or bad connection in the battery.

 

[silverstone]

My first thought is that one of the BMS is failing.
Turn off the batteries to the Inverter.
Go into the BMS app, and turn off discharge and charge.
Open the fuse/breaker
Disconnect the BMS from the battery. (power cycle the BMS).

At this point, re-tighten all your battery connections.

Reverse the process.

If the problems persist, swap the BMS's.
If the problem follows a BMS, then you have a bad BMS.
Otherwise, you might have a bad cell (s), and/or bad connection in the battery.

That's what I feared as well. It came back online a few minutes after my screenshot actually. It's now working (knock on wood) ...

If the device is offline, how can I power cycle it ??? It couldn't connect via BLE / Bluetooth. And as I said, if it allows charging, I cannot apply 5V extra (otherwise I'd need a 100kW charger or something like that ...).

Swapping the BMS is the absolute nightmare. Both my batteries are in this box. Replacement of the BMS would be *the* most impossible task. It would require turning off BOTH batteries. And of course a few cable lugs are almost impossible to access / untighten.

I will hopefully soon build a few more batteries (this time each in a separate box - easier maintenance as I do not need to turn every battery off). But that's still a few months away ...


Or ... is it possible that's it is purely a bluetooth issue and the "offline" means "bluetooth offline", and NOT the BMS being shut-off ? Why would it be like that though .... The ESPHome ESP32 using https://github.com/syssi/esphome-jk-bms is directly ABOVE the BMS within 40cm on a polycarbonate plate (yeah ... temporary solution ...).

 

[DIYrich]

If you turn off ALL batteries to the inverter, then the current from the batteries should be nominal.
Turn off discharge/charge from what batteries you can.
Open whatever fuses/breakers you can.
Check that there are no stray currents flowing.
Then physically disconnect the battery from the BMS.

You should design your batteries so that there is an easy way to turn them off / disconnect them from the system.

 

[silverstone]

If you turn off ALL batteries to the inverter, then the current from the batteries should be nominal.
Turn off discharge/charge from what batteries you can.
Open whatever fuses/breakers you can.
Check that there are no stray currents flowing.
Then physically disconnect the battery from the BMS.

You should design your batteries so that there is an easy way to turn them off / disconnect them from the system.

If I turn off ALL batteries, then the current will be ZERO. The inverters cannot feed current into the batteries if they are disconnected ...

There is an easy way to turn them off ... It's swapping the BMS that it's going to be an absolute nightmare.

 

[silverstone]

Maybe it was just a fluke and now it is working ?

But if the BMS is offline, should it still allow charging/discharging ? That seems NOT what "fail-safe" is. Seems more "BMS bad design".

If it's "just" a BLE/bluetooth issue, then maybe it's not so bad, as long as the BMS is working and protecting the battery.

 

[DIYrich]

If I turn off ALL batteries, then the current will be ZERO. The inverters cannot feed current into the batteries if they are disconnected ...

There is an easy way to turn them off ... It's swapping the BMS that it's going to be an absolute nightmare.

You need current to BMS to be 0 to turn it off (power cycle). If you don't have the button, you have to physically disconnect it.

 

[DIYrich]

Maybe it was just a fluke and now it is working ?

A lot of failing electronics work periodically, until they don't.

 

[silverstone]

A lot of failing electronics work periodically, until they don't.

It's been running for only 6 months though. Seems a bit early for that.

 

[silverstone]

I think if it comes to that, I will need to move battery v1 into battery v2 new simplified (and more accessible & maintenance friendly) enclosure.

But I would prefer to have a few extra batteries installed first before starting that MAJOR operation.

 

[ChrisG]

I think if it comes to that, I will need to move battery v1 into battery v2 new simplified (and more accessible & maintenance friendly) enclosure.

But I would prefer to have a few extra batteries installed first before starting that MAJOR operation.

Question, how is that data getting into HomeAssistsnt from the JK? I have something similar but you are getting different metrics.

 

[silverstone]

Question, how is that data getting into HomeAssistsnt from the JK? I have something similar but you are getting different metrics.

Just use https://github.com/syssi/esphome-jk-bms and an ESP32. I used BLE and it sends data to MQTT Mosquitto Server.

I also recently discovered that Home Assistant is smart enough to find any data on the MQTT network, so it automatically discovered the entities.

Then it was just a matter of adding History Graph to a new Dashboard.

 

[ChrisG]

Just use https://github.com/syssi/esphome-jk-bms and an ESP32. I used BLE and it sends data to MQTT Mosquitto Server.

I also recently discovered that Home Assistant is smart enough to find any data on the MQTT network, so it automatically discovered the entities.

Then it was just a matter of adding History Graph to a new Dashboard.

Thanks, I’m using a fork of syssi project with esp32 and pumping everything into influx with Grafana. Just realized I do have error state pumping on also.

 

[silverstone]

Thanks, I’m using a fork of syssi project with esp32 and pumping everything into influx with Grafana. Just realized I do have error state pumping on also.

Well I have the same problem. I need to import the data into InfluxDB in order to have a "proper" data storage long term. I think I need to write some Python script for it. Something like what was done here https://diysolarforum.com/threads/jk-bms-grafana-in-one-evening.61148/

But then stuff happens ... Battery decides to go offline, Podman/Docker bugs hitting me while spinning up containers, ...

NOTHING ever goes according to plan

 

[Sky-HHI]

Lets start with the simple stuff first. Did you top balance the cells before building the batteries? Can you use the jk bms app to look at your settings? I find it hard to read your charts but I think one of your bms is turning off when one cell in over voltage, what do you have the OVP set at? What is the delta reading from the bms app?