DIY Vendor agnostic Smart Shunt & Victron Smart Solar MPPT data logger based on ESP8266

[SeaGal]

All sounds good and progressing quickly

Personally, and this is just my preference and wariness of 'all-things-cloud' - and not intended as a 'design-input' as I'm unlikely to use your code, having already developed similar...

I'd far rather have a system that is totally self-contained, does not rely on Internet connectivity or other businesses computers. Who knows when the Internet goes down, signal lost if it's an RV / cabin / remote deployment. Not to mention any data gathering and monetisation of free cloud-based systems.

I'd also like to think that the frequency of needing to swap ESP devices is less than losing access to 'thingywhatsitcalled.io' or similar.

For my system, I have a spare ESP32 board (pre-loaded with the software) ready for a quick cold-swap in case of failure. The code is written so that data read from non volatile storage will load default values if that data was not found (as may happen with an ESP swap). e.g. like this...

#include <Preferences.h>
Preferences configData;
configData.begin("BMSESP", false);
...
// load stored configuration data from non-volatile storage
AllowOvernightDischarge = configData.getBool("AllowDis", false);

 

[Quattrohead]

Yeah I don't see the point of having anything cloud-based involved when you're already writing code for a local device.
I know home assistant is not everyone's favorite but at least it's local and infinitely programmable if or when you figure out what you're doing with it.
This part is the most difficult for me because I am a very electro mechanical engineer and programming is absolutely a foreign language to me. I just cannot get the hang of it and it makes my brain hurt.

 

[rin67630]

...having already developed similar...

Is your code public?

 

[rin67630]

Yeah I don't see the point of having anything cloud-based involved when you're already writing code for a local device.

For me, "local" is off-grid and I am hunting every mW that drains my battery on overcasted days.
Every computer required on site -even a Raspberry Pi- is just too much drain.
So everybody has his own reasons...

 

[SeaGal]

Is your code public?

Much of my code is way too specific to be of general use, but the guts of it - that which would be of interest to others, including the VE.direct interface - uses the code from the thread on here that I shared in post #17, the direct GitHub link is this one...

GitHub - sijones/VE.DirectMQTTCANBUS: ESP32 middleware, VEDirect Smart Shunt to Inverter PylonTech protocol

ESP32 middleware, VEDirect Smart Shunt to Inverter PylonTech protocol - sijones/VE.DirectMQTTCANBUS

github.com

I have also interfaced to the UART port of my (Overkill) JBD BMS, using this library..

GitHub - FurTrader/Overkill-Solar-BMS_2-Arduino-Library: Read and Write Library for Overkill Solar BMSs

Read and Write Library for Overkill Solar BMSs. Contribute to FurTrader/Overkill-Solar-BMS_2-Arduino-Library development by creating an account on GitHub.

github.com

 

[SeaGal]

Every computer required on site -even a Raspberry Pi- is just too much drain.

I'd assume that an ESP32 would be less, but I thought a Pi Pico or Pi Zero 2 or Orange Pi Zero were in the order of 1W consumption or less. Guess it will depend largely on how much processing is going on?

 

[rin67630]

I'd assume that an ESP32 would be less, but I thought a Pi Pico or Pi Zero 2 or Orange Pi Zero were in the order of 1W consumption or less. Guess it will depend largely on how much processing is going on?

Even with the 2W of a Pi Zero 2 you can drain a 12V/100Ah battery in less than a week.
Running things like Home Assistant requires a full blown Linux, you won't put that on a Pi Pico.
An ESP8266 is <150mW.

Edit: by the way: I work also with Raspberry Pis, which run 24h/365days a year.
Exact today I had to replace a worn SD card. When you write permanently to EEPROM you get into trouble.

 

[AlpineJoe]

Do both? Local HTML page and push data to the cloud for remote monitoring, but with the code smart enough to not die if the internet or provider is down.

I'd like to not poke holes in my firewall to access my solar data.

 

[SeaGal]

Exact today I had to replace a worn SD card. When you write permanently to EEPROM you get into trouble.

running my RPi5 with an SSD.

 

[rin67630]

I'd like to not poke holes in my firewall to access my solar data.

You don't need to open any port of the firewall to access thinger.io data.

running my RPi5 with an SSD.

Just you wait Henry Higgins...

 

[SeaGal]

Just you wait Henry Higgins...

Not sure I get your point?... surely SSD is more reliable than SD card, though obviously not infallible. Eliza.

 

[rin67630]

Not sure I get your point?... surely SSD is more reliable than SD card, though obviously not infallible. Eliza.

...and SD is much more reliable than EEPROM, which has no wear levelling at all and is designed for a few hundreds writes max.
...and SD fails. As got today.

 

[rin67630]

Do both? Local HTML page and push data to the cloud for remote monitoring, but with the code smart enough to not die if the internet or provider is down.

That is how e.g. Drok-Juntek-on-Steroids work and Victron-VE-on Steroids will work too.
The code runs 100% on the ESP and Internet is only required for monitoring and saving variables that must survive a reboot.
Without internet, you have local control of the ESP over a Telnet command line interface, you have reports to monitor the device too.

 

[rin67630]

So I needed some debugging to estimate the internal resistance, but now it seems solid.

What works:
- Framework / Network / Telnet / Telnet Menu / Telnet reporting
- Gathering data from VE-Direct
- Transform serial text into Float variables.
- Compute data not transmitted by Victron:
- Panel power (Estimated when not transmitted)
- Panel current
- Load power
- Gather battery internal resistance from variations of battery voltage/battery current.

What is in the pipeline:
- Coulomb integration
- Battery capacity (one full cycle without load current)
- Battery SOC (need input)

- Dashboards.

Here a Telnet example:
$ telnet 192.168.188.27
Trying 192.168.188.27...
Connected to victron-telnet.fritz.box.
Escape character is '^]'.

Snooze
V

Values Report
BatV:10.50 BatI:1.670 BatW: 17.53 PanV: 11.79 PanW: 17.53, LoadI 0.00, LoadW 0.00, IOhm 0.0025
BatV:10.50 BatI:1.690 BatW: 17.75 PanV: 11.47 PanW: 17.75, LoadI 0.00, LoadW 0.00, IOhm 0.0025
BatV:10.50 BatI:1.700 BatW: 17.85 PanV: 11.48 PanW: 17.85, LoadI 0.00, LoadW 0.00, IOhm 0.0025
BatV:10.50 BatI:1.700 BatW: 17.85 PanV: 11.48 PanW: 17.85, LoadI 0.00, LoadW 0.00, IOhm 0.0025
BatV:10.50 BatI:1.690 BatW: 17.75 PanV: 11.48 PanW: 17.75, LoadI 0.00, LoadW 0.00, IOhm 0.0025
BatV:10.50 BatI:1.690 BatW: 17.75 PanV: 11.48 PanW: 17.75, LoadI 0.00, LoadW 0.00, IOhm 0.0025
BatV:10.50 BatI:1.700 BatW: 17.85 PanV: 11.49 PanW: 17.85, LoadI 0.00, LoadW 0.00, IOhm 0.0025
BatV:10.50 BatI:1.670 BatW: 17.53 PanV: 11.79 PanW: 17.53, LoadI 0.00, LoadW 0.00, IOhm 0.0025
BatV:10.50 BatI:1.690 BatW: 17.75 PanV: 11.48 PanW: 17.75, LoadI 0.00, LoadW 0.00, IOhm 0.0025
BatV:10.50 BatI:1.670 BatW: 17.53 PanV: 11.77 PanW: 17.53, LoadI 0.00, LoadW 0.00, IOhm 0.0025
BatV:10.50 BatI:1.700 BatW: 17.85 PanV: 11.48 PanW: 17.85, LoadI 0.00, LoadW 0.00, IOhm 0.0025
BatV:10.50 BatI:1.670 BatW: 17.53 PanV: 11.79 PanW: 17.53, LoadI 0.00, LoadW 0.00, IOhm 0.0025
BatV:10.50 BatI:1.690 BatW: 17.75 PanV: 11.48 PanW: 17.75, LoadI 0.00, LoadW 0.00, IOhm 0.0025
BatV:10.50 BatI:1.670 BatW: 17.53 PanV: 11.79 PanW: 17.53, LoadI 0.00, LoadW 0.00, IOhm 0.0025
BatV:10.50 BatI:1.680 BatW: 17.64 PanV: 11.48 PanW: 17.64, LoadI 0.00, LoadW 0.00, IOhm 0.0025
BatV:10.50 BatI:1.670 BatW: 17.53 PanV: 11.79 PanW: 17.53, LoadI 0.00, LoadW 0.00, IOhm 0.0025
 

Snooze

After dinner I will update the code to Github.

The I will have to make a greater pause: visiting my old mother and later my children grand'children.

 

[AlpineJoe]

What’s the mating connector for connecting to the VE port? I’m going to need to order some.

 

[rin67630]

What’s the mating connector for connecting to the VE port? I’m going to need to order some.

You may use "Grove" cables and cut away 0,5 mm from the two bottom flanges
But you can also -for the time being- use a female jumper strip, you only need one wire to the second pin from the left, when you feed the ESP from the battery over a 5V buck converter. Gnd comes from the battery.

 

[SeaGal]

What’s the mating connector for connecting to the VE port? I’m going to need to order some.

For the smart-shunt it's JST PH - assume same for other VE ports. See...

DIY Battery via Smart shunt to inverter integration (Solis etc)

Hi All, I've taken some code that handles Victron's VE.Direct protocol (Smart Shunt) and added in CAN Bus / PylonTech protocol connections, this allows the State of Charge, Voltage, Amps etc all be sent to the inverter - any one that supports the PylonTech protocol. I have it connected to a...

diysolarforum.com

and

DIY Battery via Smart shunt to inverter integration (Solis etc)

I think it was this one... https://thepihut.com/products/jst-ph-4-pin-to-female-socket-cable-i2c-stemma-cable-200mm The one you linked too looks fine as well, though the description refers to a male connector, but that doesn't tally with the picture?

diysolarforum.com

JST PH 4-Pin to Female Socket Cable - I2C STEMMA Cable - 200mm

This cable will let you turn a JST PH 4-pin cable port into 4 individual wires with high-quality 0.1" female sockets on the end. We're carrying these to match up with any of our boards with 'STEMMA' connectors on them such as the Hallowing or Trellis M4. Can also be used with any STEMMA...

thepihut.com

 

[AlpineJoe]

You may use "Grove" cables and cut away 0,5 mm from the two bottom flanges
But you can also -for the time being- use a female jumper strip, you only need one wire to the second pin from the left, when you feed the ESP from the battery over a 5V buck converter. Gnd comes from the battery.

Thanks, I have some Grove cables on hand. Looking forward to doing this. I have a Wyze camera looking at my Epever MT50 displays so I can monitor charge current from them remotely (only in real time) but no way to look at my Smart Shunt or 150/45 SCC remotely.

 

[rin67630]

Thanks, I have some Grove cables on hand. Looking forward to doing this. I have a Wyze camera looking at my Epever MT50 displays so I can monitor charge current from them remotely (only in real time) but no way to look at my Smart Shunt or 150/45 SCC remotely.

What is your configuration?: Is the EpeverMT 50 and the Victron 150/45 + the Victron smart shunt on the same battery?

 

[AlpineJoe]

What is your configuration?: Is the EpeverMT 50 and the Victron 150/45 + the Victron smart shunt on the same battery?

2 Epever 4215bn, each with an MT50 display, and a Victron 150/45 SmartSolar MPPT. So 3 solar controllers, all attached in parallel to a 24V ~480AH battery pack.