Need some ideas on off-line/off-grid remote monitoring.

[PhiPhi]

Background:
I've been using off-grid solar for about 2 years now, I have built my own battery packs and performed the installation of all related solar equipment myself. I have an abiding revulsion to any and all data harvesting applications especially those that offer no offline alternatives. Yesterday I had to install a new hybrid inverter and 2 new BMS's for my battery packs, this time I chose more expensive components compared to my last set up due to the potential for a wider support on the offline data collection side which was non-existent on the old set-up. The Invertor, batteries, etc. are located in an outbuilding 5 meters from the house. BTW I live in S.E.Asia where a lot of branded equipment is not available or beyond my budget once shipping and customs is taken into account.

Equipment:
1 x Growatt SPF 6000 ES Plus WiFI / CAN / RS485 / USB
2 x JK Smart BMS 8-24s 150A/300A max Bluetooth CAN/RS485

Requirement:
To have readily available real-time and historical data accessible from my laptop running LinuxMint (able to change distro if required).
Ideally I'd like the data to be transmitted via a dedicated secure WiFi network but cables can be run if necessary.
Ability to change equipment parameters remotely.
Ability to backup/export historical data into CSV / XML or other common format.

Nice to have:
Flashy charts and graphs are nice to have but not essential at this stage
I could also see control triggers being useful i.e. controlling a relay to the immersion heater once batteries reach a certain % of charge.

Summary.
I am looking for help in narrowing down the number of blind ally's I might lead myself down or for potential solutions I might have no idea even exist. If responding to this post don't assume I know how CAN bus or MODBUS works beyond that they are communication protocols, once we have a consensus or a viable potential solution I will be spending a lot of time getting up to speed on the relevant tech. I live a sustainable lifestyle with a minimal income and budget is a concern, however, off the shelf software is not completely off the table.

I do not think I am alone in this case and I would be prepared to document this journey to encourage others who might learn from my mistakes and victories along the way.

 

[wpns]

I would look at Solar Assistant, if it’ll talk to your chosen hardware. It’s got everything builtin and you can interface with it’s MQTT broker if you want more.
I roll my own with Modbus and Python, but SA is a good starting point, and you can always supplement with Home Assistant if you want more control.

 

[PhiPhi]

I looked at their site and yes I agree Solar Assistant would be a good place to get my feet wet, unfortunately the payment systems paypal or zell are not available in my region so I would not be able to purchase the software.

 

[wpns]

I looked at their site and yes I agree Solar Assistant would be a good place to get my feet wet, unfortunately the payment systems paypal or zell are not available in my region so I would not be able to purchase the software.

Fair enough, though if you have a way to send me money I could buy it for you, DM me.

Either way look at MQTT, it’s a little different from other data gathering paradigms ( I also use SQL and Redis) but it takes the time factor out of data gathering. <This> data source updates every 5 minutes, <that> one is in a tight loop updating every second or so, a subscriber gets them all and can deal with errors whenever they arrive.

 

[PhiPhi]

Fair enough, though if you have a way to send me money I could buy it for you, DM me.

Either way look at MQTT, it’s a little different from other data gathering paradigms ( I also use SQL and Redis) but it takes the time factor out of data gathering. <This> data source updates every 5 minutes, <that> one is in a tight loop updating every second or so, a subscriber gets them all and can deal with errors whenever they arrive.

Thank you so much for the kind offer, however, I have since discovered that my friend managed to open a Paypal account through his company and will order for me.

I had a look at a few MQTT & YAML config file tutorials on Youtube which helped me understand a lot more about how the comms are working, I still not quite clear about what is happening at the equipment controller board end. Specifically does the inverter and BMS's support native MQQT or is Solar Assistant as the broker also converting that i/o from MODBUS queries/commands to MQTT?

 

[PhiPhi]

So far I have ordered the following hardware:

Orange Pi 3 LTS 2GB RAM 8GB eMMC
Case, heatsync and Fan Set for Orange Pi
DC-DC Buck Converter Step Down Module 200W 15A Adjustable Voltage 8-60V to 1-36V Power Supply

I'm going to wait until items arrive and are tested before getting SA. To keep things simple I'm going to get the inverter data acquisition up and running as stage one (hopefully it'll work with the supplies USB B to A cable) then stage 2 I'll start looking at the JK BMS's cable/set up requirements. At the moment i have each BMS connected to 2 x 48v battery packs in parallel but not to the inverter, are there any compelling reasons to connect them to the inverter data port?

Questions regarding the step down module:

I'm guessing this taps into my +/- battery leads where they exit the paralleled 48v banks rather than connecting to a single battery?

What protection is recommended (with ratings) for the input / output of the step down buck converter? i.e: wirw size, fuses ,diodes, power switch. I think the Orange Pi LTS uses 5v 3A max.

 

[ThaiTaffy]

So far I have ordered the following hardware:

Orange Pi 3 LTS 2GB RAM 8GB eMMC
Case, heatsync and Fan Set for Orange Pi
DC-DC Buck Converter Step Down Module 200W 15A Adjustable Voltage 8-60V to 1-36V Power Supply

Personally I'd get a dedicated 12v to usb supply and run the buck into that, normally they include protection and it shouldn't be affected by any voltage swings when charging like some unregulated step down psu's

I'm going to wait until items arrive and are tested before getting SA. To keep things simple I'm going to get the inverter data acquisition up and running as stage one (hopefully it'll work with the supplies USB B to A cable) then stage 2 I'll start looking at the JK BMS's cable/set up requirements.

Not sure how SA works converting rs485 but I tend to use rs485 to TTL boards for converting my can data to uart

At the moment i have each BMS connected to 2 x 48v battery packs in parallel but not to the inverter, are there any compelling reasons to connect them to the inverter data port?

Just be careful with protocols and wiring, double check everything and research before plugging anything in.

Questions regarding the step down module:

I'm guessing this taps into my +/- battery leads where they exit the paralleled 48v banks rather than connecting to a single battery?

Your step down has a max input of 60v so tapping into the same cables as what feeds your inverter should be fine.

What protection is recommended (with ratings) for the input / output of the step down buck converter? i.e: wirw size, fuses ,diodes, power switch. I think the Orange Pi LTS uses 5v 3 Watts max.

Go off the wattage of the buck rather than amperage so 200w at 48v about 4 amps so try to go lower if you can if not 5amp will atleast protect the cables output is unlikely to go above 1 amp as direct to the pi but if your to add a dedicated stepdown to usb they normally max out at 2.4 amp.

Just answering what I can, by no means and expert here anything your unsure of best wait for one of the professionals to answer.

 

[PhiPhi]

Personally I'd get a dedicated 12v to usb supply and run the buck into that, normally they include protection and it shouldn't be affected by any voltage swings when charging like some unregulated step down psu's

Not sure how SA works converting rs485 but I tend to use rs485 to TTL boards for converting my can data to uart

Just be careful with protocols and wiring, double check everything and research before plugging anything in.

Your step down has a max input of 60v so tapping into the same cables as what feeds your inverter should be fine.

Go off the wattage of the buck rather than amperage so 200w at 48v about 4 amps so try to go lower if you can if not 5amp will atleast protect the cables output is unlikely to go above 1 amp as direct to the pi but if your to add a dedicated stepdown to usb they normally max out at 2.4 amp.

Just answering what I can, by no means and expert here anything your unsure of best wait for one of the professionals to answer.

Thanks for the reply, I just edited my post to show Orange Pi used up to 3A not 3W, Doh! I was thinking of doing a 2 stage reduction but I don't know if you can feed one buck converter into another or would the USB be a different type of supply i.e. PWM? My 48v NMC batteries are actually 58v max, but I limit them to 55.6v and 46v at the low end. In fact that's given me an idea, if I run the buck converter to an online UPS I can ensure smooth, uninterrupted power to the Orange Pi, I'll look into that.

 

[ThaiTaffy]

Thanks for the reply, I just edited my post to show Orange Pi used up to 3A not 3W, Doh! I was thinking of doing a 2 stage reduction but I don't know if you can feed one buck converter into another or would the USB be a different type of supply i.e. PWM? My 48v NMC batteries are actually 58v max, but I limit them to 55.6v and 46v at the low end. In fact that's given me an idea, if I run the buck converter to an online UPS I can ensure smooth, uninterrupted power to the Orange Pi, I'll look into that.

Yep shopee/Lazada is full of small 24v/12v ups's that run off 18650 batteries but the only problem being most only have a 2.4 amp usb output which isn't enough for the pi @ 15w/3amp so be careful shopping.

 

[PhiPhi]

Yep shopee/Lazada is full of small 24v/12v ups's that run off 18650 batteries but the only problem being most only have a 2.4 amp usb output which isn't enough for the pi @ 15w/3amp so be careful shopping.

Yes, I've just found that out though Shopee seems to have a wider selection, I think I'll put this to one side and build a proper UPS with a 12v prismatic battery and A/C inverter battery when I have time I can then use it to power emergency lighting in the solar/generator shed.

 

[wpns]

Thank you so much for the kind offer, however, I have since discovered that my friend managed to open a Paypal account through his company and will order for me.

I had a look at a few MQTT & YAML config file tutorials on Youtube which helped me understand a lot more about how the comms are working, I still not quite clear about what is happening at the equipment controller board end. Specifically does the inverter and BMS's support native MQQT or is Solar Assistant as the broker also converting that i/o from MODBUS queries/commands to MQTT?

Solar Assistant runs the broker, though you can use Home Assistant’s broker or easily stand one up yourself.

 

[PhiPhi]

Solar Assistant runs the broker, though you can use Home Assistant’s broker or easily stand one up yourself.

I understood that SA acts as the broker but was not sure what was coming out of the inverter/bms, put another way is SA subscribing to the equipment control board with MQTT or does it have the MODBUS registry maps to issue requests/commands on that leg of the communication? Hope that makes sense, when I looked into getting data out of my old cheap chinesium inverter for my own use a few years ago I came up against a brick wall as I couldn't obtain the registry mapping anywhere and as a novice I did not think I stood much chance in working out the coil/register mapping myself, for some reason I thought MODBUS would be the low level protocol to read and write the registers for all of this type of equipment.

 

[ThaiTaffy]

I understood that SA acts as the broker but was not sure what was coming out of the inverter/bms, put another way is SA subscribing to the equipment control board with MQTT or does it have the MODBUS registry maps to issue requests/commands on that leg of the communication? Hope that makes sense, when I looked into getting data out of my old cheap chinesium inverter for my own use a few years ago I came up against a brick wall as I couldn't obtain the registry mapping anywhere and as a novice I did not think I stood much chance in working out the coil/register mapping myself, for some reason I thought MODBUS would be the low level protocol to read and write the registers for all of this type of equipment.

Now days everything is hackable with a esp32 and most stuff has already been hacked and freely available on GitHub as long as it's popular enough. If you have any issues finding anything, say so and myself or some other friendly person should be able to point you in the right direction.

 

[PhiPhi]

Now days everything is hackable with a esp32 and most stuff has already been hacked and freely available on GitHub as long as it's popular enough. If you have any issues finding anything, say so and myself or some other friendly person should be able to point you in the right direction.

Thanks for the offer, all I'm really trying to do at the moment is build up a mental picture of the communications on my new Growatt inverter, in my simplistic current visualisation I see sensor values being read or written to a modbus register on the inverter control board by a micro controller, that register can be written or read to by the MQTT SA broker, to do this SA would need to have the inverter register map and communicate with the inverter via modbus. SA makes the inverter modbus sensors and switches available to the client subscriber(s) and subscribes the inverter to the clients command MQTT data which SA sends to the inverter as a modbus write register command. Alternatively does the inverter some inverter control board have built-in MQTT capability either natively or layered over modbus?

I don't discount the possibility that I might be asking a questions that are so obvious it's misleading or so wrong it just doesn't make sense.

 

[ThaiTaffy]

I'm unsure with this particular growwatt and solar assistant but from my own experience via home assistant, I upload an esp32 with the modbus data and any variables, that then translates the modbus to mqtt(not in my case it's transferred directly to my ha API though I can do it via mqtt) the broker receives the information and applies it to my visual dashboard. I also struggle to understand how solar assistant receives the data from the inverter but best guess is you must have to apply the registers and some form of cable be it rs485 to rs485 or rs485 to usb.

 

[PhiPhi]

I'm unsure with this particular growwatt and solar assistant but from my own experience via home assistant, I upload an esp32 with the modbus data and any variables, that then translates the modbus to mqtt(not in my case it's transferred directly to my ha API though I can do it via mqtt) the broker receives the information and applies it to my visual dashboard. I also struggle to understand how solar assistant receives the data from the inverter but best guess is you must have to apply the registers and some form of cable be it rs485 to rs485 or rs485 to usb.

Great that was exactly how I imagined, hopefully that means I don't have to worry about the modbus and can focus on learning MQTT for customisation. I can dive into modbus in the future for single sensors like a flow meter or level gauge for my water tanks.

 

[ThaiTaffy]

Great that was exactly how I imagined, hopefully that means I don't have to worry about the modbus and can focus on learning MQTT for customisation. I can dive into modbus in the future for single sensors like a flow meter or level gauge for my water tanks.

That will be some extremely complicated flow meter or level gauge most just output voltage, no can data as far as I'm aware.

 

[wpns]

That will be some extremely complicated flow meter or level gauge most just output voltage, no can data as far as I'm aware.

Not so: https://www.aliexpress.us/item/3256804300342312.html is rs-485 and reads out in mm H2O.

 

[ThaiTaffy]

Not so: https://www.aliexpress.us/item/3256804300342312.html is rs-485 and reads out in mm H2O.

Fair play most of the people I know just use a $2 ultrasonic sensor for water level.

 

[PhiPhi]

Not so: https://www.aliexpress.us/item/3256804300342312.html is rs-485 and reads out in mm H2O.

Nice find! I was just going to stick 4 dumb sensors on a plastic pipe at 500 liter intervals.