diy solar

diy solar

Renogy Devices and Raspberry Pi - bluetooth/WiFi

thanks, I tried resinstalling via

pip3 install -r requirements.txt

I commened out the ones it says are already present, but there a couple of them that it doesn't like

json
threading

do I have install these manually?

it says this for both

ERROR: Could not find a version that satisfies the requirement json
ERROR: No matching distribution found for json
Json and threading were already present for me I think. Broker is your MQTT server, it's not necessary to get started just leave that part commented out until you figure out where you want your stats to go.
 
Also what @aps1602 said is correct, make sure you've taken the .dist extension off of the ini file and edited it according to what you need. It's essentially an example meant to copy from so you don't get errors for things not present in your environment.
 
Last edited:
I'll try to retype the instructions here tomorrow, including updates for the latest commit, and stuff I missed originally like moving or copying the example config. It was always my intention to update the original draft after answering questions because I knew I would miss a few steps. I'll start a new forum topic and link to it from here so that all of my side tracked questions and the original help for serial comms can be separate.
 
I'll try to retype the instructions here tomorrow, including updates for the latest commit, and stuff I missed originally like moving or copying the example config. It was always my intention to update the original draft after answering questions because I knew I would miss a few steps. I'll start a new forum topic and link to it from here so that all of my side tracked questions and the original help for serial comms can be separate.
Thanks that would be so much appreciated and helpful!
 
Could you ensure to mention all the python add-ons that need to be installed when using a vanilla pi where only the basic python3 is installed.
This morning my device is in a constant reset loop, caused when restarting the service after a power loss.
 
thanks @oedo808 and @aps1602 I got it to work on my Pi Zero 2 W Bullseye! After installing the libraries and dependencies, it was the dist issue. I commented out the mqtt part...but I think it stil complains a bit about it (but doesn't exit this code and just keeps reading values from the BT-1) and spits out a couple of lines like below while its reading the values from the BT-1.

How do I set up the data logging/history and visuals for plots? Do I need to make an account with a 3rd party server or can it be local data storage?

ERROR: Exception in handler for D-Bus signal:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/dbus/connection.py", line 232, in maybe_handle_message
self._handler(*args, **kwargs)
File "/home/pi/.local/lib/python3.9/site-packages/gatt/gatt_linux.py", line 398, in properties_changed
self.services_resolved()
File "/home/pi/solar-monitor/solardevice.py", line 187, in services_resolved
self.datalogger.mqtt.trigger[self.logger_name] = self.command_trigger
AttributeError: 'NoneType' object has no attribute 'trigger'
 
Last edited:
Could you ensure to mention all the python add-ons that need to be installed when using a vanilla pi where only the basic python3 is installed.
This morning my device is in a constant reset loop, caused when restarting the service after a power loss.
good point, for sure there were more than the ones listed in the requirements list, but I only figured them out after trying to install them, it would tell me missing module etc. so I would install them....off the top of my head, this is what I remember, there were maybe 6 or 7 extra ones I had to add

pip (or pip3)
python3-setuptools
npm
 
thanks @oedo808 and @aps1602 I got it to work on my Pi Zero 2 W Bullseye! After installing the libraries and dependencies, it was the dist issue. I commented out the mqtt part...but I think it stil complains a bit about it (but doesn't exit this code and just keeps reading values from the BT-1) and spits out a couple of lines like below while its reading the values from the BT-1.

How do I set up the data logging/history and visuals for plots? Do I need to make an account with a 3rd party server or can it be local data storage?

ERROR: Exception in handler for D-Bus signal:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/dbus/connection.py", line 232, in maybe_handle_message
self._handler(*args, **kwargs)
File "/home/pi/.local/lib/python3.9/site-packages/gatt/gatt_linux.py", line 398, in properties_changed
self.services_resolved()
File "/home/pi/solar-monitor/solardevice.py", line 187, in services_resolved
self.datalogger.mqtt.trigger[self.logger_name] = self.command_trigger
AttributeError: 'NoneType' object has no attribute 'trigger'

So I'm using MQTT, sending to OpenHab and persisting that to InfluxDB and Grafana is reading the data from InfluxDB to chart. There are multiple options to get a similar configuration, for instance I think everything is integrated with HomeAssistant if you went that direction.

There is a logger option in solar-monitor for sending data to a URL via JSON. I don't see any documentation about any specific service it uses and I'm only seeing binary data that can be written to influx over http: https://docs.influxdata.com/influxdb/v1.8/guides/write_data/

Ultimately your goal is to get this data into a database that is compatible with Grafana. It looks to me like you can probably take several steps out altogether and:
  1. Install an MQTT server like Mosquitto somewhere; could even be on the Pi Zero W as long as it doesn't end up consuming too many resources
  2. Configure solar-monitor to connect to MQTT
    • If you're on Windows you can verify the data is logging with something like MQTT Explorer rather than troubleshooting with the CLI tools on the MQTT host
  3. Install Grafana somewhere, could also be on the Pi Zero W. Keep in mind that this is not have as many resources as something like a Pi4 with extra RAM, but to get started with just this one project I can absolutely see you installing all of this on the Pi Zero W
  4. Install and configure this mqtt-datasource plugin for Grafana: https://github.com/grafana/mqtt-datasource

There are many ways you can accomplish your goal, this just seems like a lighter weight solution that I'm more familiar with.

Mosquitto: https://www.instructables.com/Installing-MQTT-BrokerMosquitto-on-Raspberry-Pi/
Grafana: https://grafana.com/tutorials/install-grafana-on-raspberry-pi/
- MQTT Grafana plugin: https://github.com/grafana/mqtt-datasource
 
thanks , I will likely want to go the route of putting as less as possible on the Pi Zero because its already kinda of struggling with just this. So, I will have to setup what you have basically, " use MQTT, sending to OpenHab and persisting that to InfluxDB and Grafana is reading the data from InfluxDB to chart."
 
thanks , I will likely want to go the route of putting as less as possible on the Pi Zero because its already kinda of struggling with just this. So, I will have to setup what you have basically, " use MQTT, sending to OpenHab and persisting that to InfluxDB and Grafana is reading the data from InfluxDB to chart."
Openhab is Java based and not what I would consider lightweight although I'm running it on a Pi3 Model B without any issue. I would recommend trying the Mosquitto -> Grafana route as you'll need both of those plus InfluxDB if you went with Openhab.

Start by doing the four numbered steps I mentioned. If it doesn't work correctly you can use the existing installs if you need to use Openhab or HomeAssistant with InfluxDB.

Openhab or HomeAssistant is what you would want if you were trying to go down the home automation route. I already had Openhab for working with ZWave and other MQTT IoT devices like smart switches. I installed InfluxDB and Grafana a few years back to chart an ESP8266 based Temp/Humidity sensor I put together for my cigar humidor and have been using it for other things since. That's how I was up and running with logging and charting the same day I found the solar-monitor project.

If you wanted to be ready for home automation projects I recommend picking up a separate Pi, probably a Pi 4 with 4 to 8GB of RAM to make sure you have plenty of room for all of the individual components you need. I've had to upgrade a couple of times over the past 5 years due to limited resources and with Openhab it's a pain. HomeAssistant is more popular now and if I were starting today I would go with it probably, but I prefer writing in Java over Python so maybe not, although I've written exactly 0 plugins for OpenHab this whole time.
 
Last edited:
Openhab is Java based and not what I would consider lightweight although I'm running it on a Pi3 Model B without any issue. I would recommend trying the Mosquitto -> Grafana route as you'll need both of those plus InfluxDB if you went with Openhab.

Start by doing the four numbered steps I mentioned. If it doesn't work correctly you can use the existing installs if you need to use Openhab or HomeAssistant with InfluxDB.

Openhab or HomeAssistant is what you would want if you were trying to go down the home automation route. I already had Openhab for working with ZWave and other MQTT IoT devices like smart switches. I installed InfluxDB and Grafana a few years back to chart an ESP8266 based Temp/Humidity sensor I put together for my cigar humidor and have been using it for other things since. That's how I was up and running with logging and charting the same day I found the solar-monitor project.

If you wanted to be ready for home automation projects I recommend picking up a separate Pi, probably a Pi 4 with 4 to 8GB of RAM to make sure you have plenty of room for all of the individual components you need. I've had to upgrade a couple of times over the past 5 years due to limited resources and with Openhab it's a pain. HomeAssistant is more popular now and if I were starting today I would go with it probably, but I prefer writing in Java over Python so maybe not, although I've written exactly 0 plugins for OpenHab this whole time.
Unfortubnalt,y this last step, "yarn install" (after following the long list of prerequistes) won't run on my pi for memory issues. Is there a way to limit resources for this?

RPi memory.jpg
 
Unfortubnalt,y this last step, "yarn install" (after following the long list of prerequistes) won't run on my pi for memory issues. Is there a way to limit resources for this?

View attachment 77481
I'm not sure. Do you keep a desktop running? You could run docker containers with whatever you can't run on the Pi. My Zero W is basically dedicated to the Renogy and I have other resources to handle the rest.
 
I see, my zero is lite also (not desktop). Did you install the same yarn plug-in on your zero?
 
I don't, I probably did on my Pi 3b though which is where I have Grafana running. Your screenshot is from a Windows computer, you should probably install Grafana on it, preferably in a docker container https://grafana.com/docs/grafana/latest/installation/docker/

Do you run your windows computer 24/7 or do you just boot it when you need it?that's
That's a great idea! I have an old but powerful quad core 2.7 GHz computer that I can use for this. It has windows VISTA, though. I'll give it a try!
 
so, I did one better. I converted that old computer to a Ubuntu machine. So, now I can have it one 24/7 to hepl with the resouces. I just have to figure out how to put all the peices together.
Back to your 4 step instuctions:

  1. Install an MQTT server like Mosquitto somewhere; could even be on the Pi Zero W as long as it doesn't end up consuming too many resources
  2. Configure solar-monitor to connect to MQTT
    • If you're on Windows you can verify the data is logging with something like MQTT Explorer rather than troubleshooting with the CLI tools on the MQTT host
  3. Install Grafana somewhere, could also be on the Pi Zero W. Keep in mind that this is not have as many resources as something like a Pi4 with extra RAM, but to get started with just this one project I can absolutely see you installing all of this on the Pi Zero W
  4. Install and configure this mqtt-datasource plugin for Grafana: https://github.com/grafana/mqtt-datasource
What steps are not installed on the Ubuntu machine vs Pi Zero?

I presume step 4 will be the Ubuntu machine...
 
Last edited:
Right the Mac is the BT-1 and the Broker is my MQTT Broker on my OpenHab system
 
Great! I got the Broker running on the Ubuntu machine by following these instructions:


Then, I installed two simple sub-pub scripts from here on the Pi Zero:


Using the above two scripts as is (public broker) works as described. But changing the broker address to my local Ubuntu machine address, i get connection refused errors running the same scripts above:

pi@raspberrypi:~ $ python3 sub.py
Traceback (most recent call last):
File "/home/pi/sub.py", line 46, in <module>
run()
File "/home/pi/sub.py", line 40, in run
client = connect_mqtt()
File "/home/pi/sub.py", line 27, in connect_mqtt
client.connect(broker, port)
File "/home/pi/.local/lib/python3.9/site-packages/paho/mqtt/client.py", line 914, in connect
return self.reconnect()
File "/home/pi/.local/lib/python3.9/site-packages/paho/mqtt/client.py", line 1044, in reconnect
sock = self._create_socket_connection()
File "/home/pi/.local/lib/python3.9/site-packages/paho/mqtt/client.py", line 3685, in _create_socket_connection
return socket.create_connection(addr, timeout=self._connect_timeout, source_address=source)
File "/usr/lib/python3.9/socket.py", line 843, in create_connection
raise err
File "/usr/lib/python3.9/socket.py", line 831, in create_connection
sock.connect(sa)
socket.timeout: timed out

both devices, I can ssh to eachther without a password, except I need to hostname include it, for examaple

$ssh hostname@ip

no password is required to ssh between computes.

For some reason, I cannot stop or restart the broker on my Ubunut machine, I don;t know if this part of my connection issues

sudo service mosquitto stop
sudo systemctl stop mosquitto.service

When I try to turn it back on via this:

mosquitto -v #start in verbose mode

it says that the port 1883 is alreadty in use, so its running I guess, but it might be hung?

It works once, where when I ran the above script, it says "Conneted to Broker!!!" but after that, it starting giving me connection refused issues, even runnint the script with sudo

It shouldn't be this hard to run a local broker? is there some mosquitto options I have to selected in the config file?
 
so, I got the broker working on my local Ubuntu computer and network (it had something to do with specifically giving permission to external devices by adding the IP address of the external device in the broker moquitto.conf file on the UBuntu computer)....I ran a basic mqtt publish and subcribe on the RPi Zero with the broker IP address and it works.

ok now that thats working, it seems the solar-monitor script is not liking the IP address vs address (name) Because when I change the configuration to

[mqtt]
broker = 198.168.0.68 # Address to the mqtt broker
prefix = solar # All topics are prefixed by "prefix/"
# E.g. prefix/battery_1/voltage

I get such messsages like this:

pi@raspberrypi:~/solar-monitor $ python3 solar-monitor.py
Traceback (most recent call last):
File "/home/pi/solar-monitor/solar-monitor.py", line 48, in <module>
datalogger = DataLogger(config)
File "/home/pi/solar-monitor/datalogger.py", line 184, in __init__
self.mqtt = DataLoggerMqtt(config.get('mqtt', 'broker'), 1883, prefix=config.get('mqtt', 'prefix'), username=config.get('mqtt', 'username'), password=config.get('mqtt', 'password'), hostname=config.get('mqtt', 'hostname'))
File "/home/pi/solar-monitor/datalogger.py", line 29, in __init__
self.client.connect(broker, port) # establish connection
File "/home/pi/.local/lib/python3.9/site-packages/paho/mqtt/client.py", line 914, in connect
return self.reconnect()
File "/home/pi/.local/lib/python3.9/site-packages/paho/mqtt/client.py", line 1044, in reconnect
sock = self._create_socket_connection()
File "/home/pi/.local/lib/python3.9/site-packages/paho/mqtt/client.py", line 3685, in _create_socket_connection
return socket.create_connection(addr, timeout=self._connect_timeout, source_address=source)
File "/usr/lib/python3.9/socket.py", line 822, in create_connection
for res in getaddrinfo(host, port, 0, SOCK_STREAM):
File "/usr/lib/python3.9/socket.py", line 953, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -2] Name or service not known

If you have any thoughts on this one, that would be great.
 
Does anyone know if you can change the resolution of the data messages sent over MQTT.
I'm getting values that seem to be integers except for voltages. So for example power (for load power) is either 1 or 2 when I know on the app it would read 1.54W for example.
 
Back
Top