Features for the Perfect Monitoring Software for a Residential System

[svetz]

The goal is to only to determine if the panels have gotten soiled to the point they need cleaning (individual panel issues would be detected by the method in #17). So, sensor accuracy isn't as important as sensor consistency.

As long as the sensor's data is consistently relative to the panel's output, then it should be able to be used to determine if the overall output has dropped below a desirable threshold. At least that's the theory. To test it with the wunderground data I'll need to collect a lot of data and see how much variance there is.

Of course, if the sensor is getting soiled at the same rate as the panels then it's not going to do much good. @jasonhc73 has had his weather station for a while now, perhaps he can comment on that? I'm guessing if it has a bubble top like a Pyranometer it'll probably stay relatively clean.

 

[WoodsieLord]

I can give you some ideas of what *I* would like:
- I would like snmp output to gather historic data.
- multiple temp sensors. I would like to attach one to the battery bank and one to the back side of one of my panels.
- if you want to detect dirt on the glass surface, I would suggest you experiment with a light source (led? laser diode?) and a light detector that can output analog info (you can use an arduino for analog input). I would aim the source so it must reflect on the panel's glass to reach the detector. Then you can monitor (with time) what readings you get with AND without dirt. Then you can establish some thresholds to pinpoint when the panels are dirty. Never tried this though!

I don't know if you're planning to make an "all in one" solution. If you're willing to rely on something else, I can recommend a software that I use at work and at home.
Paessler PRTG network monitor. It's meant for network and servers, but I've been using it for various raspberry pi projects. Including a mini greenhouse monitoring (and watering system) some time ago... And now, my solar mini project. Worth mentioning that the software is free up to 100 sensors (it's a lot for DIY projects)

What I do is extend python scripts to the snmp daemon, then I can gather the information from the monitoring tool. That way I can collect historical data. This is my simplified version of your truly awesome project:


Please note that this is a work in progress. Craig has been helping me quite a bit with a chip that reads shunts.
(Once I get my panels installed and some tests, I will update my post). In the screenshot you can observe that today we had a power outage in the neighborhood. What you see is the battery bank becoming fully charged by the UPS (no panels yet)

Regards!

 

[jasonhc73]

The goal is to only to determine if the panels have gotten soiled to the point they need cleaning (individual panel issues would be detected by the method in #17). So, sensor accuracy isn't as important as sensor consistency.

As long as the sensor's data is consistently relative to the panel's output, then it should be able to be used to determine if the overall output has dropped below a desirable threshold. At least that's the theory. To test it with the wunderground data I'll need to collect a lot of data and see how much variance there is.

Of course, if the sensor is getting soiled at the same rate as the panels then it's not going to do much good. @jasonhc73 has had his weather station for a while now, perhaps he can comment on that? I'm guessing if it has a bubble top like a Pyranometer it'll probably stay relatively clean.

Outdoor wifi camera. Not techy, but effective.

 

[svetz]

Outdoor wifi camera. Not techy, but effective.

There is a beauty in it's simplicity!

 

[svetz]

I can give you some ideas of what *I* would like:...

Ideas are always good! Thank you for posting that!

... I would like snmp output to gather historic data.

SNMP would require yet another tool to decode/use the alerts/traps. It would be great if you were already using an SNMP monitor for something. I'm thinking along the lines that folks just want their solar system to work and not be bothered by it... unless something is a problem, in which case they want a notification via things they normally use (e.g., email). I might be a solar nut, but I don't want to become a slave to it... my computer monitoring it is fine by me.

That said, it shouldn't be hard to replace the call to sendMail with something like sendAlert. I know of at least one free SNMP java library.

... multiple temp sensors.

That's hardware rather than software. Ideally the software would be able to interface to however they worked. The IoT has a ton of cheap sensors usually motion/temperature. For example you could attach SmartThings temperature sensors and have the monitoring software collect data from the SmartThings hub. But, all hardware dependent.

I haven't really started thinking about the batteries yet (mine haven't arrived yet), so any ideas as to what should be recorded, what to alert on, etc. would be useful. For me I'd like a log file somewhere with historical data I can go back to...but otherwise really only want to be notified when I need to take action (e.g., cut down a tree, clean the panels, replace a microinverter).

... if you want to detect [soiling]...

Good ideas there, but I'm still seeing if it can be done just using the panel outputs and a reference as mentioned previously. If it can be it's cheaper and involves fewer things that can go wrong.

... I don't know if you're planning to make an "all in one" solution...

Not really sure what I'll end up with.

... I can recommend ... Paessler PRTG network monitor.

That looks pretty cool, thanks for letting me know about it. Right now it's more a DIY challenge to see what can be done in a minimalist sort of way.

 

[svetz]

The data correlation between the Wunderground weather station 4 miles away is startlingly good in terms of predicting the panel production most of the time.

Sadly, there are times clouds go over one but not the other, making for very bad data indeed. For example:

Wunderground PWS data
          Timestamp   Watts,  UV, Temperature, Dewpoint, Humidity, Pressure,  Wind,  Wind Gust, Direction
03/17/2020 12:50:53  178.20,   2,          82,     72.7,       74,    30.22,   4.7,        6.9,       NNW

                           efficiency %
Solar Panel data Watts  %actual / %expected
panel 1 (west) : 269     (96.2% / 21.4%)
panel 2 (west) : 268     (95.9% / 21.4%)

So, in this case, a cloud blocked the sunlight from the sensor, but no cloud over the panels. So, the power from the panels / total meter-measured powered is a tad high from the expected values.

A more local one (e.g., on my roof) might work. The PWS the data came from is an Ambient Weather WS-2902 and there's an API for it so you wouldn't need an internet (only an intranet). But, a soiling check could also take dozens of measurements over a few days - it's only a problem if all are bad... that is one good one means the panels are okay.

 

[boondox]

It is easy enough to throw out any obviously bad data. Since dirt will result in a small drop in output and will occur over a long period of time massaging the data to eliminate the clouds and what not shouldn't be too hard.

 

[svetz]

See Panel or Inverter Problem for a confirmation on why its a good idea to either monitor or have a system monitoring your solar array and associated devices. I'll watch BobE's thread to see if there isn't something to be learned that should be added to the monitoring system.

 

[svetz]

Attached is a new version with the soiling alert added plus some tweaks... Keep in mind this is just example java source if you want to do something similar and is only meant as an example ... it's not runnable without configuration files and other programs (e.g., rtlamr). It's also not particularly well tuned (e.g., how many sampling failures does it take to have a good reliability that there's soiling on the panels). Starting class is NetMeterReader and starting method main. The other classes are utility classes probably/hopefully self-explanatory by name. The algorithms follow the thread discussion.

 

[jasonhc73]

PWS Weather now has a new "Beta" page. It's kind of pretty...

https://beta.pwsweather.com/station/pws/jasonhc73

 

[svetz]

...PWS Weather now has a new "Beta" page. It's kind of pretty...

Thanks for the update!

Pretty, but they seem to have dropped the solar radiation as a number. That might mean you need your own meter with an API for it to work.
It also points out the weakness of parsing the html rather than using an API, last week's code becomes obsolete.

 

[jasonhc73]

Thanks for the update!

Pretty, but they seem to have dropped the solar radiation as a number. That might mean you need your own meter with an API for it to work.
It also points out the weakness of parsing the html rather than using an API, last week's code becomes obsolete.

PwsWeather never did have the solar data.
Wunderground does though.

 

[svetz]

PwsWeather never did have the solar data....Wunderground does though.

My bad... thought it was an update for Wunderground...

 

[svetz]

Last couple of days I've been getting, and ignoring, the soiled panel alert. I figured it was a case of the tolerances being too tight. I'm sure they are, but I went up to the roof to check and sure enough, the local trees had deposited a fine layer of pollen and the last couple of inches were particularly laden (probably from a very light rain we had). To hot to rinse them off now, but I'll do it tomorrow morning.

 

[KBWaldron]

These are the reasons why in the tracking array I mentioned that we decided to go for a date/time basis for orienting the panels. We found the dance of the planet much more reliable and easy to work with than sensors.

Agreed. I too am using date time, and latitude for tracking. Fortunately I only have to track altitude and not azimuth. The latter is hard to do on a trailer.

 

[Fred S]

Real solar radiation sensors aren't cheap. There are plenty of DIY options out there, light dependent resistors, LEDs with the top filed off, temperature sensors in a jar, all kinds of ways. As you would expect, there are pros and cons to each. Luckily you are not attempting a tracking array which increases the complexity. I do think that a local sensor will be required, especially if you want more than total failure detection. If you are trying to monitor long term panel degradation that will be much harder. . . .

I don't know if it's been covered before, didn't see it, but I have a small 5w panel out with my main array which is loaded with a 200 ohm resistor. This gives me a fairly linear voltage vs insolation. If you made that panel accessible, and kept it clean, would that work as a cheap local radiation sensor?

 

[apctjb]

• How much power the panels are producing
• How much power is being exported to the grid
• The state of the battery
• Duration of battery at current consumption
• Time until the Battery is recharged
• Alert if there are any problems with a panel (e.g., under producing for conditions - that is it's sunny but not getting the expected power)
• API to the smart panel

Broader question is who is this intended for; layman with casual interest or serious hobbyist. In my experience people with monitoring systems on there PV systems lose interest overtime and the more data/complexity, the faster that happens. So if this is something intended for a broader audience best to keep it simple. Array output over time is a useful indicator for spotting problems. MoM and YoY comparisons are great. Good luck!

 

[svetz]

... If you made that panel accessible, and kept it clean, would that work as a cheap local radiation sensor?

Sure! A lot of weather stations have little solar panels on them to both power the station and report solar irradiance. Some use photo-diodes. But as @boondox points out, solar panels typically lose a couple % power in their first year and then a bit after that each year, so accuracy drifts. Pyranometers don't degrade as the measurement is the temperature of a black-coated material in a vacuum; but because the vacuum deters thermal radiation they're also slow.

This project isn't really looking for "accurate" solar irradiation numbers and has no fancy reports. It's using "relative" values so it can send an alert when there's a problem (e.g., panels are soiled and need cleaning). As @apctjb says, I have no interest in being a slave to the monitoring of a PV system. For me, monitoring's job is to tell me when there's a problem and I need to do something. Otherwise I don't generally make use of the data that's being collected (okay, yeah... I have posted some cool things I found in my system blog).

 

[apctjb]

A lot of weather stations have little solar panels on them to both power the station and report solar irradiance

Yup, reference cells or modules are fine. If a system has multiple strings then comparison of strings also is super useful in spotting problems.

 

[Fred S]

Yeah, I guess ideally the "insolation panel" would need to be recalibrated from time to time, to account for degradation. Well, my calibration will be to just note the highest voltage ever seen in bright sunshine, and call it the new max.

If a station is using the same panel for irradiation checking and for powering components, how does that work? Is the power draw perfectly constant, so as not to affect panel voltage?