An Enphase Ensemble Installation

GXMnow

Photon Sorcerer
Joined
Jul 17, 2020
Messages
1,427
With the Installer Toolkit, I can only see the last 500 log entries. In the whole 500 entries, I do not see any grid instability entries. But I do see a series of entries I have never seen before.

This is repeated for all 16 inverters
Starting download of procload, part number 520-00082-v04.27.04
Completed download of procload, part number 520-00082-r01-v04.27.04

Obviously another new firmware update. The time stamps started at 8:37 am local time and the final completion was at 8:41 am so it only took 4 minutes to do all of the downloads. The downloads say they were done on Tue Mar 09, so this was all this morning here. Looking at my production chart in Enlighten, it does look like some inverters did go offline for a bit. It was pretty sunny and clear, and the production graph looks like a good day after 9 am, but was down a bit for the previous three 15 minute reports. I guess some clouds could have gone by, but it does fit with the time the update was going on.

With only being able to see 500 entries, It would be easy for me to miss things like this. Every evening and again every morning, I get more than 4 reports from each microinverter about the DC power too low set, communication failed, DC power too low clear, etc. But this morning shortly after sun up, but before those downloads, I also see, "Commanded Reset" to all 16 inverters. Going back further in the logs, there is a complete additional set of downloads for all 16 inverters.

Starting download of parm0, part number 540-001410r01-v04.27.09

First one of these started at 8:15 am, and the last ended at 8:18 am. So the files are probably small, but doing 2 steps does explain why the production was down a bit for 45 minutes. So I have now had 2 software updates in about a month. I didn't catch the last update in my 500 entry log, so it was cool to see this one. So now I do want to try the power failure test and see how they handle it. I would expect to see a few still go offline during the transition from grid to inverter, but hopefully they come back on after the 5 minute delay.
 

svetz

Works in theory! Practice? That's something else
Joined
Sep 20, 2019
Messages
4,931
Location
Key Largo
... DC power too low set, communication failed, DC power too low clear, ...
Those are the normal sun-up/sun-down messages as they start to boot-up or shutdown.

...Starting download of procload, part number 520-00082-v04.27.04...
Looks like we're running the same image now:

Status:OK
Last Report:169 W09/03/2021 4:11 PM
Grid Profile:Profile Set (IEEE 1547:2015-WHB)
Part Number:800-00625-r02
Running Image:520-00082-r01-v04.27.04Updated 21/01/2021 4:38 PM


I got this message again: envoy.cond_flags.pcu_ctrl.bricked
Harmless AFAIK, but always scares me.
 
Last edited:

GXMnow

Photon Sorcerer
Joined
Jul 17, 2020
Messages
1,427
That is interesting that you got that update back in Jan, they upgraded me in Feb to a different version, and then in March, I finally got brought up to the version you have. I wonder if Enphase is using the Ensemble customers as beta software testers? It does make some sense. The issue I was having had to do with it switching over from grid to battery backup. I have to assume the change over glitch from grid to battery has to be pretty similar between our systems. Schneider claims an 8ms switching time on the XW-Pro. That is only 1/2 of a cycle at 60 Hz. I don't see how that could be true. Even if it is already driving the sine wave in sync with the grid, it has to see the grid drop out and open a mechanical contactor before it can truly be driving the output alone. I will have to see if I can record the sinewave at the output during my power failure test. I have a PC based Pic Scope which should be able to record a few seconds at high resolution. For safety, I will probably look at the output of a 6 volt transformer. That should still be able to show an accurate waveform without having the full mains power at my test leads.

It is nice to see that Enphase is working on it. Schneider also just issued an update for the XW-Pro, but the release notes do not mention the issue I have with it.
 

svetz

Works in theory! Practice? That's something else
Joined
Sep 20, 2019
Messages
4,931
Location
Key Largo
... I wonder if Enphase is using the Ensemble customers as beta software testers?
Huh? 😜

Guinea pig​


IMG_20190721_172034-2976x2060.jpg
 
Last edited:

svetz

Works in theory! Practice? That's something else
Joined
Sep 20, 2019
Messages
4,931
Location
Key Largo
Enphase installation locations... I'm not on it, so YMMV

Update: If I put my zip code in, I get the image to the right
showing quite a few installations in my area.

Interesting there are so many Enphase installations my area!

1619092855637.png
 
Last edited:

svetz

Works in theory! Practice? That's something else
Joined
Sep 20, 2019
Messages
4,931
Location
Key Largo
One of my batteries started discharging this morning, the others started charging to keep the reserve at 60%. I reset the envoy and opened a ticket, pretty sure that's not supposed to happen. The wayward battery seems to be coming back up, so possibly fixed.
 

GXMnow

Photon Sorcerer
Joined
Jul 17, 2020
Messages
1,427
That is pretty odd. You effectively had one of your batteries pushing it's energy into your other batteries. Not good.

What is your time stamps showing in Enlighten? This morning, we had the DST change to "spring forward" but it looks like Enlighten went forward 2 hours here, not just one. It is logging power production in the future. It is only 8:04 am as I type this, and Enlighten says I made 0.2 KWH's from 8:45 to 9:00 am. Oops. Maybe the DST change caused a glitch in your battery charging setup as well.
 

svetz

Works in theory! Practice? That's something else
Joined
Sep 20, 2019
Messages
4,931
Location
Key Largo
What is your time stamps showing in Enlighten? This morning, we had the DST change to "spring forward" but it looks like Enlighten went forward 2 hours here, not just one.
That's odd. The devices themselves don't care, they use epoch time and programs that display the information are responsible for the timeshift.
For example, when I run the wattage report the timestamp is: "lastReportDate": 1615740153, which if I put into an epoch converter for my timezone gives the current time: Sunday, March 14, 2021 12:42:33 PM. Possibly the timezone is off in your reporting tool?

Mine seem okay, for example the last event was at "Sun Mar 14, 2021 12:14 PM EDT", which is less than an hour (I suppose it could have occurred two hours ago though).

Maybe the DST change caused a glitch in your battery charging setup as well.
The good news is the reset appears to have fixed it, the low one is at 57% and the high ones have come down to 61%. Probably still equalizing.
I doubt they ever tell me, but if they do I'll share the root cause.
 

svetz

Works in theory! Practice? That's something else
Joined
Sep 20, 2019
Messages
4,931
Location
Key Largo
Last edited:

svetz

Works in theory! Practice? That's something else
Joined
Sep 20, 2019
Messages
4,931
Location
Key Largo
An Enphase moderator responded that both of yesterdays issues were from the time change. Both seem okay now for me.

The Envoy reset didn't actually fix it, but at least it reversed one battery from it's downward spiral. You can see in the graph the batteries were whackado for about 12 hours.

Capture.PNG

I upped the SoC to 100%, figure I'll eliminate any microcharging issues, and let the BMS balance the cells for a couple of days.
 
Last edited:

svetz

Works in theory! Practice? That's something else
Joined
Sep 20, 2019
Messages
4,931
Location
Key Largo
Now that my batteries are up to 100%, getting some events I haven't seen before; here's some examples:

Encharge Controller: EnchargeCellBalError : Set/Clear​
Encharge Controller: EnchargeSoCMaintenance : Set/Clear​
Encharge Controller: EnchargeSoCCalibrated​
Encharge Controller: EnchargeConstantVoltageMode​

Guessing it was time to bring them up for a balancing...
 
Last edited:

GXMnow

Photon Sorcerer
Joined
Jul 17, 2020
Messages
1,427
Looks like it did an absorption charge, had to balance up the cells, and then reset the capacity counter to 100% with the full batteries.
 

svetz

Works in theory! Practice? That's something else
Joined
Sep 20, 2019
Messages
4,931
Location
Key Largo
Hmmm... 155 Encharge events today, from 12:56 to 2:05... mainly:
  • EnchargeSoCMaintenance : Set
  • Device is excluded from the aggregate SOC
  • EnchargePCUStateThrottled
  • EnchargePCUFullyOperational
  • EnchargeOnGrid
  • Multi Mode On-Grid
  • EnchargePCUStateGridFault
  • EnchargePCUFault
  • Grid-Tied
  • under-frequency
  • EnchargeIslanding
  • Islanding
  • AC Frequency Out Of Range
No blinking red lights, everythings seems a-okay. I'm thinking the utility was doing some maintenance.... Anyway, set the batteries down to 80% SoC overnight and will ramp them up again tomorrow for a second pass. If all looks good I'll drop them back to 60% until hurricane season starts. up. (Pre-season forecast looks almost like a normal year: 16 named storms, 7 hurricanes, 3 major hurricanes) 🤞 .
 

svetz

Works in theory! Practice? That's something else
Joined
Sep 20, 2019
Messages
4,931
Location
Key Largo
Found a couple of arrays nearby and was able to compare outputs. Looks like the tech has improved a bit over the last 10 years. Really shocked to see Optimizers are 50% of the market...but that's probably more just due to a small sampleset.

Opened a ticket with Enphase (again) about those resetting microinverters. They sent back a picture from Enlighten of the array showing all the panels generating power, so everything was okay. Asked them to dig deeper and provided a list of 60ish JSON events over the last 10 days showing 7 inverters misbehaving repeatedly.
 

GXMnow

Photon Sorcerer
Joined
Jul 17, 2020
Messages
1,427
You have made me curious, so I just signed into my system again to look at the last 500 log entries in Enphase Installer Toolkit.

Just after 7 AM I see 16 x 3 messages of them all starting up when the sun comes out.
Failed to report clear
Power on reset
DC voltage too low clear

Before those it was the 16 x 2 of the DC power too low and failed to report

All as expected, and repeats each night and morning. That is until I go back to March 14th. I scrolled down to the first odd entry. At 12:49 PM one inverter reported AC Frequency Out Of Range: Set. Followed by Grid Instability: Set. Then, in the same minute the same inverter reported AC Frequency Low: Set, then AC Frequency High: Set. It looks like these 4 entries repeat for all 16 inverters. Then at 12:54 5 minutes later, all 16 report Grid Instability: Clear

At first glance, I would say we had a power glitch that upset the grid and made the frequency bounce. Looking at my Enphase production, it is a bit low in the 15 minute slice. That 15 minute slice averaged 2.7 KW and the slice after it average 3.5 KW. That is a pretty big difference, so the inverters could have shut down for it. I took a look in the Schneider logs on the XW-Pro for the same time, and there are no entries of any grid issues. At this time of day, the XW was just in charge mode, not inverting. So it is possible it would not even bother checking for grid frequency at that time, so I am not sure if was just not detected, or it was an Enphase thing only. The XW did report an over frequency event, but it was back in Feb. And that did occur while it was inverting and exporting power to the grid.

I will try to watch this a bit closer and see if anything happens again. I would expect the Schneider XW to be the pickier unit right now, as it is in the California Rule 21 mode while my Enphase iQ7's are still running in "IEEE 1547 Mainland alternate" still. I wanted to try a power fail test in this mode before I change them back to Rule 21 just to see if there is a difference when it switches from grid to battery.

Rule 21 does have a few odd requirements. And as I am trying to read and understand it, maybe it is actually LESS likely to cause a shutdown. Most of the changes from the IEEE 1547 standard is intended to help the grid and make it more stable. The last thing they want is a ton of distributed solar power all going offline at the same time. So if the grid goes too high or too low, on frequency or voltage, they want the solar generation to smoothly scale to bring it back in line. If the grid frequency falls due to a ton of load slowing a turbine, and the frequency drop causes thousands of grid tie solar inverters to shut off, then the load on the turbine spikes up much worse. Rule 21 keeps solar inverters online even with the grid further off spec, but it ramps them in the direction needed to help bring it back in spec. Low voltage and frequency are nearly ignored to help keep the grid up. Power ramping down happens fairly quick though as the frequency goes up, and also with increasing voltage. So maybe the IEEE 1547 iQ7's saw a grid instability shut down, but the Rule 21 XW-Pro was in ride through.
 

svetz

Works in theory! Practice? That's something else
Joined
Sep 20, 2019
Messages
4,931
Location
Key Largo
Wow! Noticed the thread's had 9,000 views! Time for a new recipe reward!

Click the image to get Alton's pop-tart recipie. Easy to make, inexpensive, and tasty!

Spent some time reviewing/cleaning the thread today. Mostly deleting non-useful posts like, "when will the Encharge batteries arrive?", "are they here yet?" ;-)

Noticed in #183 I reported the Enlighten numbers didn't match up exactly with the meter. But I had just checked this the other day and the numbers looked really good, so I'm guessing something got patched.

1616759973429.pngExciting to see my production go over my consumption. It should stay above until May.

I always knew the system wouldn't meet 100% of my needs all year long, but was
expecting more output in the fall (which I didn't get due to rain).

Could be anomaly, but I'll probably add some more panels when the IQ8s are available.
 

svetz

Works in theory! Practice? That's something else
Joined
Sep 20, 2019
Messages
4,931
Location
Key Largo
Had an idea to test if the microinverter issues were grid-related or not. Simplicity in itself, see what happens to them when running off-grid. If the resets occur, then it's not related to the grid. I was going to do it this morning, but we were too overcast.
 

svetz

Works in theory! Practice? That's something else
Joined
Sep 20, 2019
Messages
4,931
Location
Key Largo
Ever wonder how efficient your fixed panels are when the sun is at low angles to their fixed-tilt?
From my raw data last summer at solstice the sun is essentially straight up ( 88°). At solar noon I'm at 100% power, and four hours earlier (30° sun altitude), I'm at 50% power. The sun-angle-to-power chart suggests with atmospheric effects I should be at 50% max power at 30°; so that says my LG Neons are freaking awesome with light hitting them at 30°. Or, it could just be it was a clear morning and hazy solar noon. ;-)
1617743061732-png.44055
 

svetz

Works in theory! Practice? That's something else
Joined
Sep 20, 2019
Messages
4,931
Location
Key Largo
Had an idea to test if the microinverter issues were grid-related or not. Simplicity in itself, see what happens to them when running off-grid. If the resets occur, then it's not related to the grid. I was going to do it this morning, but we were too overcast.
Did the test this morning... what a nightmare.

All of the microinverters started flip-flopping on/off.

To the right, you can see the two states, every
5 min on then off.

So definitely not all is well. The IQ8s are
supposed to throttle back the inverters,
not start flipping them all off and on. I
suspect the January "fix" to the grid profile
being reset at frequency changes busted this.

No new news/conclusions on my 6 ill-behaved microinverters
1617811847385.png1617812169181.png


Called Enphase, after a couple of hours they said it was because the battery was approaching full charge. I explained it had been doing it since the start of the test when the battery was at 60%, so they suggested switching from savings mode to full backup.

That seems to have made the battery want to go higher, but it's still on/off rather than throttle. Hopefully, this is a one-off. I don't recall seeing this behavior before when going off-grid. Sure hope they get it fixed before hurricane season.
 
Top