An Enphase Ensemble Installation

svetz

Works in theory! Practice? That's something else
Staff member
Moderator
An Enphase moderator responded that both of yesterdays issues were from the time change. Both seem okay now for me.

The Envoy reset didn't actually fix it, but at least it reversed one battery from it's downward spiral. You can see in the graph the batteries were whackado for about 12 hours.

Capture.PNG

I upped the SoC to 100%, figure I'll eliminate any microcharging issues, and let the BMS balance the cells for a couple of days.
 
Last edited:

svetz

Works in theory! Practice? That's something else
Staff member
Moderator
Now that my batteries are up to 100%, getting some events I haven't seen before; here's some examples:

Encharge Controller: EnchargeCellBalError : Set/Clear​
Encharge Controller: EnchargeSoCMaintenance : Set/Clear​
Encharge Controller: EnchargeSoCCalibrated​
Encharge Controller: EnchargeConstantVoltageMode​

Guessing it was time to bring them up for a balancing...
 
Last edited:

GXMnow

Photon Sorcerer
Looks like it did an absorption charge, had to balance up the cells, and then reset the capacity counter to 100% with the full batteries.
 

svetz

Works in theory! Practice? That's something else
Staff member
Moderator
Hmmm... 155 Encharge events today, from 12:56 to 2:05... mainly:
  • EnchargeSoCMaintenance : Set
  • Device is excluded from the aggregate SOC
  • EnchargePCUStateThrottled
  • EnchargePCUFullyOperational
  • EnchargeOnGrid
  • Multi Mode On-Grid
  • EnchargePCUStateGridFault
  • EnchargePCUFault
  • Grid-Tied
  • under-frequency
  • EnchargeIslanding
  • Islanding
  • AC Frequency Out Of Range
No blinking red lights, everythings seems a-okay. I'm thinking the utility was doing some maintenance.... Anyway, set the batteries down to 80% SoC overnight and will ramp them up again tomorrow for a second pass. If all looks good I'll drop them back to 60% until hurricane season starts. up. (Pre-season forecast looks almost like a normal year: 16 named storms, 7 hurricanes, 3 major hurricanes) 🤞 .
 

svetz

Works in theory! Practice? That's something else
Staff member
Moderator
Found a couple of arrays nearby and was able to compare outputs. Looks like the tech has improved a bit over the last 10 years. Really shocked to see Optimizers are 50% of the market...but that's probably more just due to a small sampleset.

Opened a ticket with Enphase (again) about those resetting microinverters. They sent back a picture from Enlighten of the array showing all the panels generating power, so everything was okay. Asked them to dig deeper and provided a list of 60ish JSON events over the last 10 days showing 7 inverters misbehaving repeatedly.
 

GXMnow

Photon Sorcerer
You have made me curious, so I just signed into my system again to look at the last 500 log entries in Enphase Installer Toolkit.

Just after 7 AM I see 16 x 3 messages of them all starting up when the sun comes out.
Failed to report clear
Power on reset
DC voltage too low clear

Before those it was the 16 x 2 of the DC power too low and failed to report

All as expected, and repeats each night and morning. That is until I go back to March 14th. I scrolled down to the first odd entry. At 12:49 PM one inverter reported AC Frequency Out Of Range: Set. Followed by Grid Instability: Set. Then, in the same minute the same inverter reported AC Frequency Low: Set, then AC Frequency High: Set. It looks like these 4 entries repeat for all 16 inverters. Then at 12:54 5 minutes later, all 16 report Grid Instability: Clear

At first glance, I would say we had a power glitch that upset the grid and made the frequency bounce. Looking at my Enphase production, it is a bit low in the 15 minute slice. That 15 minute slice averaged 2.7 KW and the slice after it average 3.5 KW. That is a pretty big difference, so the inverters could have shut down for it. I took a look in the Schneider logs on the XW-Pro for the same time, and there are no entries of any grid issues. At this time of day, the XW was just in charge mode, not inverting. So it is possible it would not even bother checking for grid frequency at that time, so I am not sure if was just not detected, or it was an Enphase thing only. The XW did report an over frequency event, but it was back in Feb. And that did occur while it was inverting and exporting power to the grid.

I will try to watch this a bit closer and see if anything happens again. I would expect the Schneider XW to be the pickier unit right now, as it is in the California Rule 21 mode while my Enphase iQ7's are still running in "IEEE 1547 Mainland alternate" still. I wanted to try a power fail test in this mode before I change them back to Rule 21 just to see if there is a difference when it switches from grid to battery.

Rule 21 does have a few odd requirements. And as I am trying to read and understand it, maybe it is actually LESS likely to cause a shutdown. Most of the changes from the IEEE 1547 standard is intended to help the grid and make it more stable. The last thing they want is a ton of distributed solar power all going offline at the same time. So if the grid goes too high or too low, on frequency or voltage, they want the solar generation to smoothly scale to bring it back in line. If the grid frequency falls due to a ton of load slowing a turbine, and the frequency drop causes thousands of grid tie solar inverters to shut off, then the load on the turbine spikes up much worse. Rule 21 keeps solar inverters online even with the grid further off spec, but it ramps them in the direction needed to help bring it back in spec. Low voltage and frequency are nearly ignored to help keep the grid up. Power ramping down happens fairly quick though as the frequency goes up, and also with increasing voltage. So maybe the IEEE 1547 iQ7's saw a grid instability shut down, but the Rule 21 XW-Pro was in ride through.
 

svetz

Works in theory! Practice? That's something else
Staff member
Moderator
Wow! Noticed the thread's had 9,000 views! Time for a new recipe reward!

Click the image to get Alton's pop-tart recipie. Easy to make, inexpensive, and tasty!

Spent some time reviewing/cleaning the thread today. Mostly deleting non-useful posts like, "when will the Encharge batteries arrive?", "are they here yet?" ;-)

Noticed in #183 I reported the Enlighten numbers didn't match up exactly with the meter. But I had just checked this the other day and the numbers looked really good, so I'm guessing something got patched.

1616759973429.pngExciting to see my production go over my consumption. It should stay above until May.

I always knew the system wouldn't meet 100% of my needs all year long, but was
expecting more output in the fall (which I didn't get due to rain).

Could be anomaly, but I'll probably add some more panels when the IQ8s are available.
 

svetz

Works in theory! Practice? That's something else
Staff member
Moderator
Had an idea to test if the microinverter issues were grid-related or not. Simplicity in itself, see what happens to them when running off-grid. If the resets occur, then it's not related to the grid. I was going to do it this morning, but we were too overcast.
 

svetz

Works in theory! Practice? That's something else
Staff member
Moderator
Ever wonder how efficient your fixed panels are when the sun is at low angles to their fixed-tilt?
From my raw data last summer at solstice the sun is essentially straight up ( 88°). At solar noon I'm at 100% power, and four hours earlier (30° sun altitude), I'm at 50% power. The sun-angle-to-power chart suggests with atmospheric effects I should be at 50% max power at 30°; so that says my LG Neons are freaking awesome with light hitting them at 30°. Or, it could just be it was a clear morning and hazy solar noon. ;-)
1617743061732-png.44055
 

svetz

Works in theory! Practice? That's something else
Staff member
Moderator
Had an idea to test if the microinverter issues were grid-related or not. Simplicity in itself, see what happens to them when running off-grid. If the resets occur, then it's not related to the grid. I was going to do it this morning, but we were too overcast.
Did the test this morning... what a nightmare.

All of the microinverters started flip-flopping on/off.

To the right, you can see the two states, every
5 min on then off.

So definitely not all is well. The IQ8s are
supposed to throttle back the inverters,
not start flipping them all off and on. I
suspect the January "fix" to the grid profile
being reset at frequency changes busted this.

No new news/conclusions on my 6 ill-behaved microinverters
1617811847385.png1617812169181.png


Called Enphase, after a couple of hours they said it was because the battery was approaching full charge. I explained it had been doing it since the start of the test when the battery was at 60%, so they suggested switching from savings mode to full backup.

That seems to have made the battery want to go higher, but it's still on/off rather than throttle. Hopefully, this is a one-off. I don't recall seeing this behavior before when going off-grid. Sure hope they get it fixed before hurricane season.
 

GXMnow

Photon Sorcerer
That's not good. The whole point of Ensemble was the integrated control making it all work smoothly together. I forgot... How much Encharge battery do you have? 3,600 watts coming in from the solar, with 2,800 going to charging is a fair bit of power. I think the Encharge-10 has something like 12 iQ8's in it. If that is correct, then your 2,800 watts going to charging is still less than 240 watts each. I would hope they can take that. They list the output power at 3,800 watts, but I can't find a charge power or current limit. But even if it has a lower charging current limit, it should be able to signal the iQ7's n the roof to throttle back their output. That should be basically the same as the HECO zero export limiting. The Envoy should be commanding the production to meet demand plus the desired charge current. Obviously this is not working correctly. With the proper control from the Envoy, it should not even need to frequency shift. It should just hold 60 hz and use data control.

Maybe I can try my off grid test this weekend. I know our systems are very different, but it will be interesting to see how the same iQ7 microinverters handle running on the Schneider inverter with frequency shifting vs the iQ8 Encharge and data control.
 

svetz

Works in theory! Practice? That's something else
Staff member
Moderator
Now that the battery is at 100% got a couple of these message sequences:

JSON:
"EnchargeSoCMaintenance : Set",   "Wed Apr 07, 2021 02:27 PM EDT"
"EnchargeCellBalError : Set",    "Wed Apr 07, 2021 02:26 PM EDT"
"EnchargeConstantVoltageMode",    "Wed Apr 07, 2021 02:26 PM EDT"
"EnchargeSoCCalibrated",      "Wed Apr 07, 2021 02:26 PM EDT"

I don't see any clears for the Sets... so assume it's still balancing
Update: The clears came in about 6:30

How much Encharge battery do you have? 3,600 watts coming in from the solar, with 2,800 going to charging is a fair bit of power.
I've 13 kWh. The rated output is 5.12 kW, the datasheet doesn't mention a charging rate (possibly 3.6, as shown above, is the max). Don't forget that the system should be able to output both from the battery and the array (i.e., 5+ 7 kW = 12 kW) at the same time... not that I consume anywhere near that. I'm back on-grid now and the array is outputting 6.6 kW (about 90 min after solar noon).

...The Envoy should be commanding the production to meet demand plus the desired charge current. Obviously this is not working correctly. With the proper control from the Envoy, it should not even need to frequency shift. It should just hold 60 hz and use data control.
That's what I think too. Got an email from them, they've applied an alternate grid profile to see if it helps (that's what they did last time too).
I was IEE 1542 2545-whb, then they switched to IEEE 1547 default grid profile, and now it's IEEE 1547:2015-WHB. Feels like a hail-mary, but IDK.
The system does appear to be AOK now I'm back on grid. If it's sunshiney in the morning I'll repeat the test.

Maybe I can try my off grid test this weekend. I know our systems are very different, but it will be interesting to see how the same iQ7 microinverters handle running on the Schneider inverter with frequency shifting vs the iQ8 Encharge and data control.
Thanks! I didn't know the Schneider could frequency shift them, I thought it turned them on/off like Outback does.
 
Last edited:

GXMnow

Photon Sorcerer
Thanks! I didn't know the Schneider could frequency shift them, I thought it turned them on/off like Outback does.
Currently my iQ7s are still on the "IEEE1547 Mainland Alternate" grid profile, which I do not think will do Freq/Watt curtailment. I want to test on this grid profile first as it has been rock solid on grid. Then I will change back to the Rule 21 with (F/W) in the profile name, as I (ASSUME) that would mean frequency watt control. The XW-Pro on the software I am running (currently 1.03, they are up to 1.11) is capable of frequency shift control to both regulate the charge current and also limit power once the battery reaches full charge. According to the manual... It starts to ramp up frequency at 0.5 volts below the set boost charge voltage. I have that set ay 57.2 volts now, so it should start to shift when the battery reaches 56.7 volts. If the power does not fall fast enough, or enough overall, it will shift all the way to 62.5 Hz which should then shut the iQ7s down completely. May last 2 off grid events did not go well and Enphase has since updated my firmware twice. The last time during a real power failure, 15 of my 16 inverters were locked out in "Grid Frequency out of range" even though it was dead on 60.0 Hz on my Fluke. And they stayed locked out until they remote changed the grid profile to the "IEEE1547 Mainland Alternate" and then they all came back online, but the sun was going down so it never got close to fully charging the battery. The power cam back around midnight so I didn't see what would happen when the sun cam back full. During that outage, I was not able to get in with the installer toolkit to change the setting myself. I actually had to connect the Envoy to my WiFi hot spot to get it on the internet for Enphase support, my cable internet also went down because their battery backup failed. All the gear in my house was running off of the Schneider, and I was on internet for a solid 4 hours until the cable model lost signal. I think they need bigger batteries.

I am just a bit nervous to flip off my breaker for the test. All of my PC gear is being powered by the Schneider, so if it glitches on the transfer, it could take them all down, as well as all my network gear etc. I used to have small UPS units on my 2 PC's, but the batteries in them failed, and even with a new AGM battery, it wont come back to life now.
 

svetz

Works in theory! Practice? That's something else
Staff member
Moderator
Overnight I let the battery drain to 50%. Then this morning I flipped off the mains, came upstairs, and switched the battery to full backup mode. It was just minutes after sunrise and the panels were only producing a few watts.

The microinverters did a reset when the power was switched, and once again 5 minutes later.

After that, it was remarkably well behaved. Early morning yet, so only making 2 kW.

So, it's either the new grid profile or being switched to Full Backup mode. My money is on full backup mode. I can see it turning off the microinverters if it's at the maximum charge.

The thing is there's never a benefit, regardless of the mode, to throwing away solar power. I'm sure they'll fix that now that they know about it, and of course, we have a work-around.
1617885390291.png

Update: So the original test to see if I'd get resets on those 6 microinverters? A little inconclusive: 6 events, but two were from other microinverters.
Update 2: Switched back to "Savings Mode", still working okay. So perhaps it was the grid profile?
Update 3: Yesterday was the first time I'd seen that behavior, I wonder if there was some "instability" that it couldn't recover from (e.g., couldn't reach a steady-state of stabilized voltage/frequency)? If so, desensitizing it via a grid profile does make sense.
 
Last edited:

GXMnow

Photon Sorcerer
It sounds like Enphase is making some progress on their software. I did a bit of microcontroller coding and it can surely be tricky to test for every possible condition to ensure it keeps running and can recover from an error. Programming a watchdog on the old Motorola parts was always a pain to make work well without infringing on the code doing your main task. And that was a single thread only part. Modern MCU chips are multi threaded with out of order execution and partial sleep modes to lower power consumption. I don't have the capacity to even think about writing the low level code on these new beasts. But now they are 100 times more powerful, so most programmers use a high level language and trust pre built modules to do the heavy lifting and background chores. That can simplify the coding for a specific application, but if something doesn't work as expected, is it your code, or a third party module causing the issue? In the case of the iQ series micro inverters, they also want to keep the power consumption as small as possible. Any watt it needs to run is taking away from the efficiency. To put out 240 watts at 97% efficiency, means the whole processor, power line communication, and DC-AC sine wave inverter all has to operate at just 7.4 watts total. And it has to track the local grid and stay in sync and adjust for changing loads and solar input. Building a single large inverter is actually much easier in several ways. My Schneider XW-Pro is just 95% efficient at 3,000 watts. That means it has 158 watts to use for the processor and inverter losses. That is a big difference. The coding I did was never too concerned about power consumption, so I never even used the CPU sleep functions, I just let it run full clock the whole time. Then again, it was a 4 watt CPU running at just 2 Mhz. That is less than the sleep mode of some modern chips. But in the iQ7, that is over half of the total power budget. The actual inverter bridge would need to be 98.6% efficient if the CPU was sucking up 4 watts all the time.

So I really have to give those guys credit for their coding. We may not like having a bug once in a while in our systems, but having an idea what goes in to coding the firmware on these things does make me respect what they have done. But that does not stop me from being frustrated when it fails to make power on a sunny day.

Now Schneider on the other hand, they are just being either lazy, stupid, or they just don't care. The feature I am asking them to add is very simple and would only require a minor code change. But that is another thread altogether.
 

michael129836

New Member
I've enjoyed reading this thread so I figured I'd contribute to the latest discussion. I have an Ensemble system with an Encharge 10. I haven't tested off grid lately but it looks like the firmware hasn't changed since I lasted tested it. The behavior I saw was that if the battery is full and the sun is shining then initially the system uses only the batteries and no solar. Once the battery gets down to a bit of a buffer, like 96% it turns on the solar to match the load demand and stops using the batteries. From what I recall it uses power line communication to regulate the solar panels, the frequency stays at 60hz. Hope this helps!
 

GXMnow

Photon Sorcerer
I've enjoyed reading this thread so I figured I'd contribute to the latest discussion. I have an Ensemble system with an Encharge 10. I haven't tested off grid lately but it looks like the firmware hasn't changed since I lasted tested it. The behavior I saw was that if the battery is full and the sun is shining then initially the system uses only the batteries and no solar. Once the battery gets down to a bit of a buffer, like 96% it turns on the solar to match the load demand and stops using the batteries. From what I recall it uses power line communication to regulate the solar panels, the frequency stays at 60hz. Hope this helps!

That is how it should work from all of their white papers and online description. The idea behind it is great, and if they ever do get the cost of the Encharge batteries down, I may switch to it. Both svetz and I have seen some odd behavior from the iQ7 microinverters on the solar panels. The solar side of our systems are all Enphase with an iQ combiner. Where we differ is the storage. My backup is third party (Schneider) with no data control, just frequency shift. svetz's system (and I assume yours too) is the all Enphase gear with the Encharge and Enpower units. I almost went Encharge as well, but with the delays and the cost, I pulled the trigger and ordered my Schneider XW-Pro. I have nearly 18 KWH of Li NMC batteries on it, so it would be about the same capacity as an Encharge 10 and two Encharge 3's. I fully expect my system won't run as seamless as yours with the all Enphase solution. But some of the glitches I have seen are similar to the error svetz has also seen. Even when we don't even notice any power glitch, we have had some of the solar iQ inverters report grid stability issues.

I had several of my iQ7's go offline for 5 minutes at least twice in the last 2 months from over frequency, or grid instability, but my Schneider log shows no power issue at all since February 23 when it showed a grid frequency high disconnect. The funny thing about the 23rd, the XW switched to battery power and the Enphase iQ7's appear to have stayed working just fine. It was 6:45 PM so the sun was just about down, so it is hard to tell, and my system only shows the last 500 log entries, which is only about 5 normal days, less if there were odd errors.

Back on January 19th, I had a real power failure. The XW-Pro saw it and started grid forming before the sun came up. We also had crap weather with a lot of clouds, so I didn't get much solar. But as the day wore on, and sun did start poking through, I saw that only one iQ7 was making any power. The log showed the other 15 were all stuck in Grid Frequency out of range, even though my local grid was fine, and one of the iQ7's was completely happy and pushing about 50 wats from a grey sky. Enphase had to remote in and change the grid profile to get them back online. They have since upgraded my firmware twice, but I have not tried a power fail off grid simulation yet. The grid instability logs I saw were after the first software update, I have not seen any since the second update.

Logging into my Envoy with installer toolkit now, it shows all 16 making power, pushing 1.95 KW at 9:15 am, not bad at all. My firmware was last updated March 9, 2021 at 8:37 am running image "520-00082-r01-v04.27.04" Looking through the logs, two days ago, Apr 7, I see a few entries for "High Skip Rate" "DC Voltage Too Low" but this was at 6:37 am, I'm Guessing clouds came in and they saw just enough light to power up and then cut off again. On Apr 6 it did a few more "High Skip Rate" entries at 7:21 PM. This is as the sun goes down. All of the rest of the 500 entries are the normal power off power on each day for the 16 inverters. Without extra stuff going on, it goes back to Apr 4th as expected. So it ran this last week with no odd errors.

It looks like they are getting better. I certainly want to do a power fail test before my next real failure.
 

svetz

Works in theory! Practice? That's something else
Staff member
Moderator
I've enjoyed reading this thread so I figured I'd contribute to the latest discussion. I have an Ensemble system with an Encharge 10.
Welcome Michael! Great to have another Enphase/Ensemble user to share data/experiences with!

... The behavior I saw was that if the battery is full and the sun is shining then initially the system uses only the batteries and no solar....
When I flip the grid off at the breaker, the inverters all seem to do the grid instability/reset thing for 5 minutes (don't recall that happening originally but I've been through a couple of s/w revs now).

No problem with the battery backup, lights, clocks, and PC don't see any impact. I was at 100% charge in Full Backup mode when I started that test today. The microinverters only need 5 minutes to reset, but I didn't see the panels start to output power until a while later when the battery went to around the top of 97% (it was still showing 98% when the panels kicked in at very low output, figure it was rounding up to 98%). When the battery hit the bottom of 97%, the amount of solar started creeping up and the battery dropped to idle and then started charging. No bad behavior today either, looking solid. (y)

So that confirms (or darn close to) what you saw. Out of curiosity what software loads and grid profile are you running?
My Encharges are at 20.34 (my one 20.39 got downgraded) and the IQ7s are the same as GXMnow's: v04.27.04.
1617992885012.png

...the frequency stays at 60hz. Hope this helps!
It does help! Can't believe I forgot to measure the frequency the other day as I noticed while on battery backup one of the UPS's chirping on the day of the "event". As there was power (e.g., PCs running) that probably meant a voltage or frequency issue.

I just measured the frequency while on batteries before the microinverters came back on, it was 61 Hz*. The 1741 throttling doesn't occur until 61.5 (although I believe the throttling occurs via the Envoy's PoE communications and the 1741 shifting is for utilities to throttle grid-tied systems). When the panels started outputting, the voltage 121V at 60 Hz, and they stayed that way even into the "charging phase". I suspect when they hit 98% or so the panels will probably kick off again.

Now that the system is on-grid again it 123.6V @ 60 Hz.

*Update: That 61 Hz might be bogus, noticed the meter held at 61 for a bit before settling at 60 Hz on the last test
1617991728394.png

I had several of my iQ7's go offline for 5 minutes at least twice in the last 2 months from over frequency, or grid instability, but my Schneider log shows no power issue ...
Like @GXMnow, I have some IQ7s (six) that have 0 to 4 resets during the day, the outage time is only 5 minutes per event so not a big deal (used to be back when they'd be out all day and reset overnight (if you were lucky, as you can see from GXMnow's post he's had far more serious problems than me)). Mainly I wonder if that's a sign they're defective in some way since none of the other microinverters do that. I might replace the two worst with IQ8s when they come out if they're cheap enough, might be good to have a couple of spare microinverters.

Overall I have to agree with GXMnow in that things are getting better, and Enphase is easy to work with once they agree there's an actual problem.
 
Last edited:

svetz

Works in theory! Practice? That's something else
Staff member
Moderator
Did this in the AM to the panels...

Always amazing how a light sprinkle knocks all the dirt
out of the air and onto the panels.
ConsciousUnhappyHumpbackwhale-max-1mb.gif
 
Top