I did that, as a kid in a class. Battery, light bulb, switch. When switch was closed, light went out.
When OP turned panels off he turned switch on (not off) he was shorting out the capacitors. Sometimes, that is destructive.
<corrected>
And that is possible. The higher the charge current, the more likely. An SCC has current running through an inductor, which would cause a voltage spike when BMS disconnects. I think some sort of transient voltage suppressor is needed in BMS and/or SCC for protection. Maybe they have one already?