Both Victron Cerbo-GX's died lastnight...

[justinm001]

I have 2 completely isolated systems one 12V and other 48V, have the 48V system connected to the 12V inverter to charge but the entire communications systems are separated. Ones completely in a rear bay and other is in the front bumper.

Lastnight at 10:04pm got an error saying no data from the Cerbo to VRM. (was just the 48V so not sure if both). This morning I checked and my TVs showed no internet so I rebooted my 12V systems and they came back on and both reporting data. My 48V cerbo touch was all white and when I went in form the site to remote console it was white. The 12V was black so I unplugged the Cerbo but started getting the flashing amber light for wifi only. Which appears the device cannot boot.

48V was working for a few hours but I was getting an error from the solar saying no BMS so it was off. Both battery banks are 40-75% charged and the coach is at my office with only 15a plug so was relying on solar. Finally just got around to unplugging the 48V to reboot and now its also flashing amber light only.

Both systems have Venus OS large and guimods plus nodered enabled but nothing setup yet. Other than that all normal with nothing on any of the inputs other than victron gear. BTW the 12V is a Cerbo-S GX and the 48V is a normal Cerbo GX.

I'm about to follow the instructions below to factory reset them. But I'm at a complete loss to what could have happened. They're isolated systems and only thing connecting the 2 is my 120V breaker panel. The 48V is even on wifi where the 12V is on ethernet.

Last night I did change the 12V from on to inverter only as it uses 12a min to charge and i was plugged into 15a on the 48v so wanted to drain that battery and charge the 48v overnight so it can charge the 12V in the morning. I had 75% battery so should have lasted over 24 hours with idle load.

Any thoughts on what I should try before wiping everything and starting over?

18. Reset to factory defaults procedure

www.victronenergy.com

 

[justinm001]

Alright got everything back online... well just finishing up. Seems they both somehow ran out of space which caused all kinds of errors. Idk why they only have 1GB storage on these things then don't even use the full partition. I'm betting i'm running RC on everything so likely something updated and ate the space.

To diag I picked up a USB to TTL cable (thanks Microcenter) and opened up the CerboGX connected to 115200 baud and hopped right in as root. saw 100% disk space and looked at the boot logs showing it couldn't boot. I tried the reset to factory defaults on both a USB and SD card but nothing happened, although I could see it in linux. Ended up following another procedure to load the image file and reinstalled it. Connected to internet and back online. Now just need to make all the changes and reinstall stuff.

If not an update strange they ran out of space at the same time.

 

[Adam De Lay]

Yeah, seems very odd that both go out within hours of each other.

 

[HRTKD]

My Cerbo GX has been online for maybe two years without being shutdown except for updates. I haven't installed any add-ons like you did, nor have I added any storage. It connects to WiFi when it's available, which is only when I'm camping.

Maybe there is logging turned on that consumed all the space?

 

[sunshine_eggo]

MicroSD slot.

 

[HRTKD]

MicroSD slot.

Does the MicroSD storage become part of the whole storage system, or does it get a different "drive" designation which requires you to tell certain to write to that location instead of the default?

 

[justinm001]

Does the MicroSD storage become part of the whole storage system, or does it get a different "drive" designation which requires you to tell certain to write to that location instead of the default?

looks like different drive and you have to configure it all in linux to utilize that drive. They might have some command to move storage or logs or something to that drive, but its not as simple as plug in a 64GB and you increased it 64x.

 

[Adam De Lay]

looks like different drive and you have to configure it all in linux to utilize that drive. They might have some command to move storage or logs or something to that drive, but its not as simple as plug in a 64GB and you increased it 64x.

I just removed and re-inserted my SD card and on the screen it displayed "using external storage for logging". Not sure if that's part of the GUI Mods changes or not. Wouldn't think so.

 

[sunshine_eggo]

On a 'standard" install (no large kernel, Node Red, etc.), the primary need for space is offline VRM logging. Simply inserting the MicroSD changes the default offline logging location as @Adam De Lay 's results indicate.

 

[justinm001]

Found you can check here. I also realized I changed the logging to 1min from 15min so likely 15x the log files. Added SD card to both but only 1 had that pop-up like @Adam De Lay mentioned. The other isn't showing here so likely has to be formatted to fat32 or something.

Nice part is when reloading the CerboGX they auto configure to vrm and everything is back to normal. Only lost my ruuvi sensors which I just needed to enable the bt option and now need to rename... once I figure out which is which.

 

[Adam De Lay]

I believe I formatted mine to fat32. It's been a while since I've done it.

Glad to hear you're back up and running though!

 

[sunshine_eggo]

My CCGX memory is anemic. They can't run large kernel Venus OS and had memory on the order of < 20MB.

I can tell you, 1m logging gobbles up storage. Before I added a 32GB MicroSD to my CCGX, I could only store about 3-4 days of data.

It is my understanding that space should be freed up when the data is uploaded to VRM.

 

[HRTKD]

It is my understanding that space should be freed up when the data is uploaded to VRM.

That's what I would expect too. But the system should be trimming the logs to account for space or lack thereof.

 

[sunshine_eggo]

That's what I would expect too. But the system should be trimming the logs to account for space or lack thereof.

Good point. Agreed. My system definitely trimmed the data on a FIFO basis. there have been a few times where connectivity was down 7-10 days, and I only got the last 3-4 days.