There's no limit. But, too much becomes wasteful.
As long as you keep the VOC below the max of the charger, you won't hurt anything. The panels don't push power into the charger. The charger draws power from the panels. More panels gives the charger more available sources to draw from.
Think of the solar panels as buckets. And solar energy as rain. In a heavy rain, you can collect a lot of water, with only a few buckets. But in a light rain, it will take a lot more buckets to collect the same amount of water, in the same time frame. You just have to find a balance. So that you can collect enough from a lower production day. Without having way more than you need, on a higher production day.
No point in paying for too many panels, if you are rarely able to utilize them.