Seeing that the loads at the fuse panel will mitigate some of the current going to the batteries I see no issue with your proposal. As mentioned it is probably #6 feeding the panel, so some drop may occur-depending on the size of the array. Mock it up and measure, that's the only way to be sure.