Businesses are in business to make money. If the government is giving them money, then the products they sell will (should!) be cheaper, or at least the company will operate more closely to the necessary profit to stay viable.
I don't see how having a subsidy could possibly cause prices to rise. Eliminating the subsidy will cause prices to rise, for certain.
If we didn't subsidize agriculture, for example, we'd be buying apples for $5 each or a pound of burger for $25. I'm not saying certain industries need subsidies - ag certainly does or we would lose nearly all domestic farming - and I think a lot of companies get subsidies that shouldn't. AND I hate my tax dollars being used to line the pockets of executives. So, it's really a matter of the exact circumstances of the subsidized industry.
I'm almost certain the foreign solar panel manufacturers don't need any US subsidies, despite probably receiving a lot (I don't know). But I do think the domestic panel manufacturers do, so that we can ramp up domestic production and get on top of the game. Same for Lithium cell manufacturing. Honestly, once we get some level of good cell and panel manufacturing in the US, I say tariff the hell out of the foreign manufacturers and subsidize the domestic ones only for as long as necessary to get our domestic industry rolling.