An important point to remember in your power audit is that the inverter itself is a load, that needs to be included in the calculations. Inverter draws power just being turned on. How much depends on the brand/model. A general rule of thumb is that background idle is related to the quality of design, and the inverter size. Bigger inverters consume more power. Poorly designed inverters of a certain size consume more power then quality inverters of the same size. Most inverters publish their background idles. If they don't, that's a good reason to avoid it.
To help this thread along, I'll mention my own consumption numbers. In the winter, on days when I'm not there but the frig is on 24/7, I consume as little as 1.5kWh. In summer, when more cooling is required that's 2.5kWh. For you, to keep the frig running, some lights on, and a bit of TV, let's say you need 3.0kWh, or 3000Wh of electricity.
In Kentucky, I'd guestimate you get maybe 3sunhours in winter and maybe 5-6 sunhours in summer. Let's say your power goes off in the winter. If you need 3000Wh per day, and you get 3.0sh per day, then you need 3000Wh/3.0sh = 1000W of panels. You are better off at 24V for this much power. Charging at 25V minimum, you will max out at 40A, so a less expensive controller like an Epever Tracer 4210 will work for you. You can get 60-cell residential panels dirt-cheap. Wire four 250W panels 2S2P, and that 4210 controller will work just fine.
You could wire your existing batteries in series right now for 24V, but only 100Ah at 24V is still rather light. You might want to add a second parallel string of batteries in the near future as finances allow. If the batteries are not yet used, and they can be returned, it might be a better choice to upgrade to four 6V golf-cart batteries, such as Trojan T-105s. That would give you 250Ah of power at 24V, and a better size for 1000W of panels.