"Set and forget" time based thermostat programming only works if your daily routine doesn't change daily or weekly or have outliers. The ability to change manually, or add other factors (is anyone home? let it get a bit colder, since it doesn't matter) is pretty great.

But I would still advocate for no internet connected thermostats from the OP. Your thermostat should be isolated to your home network (via zigbee/zwave or a quality VLAN) connecting to a server/hub you control. And your app should be communicating to your server/hub. Your thermostat shouldn't be able to report back to google whether or not you are home.