Prisons operate as profit centers. The homeless are an exploitable cohort for predation - physical violence, sexual assault, sub-living wage labor, entertainment. Public housing depresses the overall real estate market and takes lots of land out of the private sector. The incentive of "low" public costs doesn't really drive political behavior, because spending and taxation are decoupled from one another for the average voter/consumer.
Then there's a certain value in reserving a small cadre of armed proles to cultivate their sadistic tendencies on a permanent underclass, in anticipation of a larger and more organized opposition of wage workers. Homeless residents become a proving ground for prospective professional head-crackers and back-breakers.