Because the Cubans that fled were the capitalists and landowners whose land Castro appropriated and nationalized. They didn't like that the Batista gravy-train was coming to an end, and their way of life would no longer be acceptable in Cuba, so they fled "oppression."

So when the GOP misuses "socialist" and "communist" to scare people, it really works on those people because they think the government is literally going to take their property away.