First, there have always been people who have thought, "I'm fine with the chance of dying to do this thing." Free climbers, for instance. If the odds of survival are zero, and your personal effort isn't going to change it, that number goes down by a lot.

Second, unless we find a FTL solution, surviving in space indefinitely is the first step in interstellar travel, because 3000 years is functionally equivalent to indefinitely. If you're response to that is sleeper ships, you only survive if the ship survives, and we're back to the same point. The reason this is important is because if the planet at the destination isn't required for your survival, you have a lot more flexibility for how you colonize that planet, which vastly improves the odds of success.

As for the Fermi paradox, it doesn't require that everyone wants to colonize a different star, build a Dyson shell, or whatever, it requires that everyone who doesn't want to do that be willing to do whatever it takes to stop anyone else from doing it (and can make it count). It's a slightly different proposition, and one that I think is less likely than other solutions.