I've heard it the exact opposite. Freedom to is positive freedom which tends to be a more social leftist or social liberal trait. Negative freedom (freedom from) is typically a more modern right wing or libertarian trait. But also you could have libertarian leftists or anarchists that lean more towards negative liberty, as well as fiscal conservatives that lean more towards positive liberty on social issues, so it's not fully a left/right thing.

Basically the difference is enabling people via common social framework that gives people options and social mobility vs complete non-interference by government or any other entity even if it limits options and social mobility for anyone but yourself due to their life circumstances.

Here's a quote from the Wikipedia article on positive liberty that backs up this interpretation of the to/from distinction. (https://en.m.wikipedia.org/wiki/Positive_liberty):

"Erich Fromm sees the distinction between the two types of freedom emerging alongside humanity's evolution away from the instinctual activity that characterizes lower animal forms. This aspect of freedom, he argues, "is here used not in its positive sense of freedom to but in its negative sense of 'freedom from', namely freedom from instinctual determination of his actions."

I don't know that I agree with that premise but it's an example of the to/from dichotomy being used in relation to positive/negative freedom just so you know I'm not making anything up.