Well, I was thinking of a quote that was much more similar to what I wrote (and it's not in the video you linked).

I had such a trouble finding it that I'm starting to feel like it might be one of those "quotes" where the credited author never really said that, but I haven't completely given up :D

Here's one closer to what I paraphrased (but not quite it)--quoting an article from cio.com

While Linux pretty much dominates almost every walk of our lives, even on the consumer devices like smartphones and smart TVs, it has not had the same success on the desktop. What does Torvalds think about it? Is Linux a failure on the desktop? Not really. “The desktop hasn’t really taken over the world like Linux has in many other areas, but just looking at my own use, my desktop looks so much better than I ever could have imagined. Despite the fact that I’m known for sometimes not being very polite to some of the desktop UI people, because I want to get my work done. Pretty is not my primary thing. I actually am very happy with the Linux desktop, and I started the project for my own needs, and my needs are very much fulfilled. That’s why, to me, it’s not a failure. I would obviously love for Linux to take over that world too, but it turns out it’s a really hard area to enter. I’m still working on it. It’s been 25 years. I can do this for another 25. I’ll wear them down.”