If they’re being loud or obnoxious then yes that is rude. But the vast majority of people are not disturbed by normal background chatter. In fact the opposite is true, many people are so comforted by the human voice that they listen to podcasts to fall asleep.

Whereas listening to a video out loud is always obnoxious and disruptive. Especially when there are numerous inexpensive ways for you to avoid doing it (headphones, eat somewhere else, mute and rely on captions, do something on your phone besides watch a video)