So it seems like it can only be a gain to turn it off for certain spells. And that is intuitively obvious, too: turning something off should save energy.

Cars get better mileage on the highway than on city roads. Drive an hour, stop, drive an hour, stop, drive an hour. It is less efficient than driving 3 hours nonstop. The more stops the more inefficient.

The under sink heaters are designed to keep water hot 24/7. I would image it is better insulated than the Zojirushi. The water is stored in a container that basically heats it up from every side. Some kettles like the one in the video have heating elements inside the water which is more efficient than just a hot bottom plate. Under sink heater has a hot plate on all sides of it's box.

From my experience, I've had one my entire life and couldn't go back. I would expect at least 7-10 years from one. One in my current house is 7+ years old and works exactly the same as the day I installed it. Never turned it off once.