I've got a spray bottle filled with windshield wiper fluid I sometimes use to "pre-treat" an icy windshield before I get to scraping it, it's often able to loosen the ice's grip on the glass so the scraper can just lift it off. Simpler and more controllable than relying on the built-in windshield sprayers.

A one-handed garden pick is a nice tool to have handy if you find your car's wheels stuck in some hard-packed snow or ice. Don't spin your wheels fruitlessly, the friction is just making the ice slicker and harder. Use the garden pick to dig the wheels out instead, creating a rough surface to get some initial traction on. There are also traction plates or mats that you can stick in there to help get moving, though you need to be able to move the car far enough to get them caught under the wheels for them to work.

Make sure your car battery is in good condition. Cold weather will reduce its power output, so if your car's going to fail to start it'll be in the dead of winter when that happens. For peace of mind I bought one of those battery booster packs that you can use to jump-start a car with and I really like it, it's got a built-in air pump, USB charger, and light source as well and I've used it for all of those things now and then. Wasn't very expensive.

Stash a warm hat and a pair of warm mittens in the car somewhere. If you end up stranded on a roadside you won't have known ahead of time that you were going to be stranded so you might not have brought adequate clothing with you. A flashlight, too. In northern latitudes there's a lot of darkness during winter time.