There are going to be more services you need than http and https. You'll also need to allow DNS queries and a way to keep your time updated, for instance.

The advantage of blocking incoming connections is that each potentially vulnerable network service isn't automatically exposed to the internet. Blocking outgoing connections isn't going to improve your security much, considering that any compromised service trying to phone home could just use http, the same protocol your browser uses.

It would still matter that your OS isn't up to date. For instance, if there is ever a vulnerability in the way Windows 10 renders an image format that allows remote code execution, Microsoft will not give you a patch.

If you must remain on Windows 10, the best way to reduce attack surface is going to be to uninstall as much software as possible that you don't use.

If you only use a browser, then you could have a quite seemless transition to Linux. That way you can have an up-to-date system. The vast majority of browsers support it natively, including Chrome and Edge.