If your ISP uses an Ethernet cable into your premises, then you most likely can do exactly that. Either you’ll get an IP from the ISP’s DHCP server or you’ll have to configure a PPPoE client with some login details before the ISP issues you an IP and connectivity.

To do that to every device on the globe you’ll need a much larger address space to IPv6 is your only choice.

That said, be mindful of the security that a router provides. Connecting your PC this way means you desperately need to keep it up to date and running restricted firewall.

Some ISPs run a coaxial cable into your house, though. Most of these use (Euro)DOCSIS and finding an interface card for a regular PC seems impossible.

All in all, it’s possible but it depends on the cabling standards and the protocols spoken. Don’t forget, at the end of the day, your modem is just a computer running Linux, most likely.

Check out OPNsense or OpenWRT if you’re interested in using a regular computer for routing. Alternatively just use Linux but it’s a lot more work.