If you have an outdoor Ethernet port—in my case with a WiFi AP connected—how can you go about protecting your network from somebody jacking in?

Is there a way to bind that port to only an approved device? I figured a firewall rule to only allow traffic to and from the WiFi AP IP address, but would that also prevent traffic from reaching any wireless clients connected to the AP?

Edit: For more context, my router is a Ubiquiti UDM and the AP is also Unifi AP