What kind of shifters are you running?

With friction shifters, the spring in the derailleur might be pulling the shift lever back. You can fix that by tightening the friction adjustment screw on the side of the lever. Backing out the limit screw might also help.

With indexed shifters, you can try increasing the cable tension a bit (not too much, or it will throw off the other gears).

Park Tool has some nice videos that take you through the process of adjusting both the front and the rear derailleur. All you need is a screw driver and a few hex keys (possibly).

https://www.youtube.com/watch?v=ZNG7g83lI-s

https://www.youtube.com/watch?v=UkZxPIZ1ngY

Calvin is great at clearly explaining these things, even if they promote their own tools quite a lot. (I am not affiliated with PT.)

If you can make it go away with the shifter, I would guess it is not the limit screw but that cable tension (I don't think the shifter would help alleviate the problem if the limit screw is set too far in). You could start with using 1/4 turn increments of the barrel adjuster at the shifter. Go slow. Do one 1/4, shift around and stop back on large cog and see how it sounds and looks.