Getting around Google's attestation with an unlocked bootloader requires root - I believe the go-to is Magisk and the Play Integrity Fix module. It's also a good idea to put the apps in question on the Magisk denylist. I've been using this for years with good results and would not describe it as "a lot of things".

Is that from installing an app or from install a malicious ROM?

A malicious app could modify the OS, but it would need root permissions. There are three ways that can happen:

• The app exploits a privilege escalation bug in the OS. This can happen even if you don't have root access yourself.
• The app exploits a bug in a superuser permission manager (e.g. Magisk) to gain root privileges without prompting you.
• A previously legitimate app you've given root privileges to gets a malicious update (a supply chain attack).

A malicious ROM is certainly possible. Some random person's LineageOS fork is slightly less trustworthy than its maintainer (due to supply chain attacks).