No, if done correctly the site verifying the token should only need to verify it was signed by the authority token provider.