I tried to give it a fair shake at this, but it didn't quite cut it for my purposes. I might be pushing it out of its wheelhouse though. My problem is that, while it can rhyme more or less adequately, it seems to have trouble with meter, and when I do this kind of thing, it revolves around rhyme/meter perfectionism. Of course, if I were trying to actually get something done with it instead of just seeing if it'll come up with something accidentally cool, it would be reasonable to take what it manages to do and refine it. I do understand to some extent how LLMs work, in terms of what tokens are and why this means it can't play Wordle, etc., and I can imagine this also has something to do with why it's bad at tightly lining up syllable counts and stress patterns.

That said, I've had LLMs come up with some pretty dank shit when given the chance: https://vgy.me/album/EJ3yPvM0

Most of it is either the LLMs shitting themselves or GPT doing that masturbatory optimism thing. Da Vinci's "Suspicious mind..." in the second image is a little bit heavyish though. And those last two ("Gangsterland" and "My name is B-Rabbit, I'm down with M.C.s, and I'm on the microphone spittin' hot shit") are god damn funny.