How should one treat fill-in-the-middle code completions? (self.fosai)

submitted 3 weeks ago by Smorty to c/fosai@lemmy.world

4 comments fedilink hide all child comments

Many code models, like the recent OpenCoder have the functionality to perform fim fill-in-the-middle tasks, similar to Microsofts Githubs Copilot.

You give the model a prefix and a suffix, and it will then try and generate what comes inbetween the two, hoping that what it comes up with is useful to the programmer.

I don't understand how we are supposed to treat these generations.

Qwen Coder (1.5B and 7B) for example likes to first generate the completion, and then it rewrites what is in the suffix. Sometimes it adds three entire new functions out of nothing, which doesn't even have anything to do with the script itself.

With both Qwen Coder and OpenCoder I have found, that if you put only whitespace as the suffix (the part which comes after your cursor essentially), the model generates a normal chat response with markdown and everything.

This is some weird behaviour. I might have to put some fake code as the suffix to get some actually useful code completions.

you are viewing a single comment's thread
view the rest of the comments

[-] Smorty 1 points 3 weeks ago

I use ollama for generations, so all the formatting is presumably done for me. Sometimes it does generate correct code though, so it seems to be working.

But thank you for the link!