Easy to setup locally hosted LLM with access to file system (programming.dev)

submitted 5 months ago by youreusingitwrong@programming.dev to c/fosai@lemmy.world

7 comments fedilink hide all child comments

Hello, I am currently using codename goose as an AI client to proofread and help me with coding. I have it setup towards Googles Gemini, however I find myself quickly running out of tokens with large files. I was wondering if there are any easy way to self host an AI with similar capabilites but still have access to read and write files. I've tried both ollama and Jan, but neither have access to my files. Any recommendations?

you are viewing a single comment's thread
view the rest of the comments

[-] nocteb@feddit.org 3 points 5 months ago* (last edited 5 months ago)

Look into setting up the "continue" plugin in vs code. It supports an ollama backend and can even do embeddings if setup correctly. That means it will try to select files itself based on your question which helps with prompt size. Here is a link to get started, you might need to choose smaller models with your card.

https://ollama.com/blog/continue-code-assistant