• 44 Posts
  • 38 Comments
Joined 1 year ago
cake
Cake day: June 12th, 2023

help-circle










  • I use text-generation-webui mostly. If you’re only using GGUF files (llama.cpp), koboldcpp is a really good option

    A lot of it is the automatic prompt formatting, there’s probably like 5-10 specific formats that are used, and using the right one for your model is very important to achieve optimal output. TheBloke usually lists the prompt format in his model card which is handy

    Rope and yarn refer to extending the default context of a model through hacky (but functional) methods and probably deserve their own write up










  • Yeah definitely need to still understand the open source limits, they’re getting pretty dam good at generating code but their comprehension isn’t quite there, I think the ideal is eventually having 2 models, one that determines the problem and what the solution would be, and another that generates the code, so that things like “fix this bug” or more vague questions like “how do I start writing this app” would be more successful




  • Yes agreed on the llama-2 models, they show a LOT of promise in the right tasks but they need some work to get back to what we remember from peak llama-1, i’m very excited for when that arrives in a week or two!

    Yeah by all means! At this time I’d say text-generation-webui is my most mature and functional image, with koboldcpp being a close second but I just don’t work as closely with it

    lollms-webui is a very interesting upcoming platform but it’s a solo dev so it’s a lot of work, my docker image works as long as you don’t need any personalities, but i’m working on that to see if I can get it sorted out :) for now though it’s definitely worth considering it beta or maybe even alpha

    Would love to keep our communities tightly knit, FOS AI and localllama both have similar ideals coming from two different angles, so keep in touch :D