• 0 Posts
  • 14 Comments
Joined 2 years ago
cake
Cake day: July 23rd, 2023

help-circle

  • Possibly. Been running it since last summer, but like i say the small models dont do much good for me. I have tried llama3.1 olmo2, deepseek r1 in a few variants, qwen2. Qwen2.5 coder, mistral, codellama, starcoder2, nemotron-mini, llama3.2, qwen2.5-coder, gamma2 and llava.

    I use perplexity and mistral as paid, with much better quality. Openwebui is great though, but my hardware is lacking

    Edit: saw that my mate is still using it a bit so i’ll update openwebu frpm 0.4 to 0.5.20 for him. Hes a bit anxious about sending data to the cloud so he dont mind the quality






  • There are other solutions than docker for that use-case that I think are better fits. It probably works fine, but for me other drivers including host mode and ipvlan seems to have been introduced to solve the wrong thing. Like how it needs privilege for them to work and how it exposes the containers network interface. For me it kinda breaks parts of why i would use docker.

    Its my personal opinion and how i like to work.

    You could probably make your setup work but it seems too complicated for me when you introduce a bridge as the root interface. Maybe with macvlan adapters on the host instead or in addition.