I just got web access working in preconfigured Open WebUI of PUU OS :-) It's amazing how much the vividity of the user experience degrades with an offline LLM ;-) Bonsai-2-27B is running on RTX 4090 24 GiB as vLLM workload hosted by K3s.
XFEDI
ActivityPubRemote profile
Jarkko Sakkinen
My in-progress (but not yet fully finished) operating system project:
1. https://github.com/puu-os/bootc
2. https://github.com/puu-os/installer
Posts
One measure I've taken with AI assisted patches is that I more likely cancel my reviewed-by when preparing a PR.
I don't feel bad about it because I'm only learning to dig out the dirt from these patches and sometimes I just miss something. Thus, the second look for almost single everyone of them.
I don't feel bad about it because I'm only learning to dig out the dirt from these patches and sometimes I just miss something. Thus, the second look for almost single everyone of them.
With the current version of pi-landstrip, you don't need pi-background-tasks plugin as subagents have gained that power. I.e. assistant can stop its turn and leave subagent tasks executing in the background.
OKSA is the model manager and hardware calibration engine of PUU OS. Taking baby steps on the web/management side. Not very flashy but at this point what is shown here is more for "power on testing" and debugging than anything else.
Likely stuff to land soon here is mainly QSFP networking configuration for parallel inference.
Given that every build has an integer number that identifies that build *uniquely* as builds are *reproducible* I also likely find a lean way to use this information to notify the user if the nodes co-operating in parallel inference have a version mismatch (i.e., non-enforcing warning).
This is because for optimal results in parallel inference it would be feasible if all nodes run exact same version of the same operating system, and with PUU OS it is possible to achieve in bit-by-bit level.
In Puu OS bootc containers and reproducible builds are not used because they are cool but instead what they bring on the table is one of the key features for implementing stable inference across a mesh of nodes.
This is fun area to innovate because not much of it is that well documented when it comes to inference hosting. And likely lot of it is also a trade secret for AI companies. So for things like hardware calibration there was no tutorial or AI assistance. I simply have to figure out how it could be done, fail, learn and retry. And still doing it :-)
It's amazing how far Buildroot can be streched.
#buildroot
Likely stuff to land soon here is mainly QSFP networking configuration for parallel inference.
Given that every build has an integer number that identifies that build *uniquely* as builds are *reproducible* I also likely find a lean way to use this information to notify the user if the nodes co-operating in parallel inference have a version mismatch (i.e., non-enforcing warning).
This is because for optimal results in parallel inference it would be feasible if all nodes run exact same version of the same operating system, and with PUU OS it is possible to achieve in bit-by-bit level.
In Puu OS bootc containers and reproducible builds are not used because they are cool but instead what they bring on the table is one of the key features for implementing stable inference across a mesh of nodes.
This is fun area to innovate because not much of it is that well documented when it comes to inference hosting. And likely lot of it is also a trade secret for AI companies. So for things like hardware calibration there was no tutorial or AI assistance. I simply have to figure out how it could be done, fail, learn and retry. And still doing it :-)
It's amazing how far Buildroot can be streched.
#buildroot
I'll enrich subagents a bit in pi-landstrip, so that plugins like pi-background-tasks are not required.
Like subagents, background execution needs to be internal feature in order to provide a sane user experience.
pi-background-tasks can be made to work with pi-landstrip thanks to TypeScript APIs Landstrip provides so that its tasks are sandboxed. It is just that two disjoint task managers is not very nice to use. A more sound angle is to extend pi-landstrip's subagents and TUI to have equivalent background processing features.
Like subagents, background execution needs to be internal feature in order to provide a sane user experience.
pi-background-tasks can be made to work with pi-landstrip thanks to TypeScript APIs Landstrip provides so that its tasks are sandboxed. It is just that two disjoint task managers is not very nice to use. A more sound angle is to extend pi-landstrip's subagents and TUI to have equivalent background processing features.
Pull requests in Github are useless because it is all shit. I can be almost sure that if I get a PR, I won't use that code (especially applies to Landstrip).
Windows LPAC support in Landstrip is getting mature and has been one of the best early investments I've made in this project.
Window support was for a long while a bad and useless piece of PoC code but as it has been carried over the releases things have gradually gotten better.
Starting with a subset of familiar OSs leads usually to an architecture that simply does not scale to OSs left out unless they are very similar to the OSs adapted in the first phase. It's a common mistake, which should be avoided because it is also a recipe for very likely failure :-)
Window support was for a long while a bad and useless piece of PoC code but as it has been carried over the releases things have gradually gotten better.
Starting with a subset of familiar OSs leads usually to an architecture that simply does not scale to OSs left out unless they are very similar to the OSs adapted in the first phase. It's a common mistake, which should be avoided because it is also a recipe for very likely failure :-)