I'm running Qwen3.6-35B-A3B on a very ordinary desktop PC (32GB DDR5, 8GB Radeon 6600XT) and getting a useful 15-20 tok/sec out of it. The MoE architecture and auto offloading from system to VRAM is just fantastic. Unsloth Q4_K_XL.
The Qwen3.6-27B is unbearably slow as it doesn't fit in VRAM, though, i think the MoE is very easy to run.
It is also extremely nice that you can just `apt install llama.cpp libggml0-backend-vulkan` now too.
People say this a lot about Linux and it somewhat rubs me the wrong way - sure, the Windows binary works if you install its library dependencies (wine). Likewise (OK, ever since libc5/glibc2 changeover in 2001) the Linux binary should work if you have its library dependencies (SDLv1 it looks like?). So what's really the problem? Your distribution stopped distributing the dependencies, making them harder to find? "DLL Hell" was a thing too.
I didn't see any binary downloads for Linux on that website, only source code.
I gave it a try anyway, the dependencies were actually not a problem for me, Debian has libsdl-{net,image}1.2-dev, libglew-dev, and so on, and if your distro does not have SDLv1 there is libsdl1.2-compat. But after the dependencies, there was a problem with the source code doing something involving bitfield packing that does not compile on x86_64.
I do see the source code has lived on, i can `apt install d2x-rebirth` on Debian 13 which has a comment about https://www.dxx-rebirth.com/ ...is that helpful?
> the Windows binary works if you install its library dependencies (wine).
Yes but all it takes is an `apt-get install wine` or `zypper install wine` or `pacman -S wine` or your distro equivalent and if you are using a Linux distro with any sort of desktop functionality, 99% of the time wine will be there.
> Your distribution stopped distributing the dependencies, making them harder to find?
Yes, this is a big problem because these dependencies not only are harder to find but even if you track all of them (there is a chance they too have their own dependencies) you still need to figure out how to build and install them. And if they are old enough for distros to drop them, then chances are building them isn't going to be a straightforward `./configure && make && make install` ("straightforward" is very relative here :-P). And woe is you if there are conflicts with newer (yet API/ABI incompatible) versions of these libraries and/or their dependencies.
Of course since you have the source, technically you can make things work, but that doesn't make the process any less of a major PITA.
Yeah, I know MacOS syscalls aren't stable. Interesting point about Go, I hadn't heard about that. Unfortunately I'm a masochist though, and want to avoid libSystem.dylib as much as possible. The only reason I link against it at all is because MacOS requires it for executables to run, I never actually call into it. Figured I'd just update the syscall numbers if/when they change.
I was pleasantly surprised to learn the architecture for this - a minimal backend that does a PAM auth and gives you a shell over websocket, with only your own Linux user credentials - and then everything else (from managing files to apache to VMs) is done in frontend javascript.
Keeps the server-side backend minimal and auditable.
There are multiple GH issues around better sync.Map. Among other alternatives, xsync.Map is also mentioned. But Golang core team doesn't seem interested in sync.Map (or a generic variant of it) improvements.
Gaseous form is a problem, but have you seen the Fraunhofer POWERPASTE? I was optimistic when the news was first announced, but that was a decade ago and of course it's not widely used.
Clearing of the secrets is a separate issue from memory allocation mechanism. It must be done all the way from the encryption layer to the program to avoid the leaks.
This is typically not done, only certain parts such as handling of the crypto keys. That's because it's pervasive and requires reworking everything with that in mind (TLS library, web framework, application).
On the other hand the centralization and global usage of GC in the process allows to modify it to always zero out the memory that it deallocated and to do GC at regular intervals so it can have advantage here (it's very easy to inadvertly leak the secrets to some string).
You can see the spirit of what they're going for also with the MIT binaries - that's also like saying the whole project is AGPL, but a loosening for using it as-is.
Given their goals seem to be
- Permissive use without modification, even in combined works ("MIT binaries"); but
- Copyleft with modification, including for the Affero "network hole", or commercial terms
could you suggest a clearer license option? AGPL triggers copyleft across combined works, LGPL doesn't cover the network hole, GPL has both problems. Their goals seem really reasonable, honestly, there should be a simple answer. It seems messy but I like it more than the SSPL/BSL/other neo-licenses.
I don't know anything more reasonable, but I would argue that this (isn't) reasonable precisely because it causes so much confusion due to the ambiguity and their refusal to clarify exactly what the terms really are.
The Qwen3.6-27B is unbearably slow as it doesn't fit in VRAM, though, i think the MoE is very easy to run.
It is also extremely nice that you can just `apt install llama.cpp libggml0-backend-vulkan` now too.
reply