What's actually in "non-free" nowadays? Do we really need that? Maybe we should get rid of it as the rules say "Free, open source software only." We might still need non-free-firmware, though.
Our Hosteroid Low End Spirit Community Server ("hlcs") is using Hosteroid's default Debian distribution, as far as I know. Hosteroid originally, and very kindly, gave me hlcs already installed.
I haven't gone through the contents of non-free, but, awhile back, I found out the hard way that gcc man pages are in non-free. Apparently there is a conflict between the GNU Free Documentation License ("GDFL") and the Debian Free Software Guidelines ("DFSG"). Please see
Without adding non-free to sources.list, one cannot use apt to install the gcc man pages, and contrib is necessary in sources.list to install additional gcc documentation (the gcc-doc package).
I'd be delighted to consider other arrangements if there is anything you want to do differently. If you want, we could consider reinstalling with a different flavor of Debian, a different Linux distribution, or a non-Linux OS.
If anyone else wants to join us, surely we would consider their view about what we might install or reinstall.
Thanks again @cmeerw! Thanks also to @Hosteroid for our fun, free hlcs server!
Not sure whether this, which involves a third-party component, is something for which you could provide a link? If yes, please do. If no, it's okay.
Thanks either way!
That's the thing with bugs that have security implications that you don't want the details to be public until everyone affected had a chance to update. I'll want to wait until I get confirmation that they are fine with making the details public first (I'll likely ping them in the next few days to check on the status - the bug report in their issue tracker is currently (still) hidden). So it's fully intended that you can't find anything yet...
Just checked on the status and they are still planning to do a formal security advisory. I'll post here once it gets published.
It would be lovely to have you join us, and so I will keep trying to give you an account on our community server.
Thanks for telling us what you want to do on the server (make a forum with Flarum or Discourse) and for giving us a link to your Github profile (at https://github.com/kk14569).
Unfortunately, it hasn't been possible for me to understand who and where you are.
Quoting from the OP:
Please remember that, quoting from the Rules above, "@Not_Oles must know who and where you are." Some people send scans of government and university ID. Other people send other or additional things.
If you want to see an example for what identity information is needed, please check the example given in the OP:
One of our friendly Administrators here at LES, Mason, is a great example of somebody whose request for an account would be accepted right away. Thanks to @Mason for kind permission to link to his pages.
When we look at Mason's example, we see his community presence here at LES via his LES profile. We also see who he is and where he is and what he does for work.
Your profile here and your Github seem impossible to use to determine who and where you are or what you do for work. Of course, I can imagine several very good reasons why you might not want to disclose your identity.
Again quoting from the OP:
I appreciate that many people here on the Low End have important and fully legitimate concerns about privacy. I respect those concerns! Happily there are many options available here on our beautiful Low End for people concerned about privacy.
If you wish, please feel free to provide more information about yourself. As I said, I will keep trying to get you an account.
Is it a good idea to use zfs on Linux tho? Worth the trouble or not?
Pretty sure it's a horrible idea to use it on Arch Linux but what about Debian stable?
@ususk said:
Is it a good idea to use zfs on Linux tho? Worth the trouble or not?
I really don't know! But @Cloud suggested trying it. So, why not try iit?
We can leave ext4, xfs, btrfs, and zfs on the server until somebody needs the space for something. Or we can wipe the server and do something different if everyone wants to go differently.
Pretty sure it's a horrible idea to use it on Arch Linux but what about Debian stable?
What difference does the Linux distro make on file system choice?
I use btrfs on Arch Linux for the snapshot feature. Sometimes need to roll back after I pacman -Syu and break something. I know zfs is capable of this and everyone tells me zfs is better than btrfs so I've been wanting to give it a try if it's worth the trouble
It would be lovely to have you join us, and so I will keep trying to give you an account on our community server.
Thanks for telling us what you want to do on the server (make a forum with Flarum or Discourse) and for giving us a link to your Github profile (at https://github.com/kk14569).
Unfortunately, it hasn't been possible for me to understand who and where you are.
Quoting from the OP:
Please remember that, quoting from the Rules above, "@Not_Oles must know who and where you are." Some people send scans of government and university ID. Other people send other or additional things.
If you want to see an example for what identity information is needed, please check the example given in the OP:
One of our friendly Administrators here at LES, Mason, is a great example of somebody whose request for an account would be accepted right away. Thanks to @Mason for kind permission to link to his pages.
When we look at Mason's example, we see his community presence here at LES via his LES profile. We also see who he is and where he is and what he does for work.
Your profile here and your Github seem impossible to use to determine who and where you are or what you do for work. Of course, I can imagine several very good reasons why you might not want to disclose your identity.
Again quoting from the OP:
I appreciate that many people here on the Low End have important and fully legitimate concerns about privacy. I respect those concerns! Happily there are many options available here on our beautiful Low End for people concerned about privacy.
If you wish, please feel free to provide more information about yourself. As I said, I will keep trying to get you an account.
@ususk said: Is it a good idea to use zfs on Linux tho? Worth the trouble or not?
Personally, I try to stay away from anything that's not in the mainline kernel.
@Not_Oles said: We can leave ext4, xfs, btrfs, and zfs on the server until somebody needs the space for something. Or we can wipe the server and do something different if everyone wants to go differently.
So far we are only using 64 GB for each of ext4, xfs, and btfs, so there is still plenty of space available. And as it's all logical volumes, we can extend each as needed. As long as we are reasonably conservative with allocating space to each of these logical volumes, we should be fine.
I am just hoping that ZFS doesn't get too upset when it's not allowed to use up all the space available, and gets put onto a logical volume (like everyone else).
@cmeerw said:
I am just hoping that ZFS doesn't get too upset when it's not allowed to use up all the space available, and gets put onto a logical volume (like everyone else).
When ZFS gets upset, it is because it is not allowed to use up all the memory available
We seem to be close to articulating the reason why I stick to ext4. There are so many technical advantage to the advanced file systems. These make me jealous! But then there also are horror stories every once in awhile on HN. The file systems seem a lot like big cloud provider billing. Every once in awhile. . . .
Life seems peaceful on a little, old dedi with ext4.
I was primarily interested in the performance numbers between ZFS and the others as you already did some measurements. I am not expecting any surprises really, but just curious to how they compare.
Yes ZFS has its reputation of being a memory hog, but you can tune ZFS pretty well to run nicely even on a low memory environment. I have it running on 512MB BuyVM boxes just fine. Snapshots, checksums, encryption etc are all very useful.
then, maybe, I will be able to run Yabs on ZFS to get its fio tests.
Additionally, I hope to invest a few seconds glancing at the source code for some of these file systems.
So that's my fun plan for the next few hours, days, or weeks on hlcs , depending on how long it takes and on how much other fun stuff I am doing elsewhere. Probably each step will show up in a post in this thread.
That sounds like a solid plan and I'm sure you are going to have a lot of fun with it. As a little twist, you can also try zfs-fuse to understand how much performance is being lost with the FUSE layer compared to a kernel module.
Just submitted a change for Haiku to fix an issue with the daily builds affecting Emacs (that's just a drive-by fix and not yet the fix I was talking about earlier).
Thanks to @hosteroid for providing the dev machine.
I wanted to glance at the Linux kernel filesystem code before installing more filesystems. In order to be 100% sure I wouldn't mess up our Community Server (Thanks @Hosteroid!) I cloned the kernel code with git on a VPS elsewhere.
Since there might be a few people here at LES who haven't cloned the Linux kernel code, here are the steps I followed.
I copied the terminal output during the clone, when the receiving objects step was 91% done:
root@manassas:/usr# ls
bin games include lib libexec local sbin share src
root@manassas:/usr# tar cvzf local-revert.tgz local
local/
local/sbin/
local/man
local/etc/
local/src/
local/share/
local/share/man/
local/share/ca-certificates/
local/lib/
local/lib/python3.12/
local/lib/python3.12/dist-packages/
local/bin/
local/include/
local/games/
root@manassas:/usr# cd local/src
root@manassas:/usr/local/src# ls
root@manassas:/usr/local/src# git clone https://github.com/torvalds/linux.git
Cloning into 'linux'...
remote: Enumerating objects: 10579170, done.
Receiving objects: 91% (9627045/10579170), 4.93 GiB | 37.78 MiB/s
A moment later, when the clone was finished, I could see
I hear from Google Gemini that the contents of the fs directory includes:
The Virtual File System Layer (VFS) -- abstracts common operations across various file systems to enable presenting a unified interface to user applications,
Specific File System Implementations -- btrfs, ext4, xfs, etc. -- each file systems implementation contains a "file_operations" structure with pointers to the filesystem-specific functions which implement user-level system call operations, such as read, write, open, and ioctl, and
Additional, related "helper" implementations -- e.g., read-write.c which handles certain generalized parts of read/write operations such as buffering and page-caching. For example, the functions in read-write.c might be used by multiple specific file system implementations.
I asked Google Gemini to organize the files within fs by listing the top level functions together with the files which implement each of the top level functions. Here below is Gemini's list. I commented a few places where Gemini's list didn't seem in accordance with Linus' HEAD.
Key Subsystems and Representative Files (fs/ directory):
Virtual Filesystem (VFS) Core: (Provides the common interface and infrastructure for all filesystems)
fs/namei.c: Pathname lookup (resolving paths to inodes).
fs/file.c: File object management (representing open files).
fs/stat.c: File status information (stat() system call).
fs/open.c: File opening (open() system call).
fs/read_write.c: Core read/write operations and helpers (often used by specific filesystems). Not the primary entry point, but provides important utilities.
I'm curious why the fs directory doesn't have subdirectories organized according to function. For example, there could be fs/vfs-core, fs/implementations, fs/caching, etc.
Apparently BSD kernels also have a Virtual Filesystem Layer ("VFS"). How early in Unix did the VFS layer appear? Was a VFS common in pre-Unix operating systems?
It's easy to imagine that ZFS conceivably might have its own implementation of some of the helper systems or even its own version of the VFS, and that these within-the-specific-filesystem implementations might have contributed to possible "difficulty" of including ZFS in the Linus kernel proper. I'm curious why ZFS wasn't included. Do we know?
Is anybody aware of a well recognized, introductory Linux kernel or BSD source code discussion which follows the "overview / top-down" style of analysis which I am trying to use here?
Is anybody up on the mistakes Google Gemini apparently made? Could these be as simple as, "Oh, that's version x.xx?"
Thanks for any help on any of these questions! Thanks for additional observations and comments!
Does anybody else want to join the server? If yes, please check How to Apply in the OP. Thanks @Hosteroid!
Hello!
I'd like a slice for experimenting with NixOS and hosting some open-source projects (Vaultwarden, Whoogle search, Openwebui). If you need any additional infos about me, I'll happy to provide through PM.
My ssh public key is ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIHZ7KokkDS4XU9M15R3htHbt4ZJ9NQeYxVbKWinbE3n5.
Thank you!
@Not_Oles said: I'm curious why ZFS wasn't included. Do we know?
Not being that familiar with the history of the relevant discussions I don't know if there are also other (technical, political, social, whatever) reasons, but the reason that is usually given is the legal uncertainty of GPLv2 (the Linux license) and CDDL (the ZFS license) compatibility. Here's one summary, with links to other discussion (including opposing opinions):
@quangthang said:
Hello!
I'd like a slice for experimenting with NixOS and hosting some open-source projects (Vaultwarden, Whoogle search, Openwebui). If you need any additional infos about me, I'll happy to provide through PM.
My ssh public key is ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIHZ7KokkDS4XU9M15R3htHbt4ZJ9NQeYxVbKWinbE3n5.
Thank you!
Thanks for your request, which makes me happy! Of course, I definitely do remember you from previous servers, and so I am very delighted to have you joining hlcs!
I sent you login info. You are sharing root with me and with @cmeerw.
Please feel free to do whatever you want as long as it's White Hat. Please post here about what you are doing and about any questions you have.
Hello everyone.
I just got around to do some basic setup (set up user account, change to fish shell).
BashVM is installed
It seems like it's not installed globally. I cloned the repo into my home dir, ran the install script again but it mostly skipping cause required pkgs and configs are already installed
...
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
dnsmasq is already installed.
Default storage pool is already active.
Default network is already active.
...
I'm gonna go ahead and see if I can make a NixOS VM.
The installation process was successful but it's stuck on this screen. I've must have done something wrong. I think I'll grab the graphical iso to use their graphical installer.
One of dependencies on my config keep failing build. Which is weird cause it built just fine in other instances and my local laptop. Anyway, I'm going to call it a day.
@quangthang said:
One of dependencies on my config keep failing build. Which is weird cause it built just fine in other instances and my local laptop. Anyway, I'm going to call it a day.
Isn't that just a timing thing, i.e. it took 6 ms too long?
@Not_Oles said: I'm curious why ZFS wasn't included. Do we know?
Not being that familiar with the history of the relevant discussions I don't know if there are also other (technical, political, social, whatever) reasons, but the reason that is usually given is the legal uncertainty of GPLv2 (the Linux license) and CDDL (the ZFS license) compatibility. Here's one summary, with links to other discussion (including opposing opinions):
Sorry about my delayed response to your comment. I wanted to wait until I had a chance to look at the SFConservancy link you provided.
I really appreciate your comment for at least four reasons.
First, your comment concisely and directly answers the question I asked and not some other question.
Second, your comment provides excellent context: you sketched a typology of possible reasons why some software project might or might not be included in another software project: licensing, technical, social, political, other.
Third, your comment includes a link to relevant and comprehensive further discussion from multiple, inconsistent points-of-view.
Fourth, the tone of your comment is friendly and positive and helpful instead of negative and snarky.
Now that, thanks to you, I might understand a little better, I have to decide what to do.
It seems like Debian and the Conservancy and Oracle might have arrived, for the time being, at somewhat of a compromise regarding the use of ZFS on Linux. I guess that compromise might be good enough for me to continue glancing at ZFS on Debian on our Community Server. Nevertheless, I now imagine that I might want a really important reason before continuing with ZFS beyond an educational glance. I might want some specific context where ZFS clearly would be a better choice than any of the other available file systems.
Thanks again for your really helpful comment! <3 Thanks to @Hosteroid for our beautiful Community Server!
Hi @quangthang! I've seen your posts above but I haven't had a chance to read them carefully. Luckily @cmeerw was here and kindly responded! I'm glad you got in to the server! Yaaay!
I got a chance to read your posts and to look up a few things. I've never used NixOS, so I don't have anything to add to what you and @cmeerw already have said.
I hope you go ahead on the server and have a lot of fun! Best! <)
@quangthang said:
One of dependencies on my config keep failing build. Which is weird cause it built just fine in other instances and my local laptop. Anyway, I'm going to call it a day.
Isn't that just a timing thing, i.e. it took 6 ms too long?
Yes, It looks like the unit test is running performance evaluation. I don't expect it to fail by that small margin though 😭
If it's okay with @cmeerw, maybe @quangthang might like to try installing NixOS on the entire bare metal server? The server has an HTML5 IPMI available. We could then keep going with NixOS or change back to Debian or to something else later. Whatever people want is fine with me.
@Not_Oles said:
If it's okay with @cmeerw, maybe @quangthang might like to try installing NixOS on the entire bare metal server? The server has an HTML5 IPMI available. We could then keep going with NixOS or change back to Debian or to something else later. Whatever people want is fine with me.
I would prefer keeping the existing Debian installation on the server, and just installing NixOS onto its own logical volume to (dual-)boot into NixOS (so we could either relatively easily switch back to Debian or mount the existing data/home partition in NixOS), if @quangthang wants to run NixOS on bare metal.
When I installed bashvm I wasn't sure where to put it. I wanted everybody to be able to read it, but I didn't want to change the root $PATH or the user $PATH.
What I did was put bashvm in /usr/local.
root@hlcs:~/.ssh# cd /usr/local
root@hlcs:/usr/local# ls
bashvm bin etc games include lib man plan9 sbin share src
root@hlcs:/usr/local# echo $PATH
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/pkg/bin:/usr/local/plan9/bin
root@hlcs:/usr/local# cd bashvm
root@hlcs:/usr/local/bashvm# ls
arm bashvm-dhcpv6-network-auto.sh bashvm.sh
bashvm-add-port-forwarding.sh bashvm-dhcpv6-network-manual.sh bashvm-show-port-forwarding.sh
bashvm-create-auto-vm.sh bashvm-installer.sh LICENSE
bashvm-create-vm.sh bashvm-monitor.sh README.md
bashvm-delete-auto-vm.sh bashvm-remove-port-forwarding.sh
root@hlcs:/usr/local/bashvm#
Since everybody has root, maybe it might be okay to add bashvm to the $PATH?
root@hlcs:~# cp -p .bashrc .bashrc~
root@hlcs:~# ed .bashrc # This is GNU ed. Maybe I should use `9 ed`. /s
769
a
PATH=$PATH:/usr/local/bashvm export PATH
.
w
810
q
root@hlcs:~# tail -n 5 .bashrc
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/pkg/bin export PATH
MANPATH=:/usr/pkg/man export MANPATH
PLAN9=/usr/local/plan9 export PLAN9
PATH=$PATH:$PLAN9/bin export PATH
PATH=$PATH:/usr/local/bashvm export PATH
root@hlcs:~# logout
Connection to xxx.xxx.xxx.xxx closed.
chronos@penguin:~/servers/hosteroid$ `head -n 1 login` # In bash, I type this as Ctrl-r ` /s
Linux hlcs.metalvps.com 6.1.0-28-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.119-1 (2024-11-22) x86_64
[ . . . ]
Last login: Fri Jan 10 23:35:26 2025 from xxx.xxx.xxx.xxx
root@hlcs:~# echo $PATH
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/pkg/bin:/usr/local/plan9/bin:/usr/local/bashvm
root@hlcs:~#
@Not_Oles said:
If it's okay with @cmeerw, maybe @quangthang might like to try installing NixOS on the entire bare metal server? The server has an HTML5 IPMI available. We could then keep going with NixOS or change back to Debian or to something else later. Whatever people want is fine with me.
I would prefer keeping the existing Debian installation on the server, and just installing NixOS onto its own logical volume to (dual-)boot into NixOS (so we could either relatively easily switch back to Debian or mount the existing data/home partition in NixOS), if @quangthang wants to run NixOS on bare metal.
Hello! I got another server temporarily so that I can mess with LVM without risk of causing disruption here. More on LVM in a little while.
If I understand right, @quangthang wants to try running Nix on ARM architecture, and our server is Intel.
@cmeerw Might you have plans to use our nice Community Server for another project any time before too long?
Does anyone else want to use our nice Community Server? Please check the OP section on How to Apply. Thanks!
Does anybody else (@cmeerw?) want to take over from me the administration of our nice Community Server?
If no LESbians want to use our server in the near future and if no one among us wants to take over its administration, maybe we should consider returning the server to Hosteroid?
@Not_Oles said:
Hello! I got another server temporarily so that I can mess with LVM without risk of causing disruption here. More on LVM in a little while.
If I understand right, @quangthang wants to try running Nix on ARM architecture, and our server is Intel.
@cmeerw Might you have plans to use our nice Community Server for another project any time before too long?
Not sure... one thing that's on the back of my mind is the use of OpenStreetMap data - so maybe I could start looking at tilemaker. What's never clear with all of these OSM related projects is how much resources these projects need - so I guess you just have to try and see.
Does anyone else want to use our nice Community Server? Please check the OP section on How to Apply. Thanks!
Does anybody else (@cmeerw?) want to take over from me the administration of our nice Community Server?
Do you mean just the technical side or also the non-technical side (like user sign-ups)?
@cmeerw said: Not sure... one thing that's on the back of my mind is the use of OpenStreetMap data - so maybe I could start looking at tilemaker. What's never clear with all of these OSM related projects is how much resources these projects need - so I guess you just have to try and see.
@cmeerw said: maybe I could start looking at tilemaker.
Sounds great! No rush.
As I said, I'm working on the filesystem stuff on another server, for a little while.
To be clear, my main concern is staying fair with Hosteroid. If at least somebody has an upcoming use for the Community Server -- like your tilemaker idea -- then that's great! On the other hand, if nobody has anything coming up, then that's still great, but conceivably better use might be made of our fine Community Server.
I'd be happiest if we had a few more Neighbors, maybe two or three? Enough people who want to post actively to maybe keep the thread on the front page most of the time while providing fascinating content for LES.
If the server continues with just you and me, that's super great -- as long as it's also okay with Hosteroid.
@cmeerw said: Do you mean just the technical side or also the non-technical side (like user sign-ups)?
I meant both, although without trying to push anything in any particular direction. I wondered if there is something about me which prevents our LESbian friends from using the Community Server.
Maybe it's the ID out-in-public or known-by-me requirement. I would stand on that requirement until I know another way of helping the kiddies enjoy their fun elsewhere.
Maybe it's my relative lack of technical ability and experience which could be keeping more of our LESbian friends away? Certainly, if you were to formally take over the tech side, nobody could have any complaints. Of course, generous guy that you are, you're already doing server administration whenever you see something that needs to be done. Also, since you are sharing the root, I am never in your way, which is how I like things to be.
Processing user sign ups is mostly just saying no to people who haven't read the OP. Others, like you and quangthang just get in right away, without much processing beyond adding their keys.
If you want to do anything differently, please just say so. Same with @Hosteroid. I'm glad to have learned that you do have the tilemaker idea coming up! I'm glad for having given you and Hosteroid an opportunity to help decide what happens. Thanks to both of you for listening!
@cmeerw said: maybe I could start looking at tilemaker.
Sounds great! No rush.
As I said, I'm working on the filesystem stuff on another server, for a little while.
To be clear, my main concern is staying fair with Hosteroid. If at least somebody has an upcoming use for the Community Server -- like your tilemaker idea -- then that's great! On the other hand, if nobody has anything coming up, then that's still great, but conceivably better use might be made of our fine Community Server.
It definitely shouldn't just idle all the time - that would be too wasteful.
BTW, did some test runs yesterday with tilemaker, and got it to convert "South America" to vector tiles (but already using around 11 GB memory during the process and creating a 5 GB output file). Will have to see if it will be able to handle the larger regions from https://download.geofabrik.de/ (Europe is about 10 times the size for the input file) when not keeping everything in RAM.
I'd be happiest if we had a few more Neighbors, maybe two or three? Enough people who want to post actively to maybe keep the thread on the front page most of the time while providing fascinating content for LES.
Absolutely!
If the server continues with just you and me, that's super great -- as long as it's also okay with Hosteroid.
@cmeerw said: Do you mean just the technical side or also the non-technical side (like user sign-ups)?
I meant both, although without trying to push anything in any particular direction. I wondered if there is something about me which prevents our LESbian friends from using the Community Server.
As long as the server is running something Debian based, I am happy to keep an eye on the server. But I don't think I would be any good at handling user sign-ups - I am probably way too opinionated here (as soon as the main reason for signing up is to run some pre-made VM image, it's already a "no" for me)
As long as the server is running something Debian based, I am happy to keep an eye on the server.
Thanks so much!
Debian is great, of course, and Debian also is very widely adopted. That's why, originally, I asked @Hosteroid for Debian, since I thought Debian would be the most appreciated OS by the majority here at LES. However, since it seems just you and me on the Community Server for a while, we could switch to NetBSD, or to anything else, any time you wish. Whatever OS you want to run, and whenever you want to switch, or not, is okay with me.
But I don't think I would be any good at handling user sign-ups - I am probably way too opinionated here (as soon as the main reason for signing up is to run some pre-made VM image, it's already a "no" for me)
Haha, somehow I think you would be great at user sign-ups!
@Not_Oles said:
I wondered if there is something about me which prevents our LESbian friends from using the Community Server.
Nobody dares to rely on your "community" server because it's "ephemeral".
Reboot every day!
Reinstall every week!
Cancel next month!
Sure, if you want something you can rely on, sign up with a commercial provider. I don't think the "community" server should in any way try to compete with commercial providers or take away business from them (particularly as it's being sponsored by a commercial provider).
In my mind, the idea is to provide the resources where you can try out an idea (that you otherwise wouldn't do), and if it works out and you want to continue with it long-term, you can then look for more stable hosting options (and maybe even sign up with @Hosteroid)
Comments
Our Hosteroid Low End Spirit Community Server ("hlcs") is using Hosteroid's default Debian distribution, as far as I know. Hosteroid originally, and very kindly, gave me hlcs already installed.
I haven't gone through the contents of non-free, but, awhile back, I found out the hard way that gcc man pages are in non-free. Apparently there is a conflict between the GNU Free Documentation License ("GDFL") and the Debian Free Software Guidelines ("DFSG"). Please see
https://unix.stackexchange.com/questions/287075/why-cant-i-fetch-the-gcc-doc-package-on-debian
https://people.debian.org/~srivasta/Position_Statement.xhtml
Without adding non-free to sources.list, one cannot use apt to install the gcc man pages, and contrib is necessary in sources.list to install additional gcc documentation (the gcc-doc package).
I'd be delighted to consider other arrangements if there is anything you want to do differently. If you want, we could consider reinstalling with a different flavor of Debian, a different Linux distribution, or a non-Linux OS.
If anyone else wants to join us, surely we would consider their view about what we might install or reinstall.
Thanks again @cmeerw! Thanks also to @Hosteroid for our fun, free hlcs server!
looks like @Not_Oles might have some @bikegremlin's in the mix
The Yeti has left the building.
I want one to build a forum and use Flarum or Discourse.
My GitHub: https://github.com/kk14569
I have a signature.
Just checked on the status and they are still planning to do a formal security advisory. I'll post here once it gets published.
Hi @14569!
Thanks for your request!
It would be lovely to have you join us, and so I will keep trying to give you an account on our community server.
Thanks for telling us what you want to do on the server (make a forum with Flarum or Discourse) and for giving us a link to your Github profile (at https://github.com/kk14569).
Unfortunately, it hasn't been possible for me to understand who and where you are.
Quoting from the OP:
If you want to see an example for what identity information is needed, please check the example given in the OP:
When we look at Mason's example, we see his community presence here at LES via his LES profile. We also see who he is and where he is and what he does for work.
Your profile here and your Github seem impossible to use to determine who and where you are or what you do for work. Of course, I can imagine several very good reasons why you might not want to disclose your identity.
Again quoting from the OP:
If you wish, please feel free to provide more information about yourself. As I said, I will keep trying to get you an account.
Thanks again! Best wishes for the new year 2025!
Tom
Happy holidays @Not_Oles and @Hosteroid. Thanks for being awesome and helping people eliminate barriers to self discovery and education.
I moved all my websites to Carrd, and cancelled all my idlers.
Save the earth, don't idle.
broked it? can I brokeded it?
The Yeti has left the building.
Is it a good idea to use zfs on Linux tho? Worth the trouble or not?
Pretty sure it's a horrible idea to use it on Arch Linux but what about Debian stable?
I really don't know! But @Cloud suggested trying it. So, why not try iit?
We can leave ext4, xfs, btrfs, and zfs on the server until somebody needs the space for something. Or we can wipe the server and do something different if everyone wants to go differently.
I see your recommendation here:
What difference does the Linux distro make on file system choice?
Best wishes!
Thanks to you, @ususk,
and to @Hosteroid! 
I use btrfs on Arch Linux for the snapshot feature. Sometimes need to roll back after I pacman -Syu and break something. I know zfs is capable of this and everyone tells me zfs is better than btrfs so I've been wanting to give it a try if it's worth the trouble
Ok, I'll send you my info via messages.
I have a signature.
Personally, I try to stay away from anything that's not in the mainline kernel.
So far we are only using 64 GB for each of ext4, xfs, and btfs, so there is still plenty of space available. And as it's all logical volumes, we can extend each as needed. As long as we are reasonably conservative with allocating space to each of these logical volumes, we should be fine.
I am just hoping that ZFS doesn't get too upset when it's not allowed to use up all the space available, and gets put onto a logical volume (like everyone else).
When ZFS gets upset, it is because it is not allowed to use up all the memory available
We seem to be close to articulating the reason why I stick to ext4. There are so many technical advantage to the advanced file systems. These make me jealous! But then there also are horror stories every once in awhile on HN. The file systems seem a lot like big cloud provider billing. Every once in awhile. . . .
Life seems peaceful on a little, old dedi with ext4.
I was primarily interested in the performance numbers between ZFS and the others as you already did some measurements. I am not expecting any surprises really, but just curious to how they compare.
Yes ZFS has its reputation of being a memory hog, but you can tune ZFS pretty well to run nicely even on a low memory environment. I have it running on 512MB BuyVM boxes just fine. Snapshots, checksums, encryption etc are all very useful.
@Crab
Following up on your kind suggestion to try ZFS, I posted above:
Besides having to study up on ZFS itself, at minimum, I also need at least to
study up on logical block addressing ("LBA");
install the ZFS packages @Crab mentioned above:
set up the logical volume for ZFS (see @cmeerw's use of
lvcreateat https://lowendspirit.com/discussion/comment/198328/#Comment_198328 and at https://lowendspirit.com/discussion/comment/199230/#Comment_199230)format the ZFS partition;
add the ZFS partition to /etc/fstab; and
then, maybe, I will be able to run Yabs on ZFS to get its
fiotests.Additionally, I hope to invest a few seconds glancing at the source code for some of these file systems.
So that's my fun plan for the next few hours, days, or weeks on hlcs
, depending on how long it takes and on how much other fun stuff I am doing elsewhere. Probably each step will show up in a post in this thread.
Any goofs? Any gripes?
Happy New Year @Hosteroid! @cmeerw! @Crab! @everyone!
That sounds like a solid plan and I'm sure you are going to have a lot of fun with it. As a little twist, you can also try zfs-fuse to understand how much performance is being lost with the FUSE layer compared to a kernel module.
Happy New Year everybody!
Thanks! Appreciate your looking out for me!
https://en.wikipedia.org/wiki/Filesystem_in_Userspace
https://packages.debian.org/bookworm/zfs-fuse
Sounds like even more fun! Thanks for the suggestion!
Just submitted a change for Haiku to fix an issue with the daily builds affecting Emacs (that's just a drive-by fix and not yet the fix I was talking about earlier).
Thanks to @hosteroid for providing the dev machine.
Hello!
I wanted to glance at the Linux kernel filesystem code before installing more filesystems. In order to be 100% sure I wouldn't mess up our Community Server (Thanks @Hosteroid!) I cloned the kernel code with git on a VPS elsewhere.
Since there might be a few people here at LES who haven't cloned the Linux kernel code, here are the steps I followed.
I copied the terminal output during the clone, when the receiving objects step was 91% done:
A moment later, when the clone was finished, I could see
I heard the the filesystem code lives in the
fsdirectory. So I took a look.Hey! We can see ext4 in there! And xfs! And btrfs! If I understand right, the ZFS code is not present, but needs to be added via a kernel module.
I hear from Google Gemini that the contents of the fs directory includes:
The Virtual File System Layer (VFS) -- abstracts common operations across various file systems to enable presenting a unified interface to user applications,
Specific File System Implementations -- btrfs, ext4, xfs, etc. -- each file systems implementation contains a "file_operations" structure with pointers to the filesystem-specific functions which implement user-level system call operations, such as read, write, open, and ioctl, and
Additional, related "helper" implementations -- e.g., read-write.c which handles certain generalized parts of read/write operations such as buffering and page-caching. For example, the functions in read-write.c might be used by multiple specific file system implementations.
I asked Google Gemini to organize the files within fs by listing the top level functions together with the files which implement each of the top level functions. Here below is Gemini's list. I commented a few places where Gemini's list didn't seem in accordance with Linus' HEAD.
Key Subsystems and Representative Files (fs/ directory):
Virtual Filesystem (VFS) Core: (Provides the common interface and infrastructure for all filesystems)
fs/dcache.c: Directory entry cache (dentry).fs/inode.c: Inode management (core inode operations).fs/namei.c: Pathname lookup (resolving paths to inodes).fs/file.c: File object management (representing open files).fs/stat.c: File status information (stat()system call).fs/open.c: File opening (open()system call).fs/read_write.c: Core read/write operations and helpers (often used by specific filesystems). Not the primary entry point, but provides important utilities.fs/super.c: Superblock management (representing mounted filesystems).fs/vfs_kern.c: Miscellaneous VFS kernel functions. # No vfs_kern.cfs/libfs.c: Helper functions for various filesystem operations.fs/mount.c: Mounting and unmounting filesystems. # No mount.c. There is mount.h.Specific Filesystem Implementations: (Provide the concrete implementation for each filesystem type)
fs/ext4/: (Entire directory) Ext4 filesystem.fs/xfs/: (Entire directory) XFS filesystem.fs/btrfs/: (Entire directory) Btrfs filesystem.fs/fat/: (Entire directory) FAT filesystem.fs/ntfs/: (Entire directory) NTFS filesystem support.Buffer and Page Caching: (Manage caching of disk blocks and file data in memory)
fs/buffer.c: Buffer cache management (for disk blocks).fs/pagemap.c: Page cache management (for file data). # No pagemap.cfs/bio.c: Block I/O layer. Handles submission and completion of block I/O requests. #No bio.cQuota Management: (Limit disk usage by users and groups)
fs/quota/: (Entire directory) Disk quota management.File Locking: (Mechanisms for coordinating access to files)
fs/locks.c: File locking (e.g., flock, fcntl locks).Virtual Filesystems: (Provide filesystem-like interfaces to kernel resources)
fs/proc/: (Entire directory) procfs (process information).fs/sysfs/: (Entire directory) sysfs (kernel parameters and system information).fs/debugfs.c: debugfs (kernel debugging).fs/tmpfs/: (Entire directory) tmpfs (RAM-based filesystem). # No tmpfs subdirectoryNamespaces: (Isolate filesystems and other resources)
fs/nsfs.c: Namespace support (mount namespaces).Other Utilities and Helpers: (Various supporting functions and modules)
fs/ioctl.c: ioctl system calls (device-specific operations).fs/exec.c: Program execution (related to theexecve()system call).fs/fs-writeback.c: Writeback mechanism (flushing dirty data to disk).fs/char_dev.c: Character device management (though some character device drivers might be elsewhere).fs/fs_struct.c: Per-process filesystem information.I'm curious why the fs directory doesn't have subdirectories organized according to function. For example, there could be fs/vfs-core, fs/implementations, fs/caching, etc.
Apparently BSD kernels also have a Virtual Filesystem Layer ("VFS"). How early in Unix did the VFS layer appear? Was a VFS common in pre-Unix operating systems?
It's easy to imagine that ZFS conceivably might have its own implementation of some of the helper systems or even its own version of the VFS, and that these within-the-specific-filesystem implementations might have contributed to possible "difficulty" of including ZFS in the Linus kernel proper. I'm curious why ZFS wasn't included. Do we know?
Is anybody aware of a well recognized, introductory Linux kernel or BSD source code discussion which follows the "overview / top-down" style of analysis which I am trying to use here?
Is anybody up on the mistakes Google Gemini apparently made? Could these be as simple as, "Oh, that's version x.xx?"
Thanks for any help on any of these questions!
Thanks for additional observations and comments! 
Does anybody else want to join the server? If yes, please check How to Apply in the OP.
Thanks @Hosteroid! 
Hello!
I'd like a slice for experimenting with NixOS and hosting some open-source projects (Vaultwarden, Whoogle search, Openwebui). If you need any additional infos about me, I'll happy to provide through PM.
My ssh public key is
ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIHZ7KokkDS4XU9M15R3htHbt4ZJ9NQeYxVbKWinbE3n5.Thank you!
MetalVPS
Not being that familiar with the history of the relevant discussions I don't know if there are also other (technical, political, social, whatever) reasons, but the reason that is usually given is the legal uncertainty of GPLv2 (the Linux license) and CDDL (the ZFS license) compatibility. Here's one summary, with links to other discussion (including opposing opinions):
https://sfconservancy.org/blog/2016/feb/25/zfs-and-linux/
Hi @quangthang!
Thanks for your request, which makes me happy! Of course, I definitely do remember you from previous servers, and so I am very delighted to have you joining hlcs!
I sent you login info. You are sharing root with me and with @cmeerw.
Please feel free to do whatever you want as long as it's White Hat. Please post here about what you are doing and about any questions you have.
Always best wishes!
Tom
Thanks to @Hosteroid for our very nice server!
Hello everyone.
I just got around to do some basic setup (set up user account, change to fish shell).
It seems like it's not installed globally. I cloned the repo into my home dir, ran the install script again but it mostly skipping cause required pkgs and configs are already installed
I'm gonna go ahead and see if I can make a NixOS VM.
MetalVPS
VM created, now to the installation. Not sure if it's my part but i'm getting constant vnc timed out
MetalVPS
The installation process was successful but it's stuck on this screen. I've must have done something wrong. I think I'll grab the graphical iso to use their graphical installer.
MetalVPS
It's up!
MetalVPS
One of dependencies on my config keep failing build. Which is weird cause it built just fine in other instances and my local laptop. Anyway, I'm going to call it a day.
MetalVPS
Isn't that just a timing thing, i.e. it took 6 ms too long?
Hi @hobofl!
Sorry about my delayed response to your comment. I wanted to wait until I had a chance to look at the SFConservancy link you provided.
I really appreciate your comment for at least four reasons.
First, your comment concisely and directly answers the question I asked and not some other question.
Second, your comment provides excellent context: you sketched a typology of possible reasons why some software project might or might not be included in another software project: licensing, technical, social, political, other.
Third, your comment includes a link to relevant and comprehensive further discussion from multiple, inconsistent points-of-view.
Fourth, the tone of your comment is friendly and positive and helpful instead of negative and snarky.
Now that, thanks to you, I might understand a little better, I have to decide what to do.
It seems like Debian and the Conservancy and Oracle might have arrived, for the time being, at somewhat of a compromise regarding the use of ZFS on Linux. I guess that compromise might be good enough for me to continue glancing at ZFS on Debian on our Community Server. Nevertheless, I now imagine that I might want a really important reason before continuing with ZFS beyond an educational glance. I might want some specific context where ZFS clearly would be a better choice than any of the other available file systems.
Thanks again for your really helpful comment! <3 Thanks to @Hosteroid for our beautiful Community Server!
Hi @quangthang! I've seen your posts above but I haven't had a chance to read them carefully. Luckily @cmeerw was here and kindly responded! I'm glad you got in to the server! Yaaay!
Hi again @quangthang!
I got a chance to read your posts and to look up a few things. I've never used NixOS, so I don't have anything to add to what you and @cmeerw already have said.
I hope you go ahead on the server and have a lot of fun! Best! <)
Thanks to @Hosteroid for the lovely server!
Yes, It looks like the unit test is running performance evaluation. I don't expect it to fail by that small margin though 😭
MetalVPS
Make sure you use the same NixOS channel checkout to eliminate the "it works in my other computers, just not this one" issue.
DM us for private tracker invite.
Yes, I've basically copied my flake config over, with some boot loader config modification.
MetalVPS
If it's okay with @cmeerw, maybe @quangthang might like to try installing NixOS on the entire bare metal server? The server has an HTML5 IPMI available. We could then keep going with NixOS or change back to Debian or to something else later. Whatever people want is fine with me.
I would prefer keeping the existing Debian installation on the server, and just installing NixOS onto its own logical volume to (dual-)boot into NixOS (so we could either relatively easily switch back to Debian or mount the existing data/home partition in NixOS), if @quangthang wants to run NixOS on bare metal.
When I installed bashvm I wasn't sure where to put it. I wanted everybody to be able to read it, but I didn't want to change the root $PATH or the user $PATH.
What I did was put bashvm in
/usr/local.Since everybody has root, maybe it might be okay to add bashvm to the $PATH?
@cmeerw Excellent! Thanks for letting us know!
Just did an
apt-get dist-upgradeand a reboot to apply the Debian 12.9 updates.Hello! I got another server temporarily so that I can mess with LVM without risk of causing disruption here. More on LVM in a little while.
If I understand right, @quangthang wants to try running Nix on ARM architecture, and our server is Intel.
@cmeerw Might you have plans to use our nice Community Server for another project any time before too long?
Does anyone else want to use our nice Community Server? Please check the OP section on How to Apply. Thanks!
Does anybody else (@cmeerw?) want to take over from me the administration of our nice Community Server?
If no LESbians want to use our server in the near future and if no one among us wants to take over its administration, maybe we should consider returning the server to Hosteroid?
Thanks @Hosteroid!
Thanks LES! 
Not sure... one thing that's on the back of my mind is the use of OpenStreetMap data - so maybe I could start looking at tilemaker. What's never clear with all of these OSM related projects is how much resources these projects need - so I guess you just have to try and see.
Do you mean just the technical side or also the non-technical side (like user sign-ups)?
Just installed a few required packages for that
and with that it compiled fine so far
Sounds great! No rush.
As I said, I'm working on the filesystem stuff on another server, for a little while.
To be clear, my main concern is staying fair with Hosteroid. If at least somebody has an upcoming use for the Community Server -- like your tilemaker idea -- then that's great! On the other hand, if nobody has anything coming up, then that's still great, but conceivably better use might be made of our fine Community Server.
I'd be happiest if we had a few more Neighbors, maybe two or three? Enough people who want to post actively to maybe keep the thread on the front page most of the time while providing fascinating content for LES.
If the server continues with just you and me, that's super great -- as long as it's also okay with Hosteroid.
I meant both, although without trying to push anything in any particular direction. I wondered if there is something about me which prevents our LESbian friends from using the Community Server.
Maybe it's the ID out-in-public or known-by-me requirement. I would stand on that requirement until I know another way of helping the kiddies enjoy their fun elsewhere.
Maybe it's my relative lack of technical ability and experience which could be keeping more of our LESbian friends away? Certainly, if you were to formally take over the tech side, nobody could have any complaints. Of course, generous guy that you are, you're already doing server administration whenever you see something that needs to be done. Also, since you are sharing the root, I am never in your way, which is how I like things to be.
Processing user sign ups is mostly just saying no to people who haven't read the OP. Others, like you and quangthang just get in right away, without much processing beyond adding their keys.
If you want to do anything differently, please just say so. Same with @Hosteroid. I'm glad to have learned that you do have the tilemaker idea coming up! I'm glad for having given you and Hosteroid an opportunity to help decide what happens. Thanks to both of you for listening!
It definitely shouldn't just idle all the time - that would be too wasteful.
BTW, did some test runs yesterday with tilemaker, and got it to convert "South America" to vector tiles (but already using around 11 GB memory during the process and creating a 5 GB output file). Will have to see if it will be able to handle the larger regions from https://download.geofabrik.de/ (Europe is about 10 times the size for the input file) when not keeping everything in RAM.
Absolutely!
As long as the server is running something Debian based, I am happy to keep an eye on the server. But I don't think I would be any good at handling user sign-ups - I am probably way too opinionated here (as soon as the main reason for signing up is to run some pre-made VM image, it's already a "no" for me)
Thanks so much!
Debian is great, of course, and Debian also is very widely adopted. That's why, originally, I asked @Hosteroid
for Debian, since I thought Debian would be the most appreciated OS by the majority here at LES. However, since it seems just you and me on the Community Server for a while, we could switch to NetBSD, or to anything else, any time you wish. Whatever OS you want to run, and whenever you want to switch, or not, is okay with me.
Haha, somehow I think you would be great at user sign-ups!
Nobody dares to rely on your "community" server because it's "ephemeral".
Reboot every day!
Reinstall every week!
Cancel next month!
We accept Karma donations for the last flan. 🍮 affbrr
Sure, if you want something you can rely on, sign up with a commercial provider. I don't think the "community" server should in any way try to compete with commercial providers or take away business from them (particularly as it's being sponsored by a commercial provider).
In my mind, the idea is to provide the resources where you can try out an idea (that you otherwise wouldn't do), and if it works out and you want to continue with it long-term, you can then look for more stable hosting options (and maybe even sign up with @Hosteroid)