@linveo said: I have loaded up the latest NetBSD 10 template and added a new one for 9.4. Thanks again!
Thanks so much. The NetBSD 10 image works as expected.
The NetBSD 9.4 image works, except that it doesn't see any configuration data, so no network or SSH keys are configured. I am not sure if this an issue with NetBSD 9.4 or if that config data is missing from the VirtFusion side (on my local tests, NetBSD 9.4 seemed to see the same drives as NetBSD 10).
If you have any instructions I can follow to create qcow2 templates, I can try as well. I was given some documentation from VF to create my own, but not sure how well it will work.
I have made my scripts available here - this script just needs to be run as root on a NetBSD 10 host (it should work without any additional dependencies on a minimal NetBSD 10 installation)
./build.sh -e -v 10.0
will create a netbsd-10.0.raw file.
That can then be converted with qemu-img -c -O qcow2 netbsd-10.0.raw netbsd-10.0.qcow2 to a qcow2 image (qemu-img is not installed on the minimal NetBSD 10 image, but you could do that step on Linux).
(for NetBSD 9.4 I use ./build.sh -v 9.4 without the -e (EFI boot) option as that didn't seem to work, so the 9.4 image is BIOS boot only - which is what VirtFusion is using anyway).
I followed the above linked steps in the Guide, Now I have my unprivileged user, tom, who successfully can connect via ssh and use su.
I hope that my posting reference links and steps I followed might encourage others who haven't yet tried building their entire NetBSD system to actually try it.
Next up will be making the build directories and downloading and updating the sources.
Thanks to @linveo for providing the fun test VM for free and to the BSD guys here for watching my back and catching my mistakes.
Switching to tom, checking the environment variables, then switching to tom with su and double checking the environment variables. Skipping the X Window System sources for now.
linveo# su tom
linveo$ whoami
tom
linveo$ echo $CVSROOT
[email protected]:/cvsroot
linveo$ echo $CVS_RSH
ssh
linveo$ whoami
tom
linveo$
Haha, since it wasn't running inside tmux, I borked the above checkout by shutting my Chromebook when I wanted to go to sleep. Probably all that was necessary was to rerun the checkout command, which I did, and the results seemed okay.
For the sake of completeness, I moved src to src-old, made a new src directory owned by tom, and reran the checkout from the beginning, inside tmux. A few lines of terminal output from new checkout are shown below. It looks like the full checkout took around 3.5 hours.
I think that the final checkout "Updating" lines which mention X are not a problem. If I remember right, NetBSD together with pkgsrc offer X.org and XFree86.org versions of X. There also seem to be some X libraries in the main NetBSD sources.
Next up might be to see whether the newly checked out sources build successfully. In the newly checked out README.md file, it says:
Building
You can cross-build NetBSD from most UNIX-like operating systems.
To build for amd64 (x86_64), in the src directory:
./build.sh -U -u -j4 -m amd64 -O ~/obj release
I think I'd have to use -j 2 on my two vCore VPS. I'm not sure if ~/obj needs to exist. I should skim build.sh.
Wondering about whether the VPS has enough memory. Wondering about the effect of the inode issue mentioned above. Wondering how long the build will take, if it completes.
Decided to try a compile, because, why not? Following nia's procedure in the above linked page, I extracted the tar file into /home/tom and made a /home/tom/obj directory.
Ran this build.sh command inside tmux: time ./build.sh -U -u -j 2 -m amd64 -O ~/obj release
Since starting the build, it's been maybe half an hour while I have been writing this post, and the build seems to be still running.
@linveo Linveo control panel shows CPU utilization ranging from about 88% to about 102.3%.
linveo# su - tom
linveo$ pwd
/home/tom
linveo$ whoami
tom
linveo$ mkdir obj
linveo$ tar xf /usr/src-new-checkout-20241014.tar
linveo$ ls -l
total 3
drwxr-xr-x 2 tom users 512 Oct 14 21:11 obj
-rw------- 1 tom users 14 Oct 11 02:24 password
drwxr-xr-x 25 tom users 512 Oct 14 09:15 src
linveo$ cd src
linveo$ ls
BUILDING UPDATING crypto external regress tests
CVS bin dist games rescue tools
Makefile build.sh distrib include sbin usr.bin
Makefile.inc common doc lib share usr.sbin
README.md compat etc libexec sys
linveo$ ls -l
total 373
-rw-r--r-- 1 tom users 42791 Apr 26 17:38 BUILDING
drwxr-xr-x 2 tom users 512 Oct 14 09:15 CVS
-rw-r--r-- 1 tom users 16367 Sep 8 2023 Makefile
-rw-r--r-- 1 tom users 355 May 2 2018 Makefile.inc
-rw-r--r-- 1 tom users 1748 Sep 5 2021 README.md
-rw-r--r-- 1 tom users 20004 Sep 26 20:08 UPDATING
drwxr-xr-x 37 tom users 1024 Oct 14 09:15 bin
-rwxr-xr-x 1 tom users 73189 Jul 23 20:46 build.sh
drwxr-xr-x 6 tom users 512 Oct 14 05:45 common
drwxr-xr-x 10 tom users 512 Oct 14 09:15 compat
drwxr-xr-x 5 tom users 512 Oct 14 05:47 crypto
drwxr-xr-x 4 tom users 512 Oct 14 09:14 dist
drwxr-xr-x 61 tom users 1536 Oct 14 09:14 distrib
drwxr-xr-x 4 tom users 512 Oct 14 06:05 doc
drwxr-xr-x 77 tom users 3072 Oct 14 09:13 etc
drwxr-xr-x 23 tom users 512 Oct 14 09:02 external
drwxr-xr-x 54 tom users 1024 Oct 14 09:01 games
drwxr-xr-x 8 tom users 2048 Oct 14 08:59 include
drwxr-xr-x 67 tom users 1536 Oct 14 08:59 lib
drwxr-xr-x 29 tom users 1024 Oct 14 08:59 libexec
drwxr-xr-x 6 tom users 512 Oct 14 08:59 regress
drwxr-xr-x 3 tom users 512 Oct 14 08:14 rescue
drwxr-xr-x 116 tom users 2560 Oct 14 08:59 sbin
drwxr-xr-x 20 tom users 512 Oct 14 08:59 share
drwxr-xr-x 37 tom users 1024 Oct 14 08:59 sys
drwxr-xr-x 22 tom users 512 Oct 14 08:57 tests
drwxr-xr-x 120 tom users 2560 Oct 14 08:57 tools
drwxr-xr-x 260 tom users 4608 Oct 14 08:57 usr.bin
drwxr-xr-x 171 tom users 3072 Oct 14 08:57 usr.sbin
linveo$ date
Tue Oct 15 00:32:09 UTC 2024
linveo$ time ./build.sh -U -u -j 2 -m amd64 -O ~/obj release
===> build.sh command: ./build.sh -U -u -j 2 -m amd64 -O /home/tom/obj release
===> build.sh started: Tue Oct 15 00:32:27 UTC 2024
===> NetBSD version: 10.99.12
===> MACHINE: amd64
===> MACHINE_ARCH: x86_64
===> Build platform: NetBSD 10.99.12 amd64
===> HOST_SH: /bin/sh
===> No $TOOLDIR/bin/nbmake, needs building.
===> Bootstrapping nbmake
[ . . . ]
I just grabbed the following output lines as they flashed by:
make distribution started at: Tue Oct 15 00:32:39 UTC 2024
make distribution finished at: Tue Oct 15 04:45:35 UTC 2024
[ . . . ]
/home/tom/obj/tooldir.NetBSD-10.99.12-amd64/bin/nbmake -C /home/tom/obj/sys/arch/amd64/compile/GENERIC depend && /home/tom/obj/tooldir.NetBSD-10.99.12-amd64/bin/nbmake -C /home/tom/obj/sys/arch/amd64/compile/GENERIC && /home/tom/obj/tooldir.NetBSD-10.99.12-amd64/bin/nbmake -C /home/tom/obj/sys/arch/amd64/compile/GENERIC debuginstall
Time for sleep soon.
I wonder if we are going to run out if disk space.
linveo# df -h .
Filesystem Size Used Avail %Cap Mounted on
/dev/dk2 20G 15G 4.0G 78% /
linveo#
[ 874397.7878684] /: write failed, file system is full
[ 874397.7878684] /: write failed, file system is full
[ 874397.7878684] /: write failed, file system is full
[ 874397.7878684] /: write failed, file system is full
[ 874397.7878684] /: write failed, file system is full
[ 874397.7878684] /: write failed, file system is full
[ 874397.7878684] /: write failed, file system is full
[ 874397.7878684] /: write failed, file system is full
you have mail
linveo$ df -h .
Filesystem Size Used Avail %Cap Mounted on
/dev/dk2 20G 19G 540K 99% /
linveo$
@Not_Oles said: Anybody have a genius idea? Or two or three?
Looks like you didn't start with the new NetBSD template, which would have given you maybe 4 GB more space on the file system (not sure if that would have been enough space).
Just out of interest, how many inodes are you using? df -hi /
Filesystem Size Used Avail %Cap iUsed iAvail %iCap Mounted on
/dev/dk2 24G 1.3G 21G 5% 16853 3100457 0% /
One of the reasons why I went ahead with the old image was that I wondered whether the large number of inodes would cause any kind of a problem. Looks like, until we ran out of space, everything went fine despite the large number of inodes.
When things started looking like space was going to get tight, I deleted pkgsrc and also my previously mentioned initial src checkout which had been moved to src-old.
If @linveo wants to bump my VM's disk size, I am happy to rebuild NetBSD, this time beginning by reinstalling with @cmeerw's latest image. @linveo Rather than increasing disk size, if it's easier to wipe the existing VM and provision a replacement, that's fine. If increasing my VM's disk size causes any issues, it's also fine to leave it as is. I really like this VM! It has a fast processor, and it has only 23 ms ping from my current location in Sonora.
Google ""NetBSD" inode max 3100457" says "Your search did not match any documents".
@cmeerw If you do not mind, may I please ask, what is the best way to find out where, when, and why the inode max was increased? Why was 3100457 selected?
Google ""NetBSD" inode max 3100457" says "Your search did not match any documents".
@cmeerw If you do not mind, may I please ask, what is the best way to find out where, when, and why the inode max was increased? Why was 3100457 selected?
Sorry... that's just the (max) number of inodes you will get when re-installing NetBSD from my template on that particular VM.
-i bytes-per-inode
This specifies the density of inodes in the file system. If
fewer inodes are desired, a larger number should be used; to
create more inodes a smaller number should be given. The
default is to create an inode for every (4 * frag-size) bytes
of data space:
Note that when creating the NetBSD image I am creating a 512 MB root filesystem, but as I am expecting the file system to be resized during first boot, I am setting "-b 16384 -f 2048" (so we get more reasonable defaults for the final filesystem size).
So the initial 512 MB file system will get approximately 65 thousand inodes, and when the filesystem gets resized to 24.5 GB, the inode density stays the same and you'll end up with roughly 3 million inodes.
BTW, the actual maximum number of inodes is 3117310 (3100457 was the number of available inodes on my system), but I think df only shows that number when using df -G / (and the values are currently swapped around on 10.0).
@Not_Oles said:
If @linveo wants to bump my VM's disk size, I am happy to rebuild NetBSD, this time beginning by reinstalling with @cmeerw's latest image. @linveo Rather than increasing disk size, if it's easier to wipe the existing VM and provision a replacement, that's fine. If increasing my VM's disk size causes any issues, it's also fine to leave it as is. I really like this VM! It has a fast processor, and it has only 23 ms ping from my current location in Sonora.
You got it! I upped your VM to 50GB, just need a reboot. You might need to manually increase the FS too if cloud-init is not able to interact.
@linveo said: You got it! I upped your VM to 50GB, just need a reboot.
@linveo The pending changes show up on the Control Panel at my end. Before I do anything, though, I just want to double check with @cmeerw because he's been working on the NetBSD images. If there is anything that @cmeerw might want to do or check or ask me to do or check, I want to make sure that whatever @cmeerw wants is done.
My inclination might be to reinstall from @cmeerw's latest NetBSD image. I might just do Debian for a moment first, simply to exercise the system and to double check that everything works with an OS that VF might more likely be expecting. But I could proceed differently if @cmeerw or anybody else wants. Thanks!
@Not_Oles said: Before I do anything, though, I just want to double check with @cmeerw because he's been working on the NetBSD images. If there is anything that @cmeerw might want to do or check or ask me to do or check, I want to make sure that whatever @cmeerw wants is done.
I believe your version of the image might actually just resize the partition and file system on the next reboot (that's because in the first version I didn't mount the filesystem with the "log" option).
Later versions now use the "log" option - this is preferable, but it means that resizing (after the first boot) gets more complicated (or even dangerous).
So I think, just try a restart of the machine and see if the filesystem gets resized.
After that, I think it still makes sense to move to the new image to get some more useful block sizes and inode counts for the filesystem (it should get resized to the full capacity on the first boot, but further resizes won't be automatic).
BTW, I actually tried booting into the OpenBSD 7.6 installer (via netboot.xyz) when it was released, but it did hang just before getting to user space.
BTW, I actually tried booting into the OpenBSD 7.6 installer (via netboot.xyz) when it was released, but it did hang just before getting to user space.
I've upgraded a handful of my 7.5 instances to 7.6 thus far with no issues. I haven't done my Linveo VM yet but when I do if I run into any problems I'll let you know. I didn't have any problems getting 7.5 installed using netboot.xyz though so I don't expect any issues with 7.6 either.
How come only 40880 1M-blocks total when @linveo increased the size to 50GB? Does using so many extra inodes somehow take up a substantial portion of the "missing" 9120 blocks?
Since there now is more disk space and since the system still seems to be working okay, it might be fun to continue the system build, just to see whether it breaks, and, if so, where and how and why. Of course, after retrying the system build, we can go ahead and reinstall.
@Not_Oles said: How come only 40880 1M-blocks total when @linveo increased the size to 50GB? Does using so many extra inodes somehow take up a substantial portion of the "missing" 9120 blocks?
Yes, that's the inodes/filesystem overhead (and there is a 512 MB swap partition).
BTW, you can also look at the partitions with gpt show ld0 and dkctl ld0 listwedges
It's probably very simple, but I have to figure out how to install the newly built distribution and then see if the VM will reboot.
Then we can reinstall from @cmeerw's new image to fix the inode issue.
Hmm. I wonder if there might be a way to fix the inode issue without reinstalling the VM from the new image. Like, for example, create and move critical parts of the OS into a memory resident filesystem, then wipe and reinstall the disk based filesystem, download full NetBDSD to the new disk based filesystem, get sources, and rebuild.
Since everything "just works" every day and night, there's not much to report but 100% uptime across the board!
FreeBSD 14.2 is going to arrive in early December, so that'll bring some new excitement and bunch of work to get all the systems updated, but not expecting any hiccups there.
People have been very busy with this thread and I am super happy to see that. Thank you @linveo for providing these great resources to the community and @cmeerw for the excellent work on NetBSD!
Also I wanted to add a tip of the day, check out this great command line utility with an awesome modern approach: https://github.com/aristocratos/bpytop
In stasis until the shitposting stops/abates.
Than=compare;then=sequence:brought=bring;bought=buy:staffs=pile of sticks:informations/infos=no plural. It wisnae me! A big boy done it and ran away. || NVMe2G for life! until death (the end is nigh).
@Not_Oles said: It looks like we only needed a little more space than we had.
So it would have worked with the new filesystem settings with the original disk space allocation.
@Not_Oles said: Hmm. I wonder if there might be a way to fix the inode issue without reinstalling the VM from the new image. Like, for example, create and move critical parts of the OS into a memory resident filesystem, then wipe and reinstall the disk based filesystem, download full NetBDSD to the new disk based filesystem, get sources, and rebuild.
Well... there is a 512 MB swap partition you could use as a temporary alternative root filesystem.
Thanks @linveo! The OpenBSD install post looks great! From the above linked overview, there are links to a step-by-step walkthrough, which links to a page about setting up a desktop environment.
I hope to try the OpenBSD install after I get NetBSD to rebuild itself as -current (done), install the newly rebuilt kernel (upcoming), and reboot (upcoming), and then install (upcoming) the already built, -current userland (done).
@linveo If you haven't already, maybe you could kindly pass on to your customer our invitation to join us here in the LES BSD thread? Thanks again!
Here is moving our previous faithful and hard working kernel to "retired" status, copying the newly compiled kernel from the /obj directory to /, and adjusting the mode to match the previous kernel.
linveo# date
Tue Oct 22 18:55:26 UTC 2024
linveo# pwd
/home/tom/obj/sys/arch/amd64/compile/GENERIC
linveo# ls -l netbsd
-rwxr-xr-x 1 tom users 29572928 Oct 19 23:56 netbsd
linveo# ls -l /netbsd
-rw-r--r-- 1 root wheel 29572760 Oct 5 00:39 /netbsd
linveo# mv /netbsd /netbsd-old
linveo# cp netbsd /
linveo# ls -l /netbsd*
-rwxr-xr-x 1 root wheel 29572928 Oct 22 18:59 /netbsd
-rw-r--r-- 1 root wheel 29572760 Oct 5 00:39 /netbsd-old
linveo# chmod 644 /netbsd
linveo# ls -l /netbsd*
-rw-r--r-- 1 root wheel 29572928 Oct 22 18:59 /netbsd
-rw-r--r-- 1 root wheel 29572760 Oct 5 00:39 /netbsd-old
linveo# date; shutdown -r now
Tue Oct 22 19:07:28 UTC 2024
Shutdown NOW!
shutdown: [pid 5108]
linveo#
*** FINAL System shutdown message from [email protected] ***
System going down IMMEDIATELY
System shutdown time has arrived
About to run shutdown hooks...
Stopping cron.
Stopping inetd.
Saved entropy to /var/db/entropy-file.
Forcibly unmounting /tmp
Forcibly unmounting /var/shm
Removing block-type swap devices
swapctl: removing /dev/dk1 as swap device
Tue Oct 22 19:07:33 UTC 2024
Done running shutdown hooks.
Connection to xxx.xxx.xxx.xxx closed by remote host.
Connection to xxx.xxx.xxx.xxx closed.
chronos@penguin:~/servers/linveo$
Now to reboot and hopefully come back up running our self-compiled NetBSD-current GENERIC kernel.
Did it work?
chronos@penguin:~/servers/linveo$ `head -n 1 login`
Last login: Tue Oct 22 18:17:38 2024 from xxx.xxx.xxx.xxx
NetBSD 10.99.12 (GENERIC) #1: Sat Oct 19 23:52:58 UTC 2024
Welcome to NetBSD!
We recommend that you create a non-root account and use su(1) for root access.
linveo# date
Tue Oct 22 19:09:54 UTC 2024
linveo#
Next up is to install and, if changes are needed, to adjust the configuration of our newly compiled NetBSD-current userland. When the new userland has been installed, we will have reached our goal of running self-compiled NetBSD-current! All the sources will be really handy -- we can look at or change anything. If we get the userland up, then maybe we can try @AlwaysSkint's risky method of restructuring the filesystem to fix the inodes issue, or just reinstall from an ISO where the inodes issue already is fixed.
Thanks to @linveo for the really nice VPS! Thanks to @cmeerw, @AlwaysSkint and the other guys here for suggestions and for catching my mistakes.
I tweak the superblock allocation on some filesystems to eke out a little more space
egs. tune2fs -m0 /dev/system-backups and tune2fs -m1 /dev/system-tmp
Typically, I don't reduce root, however.
In stasis until the shitposting stops/abates.
Than=compare;then=sequence:brought=bring;bought=buy:staffs=pile of sticks:informations/infos=no plural. It wisnae me! A big boy done it and ran away. || NVMe2G for life! until death (the end is nigh).
@cmeerw said:
BTW, a few weeks ago I mentioned the cpuid tool - as of cpuid 20241023 my changes for NetBSD have been integrated, so it now also works on NetBSD.
For convenient reference, here is a link to the mention of cpuid by @cmeerw on September 24:
finally, today, I got around to trying to install the a-few-days-ago compiled userland.
I think I am in the same /home/tom/src directory that I used to build the distribution, but some things don't seem quite right. For example, build.sh is empty.
NetBSD 10.99.12 (GENERIC) #1: Sat Oct 19 23:52:58 UTC 2024
Welcome to NetBSD!
We recommend that you create a non-root account and use su(1) for root access.
linveo# cd /home/tom
linveo# cd src
linveo# pwd
/home/tom/src
linveo# ls
BUILDING Successful common doc lib share usr.sbin
CVS Summary compat etc libexec sys
Makefile UPDATING crypto external regress tests
Makefile.inc bin dist games rescue tools
README.md build.sh distrib include sbin usr.bin
linveo# date
Sat Oct 26 23:17:34 UTC 2024
linveo# time ./build.sh -O ../obj -T ../tools -U install=/
0.00 real 0.00 user 0.00 sys
linveo# linveo# wc S*
0 0 0 Successful
0 0 0 Summary
0 0 0 total
linveo# wc build.sh
0 0 0 build.sh
linveo# cat build.sh
linveo#
I do have
linveo# pwd
/home/tom/obj
linveo# linveo# ls -l releasedir/amd64/binary/sets/
total 205996
-rw-r--r-- 1 tom users 954 Oct 20 00:10 MD5
-rw-r--r-- 1 tom users 2637 Oct 20 00:10 SHA512
-rw-r--r-- 1 tom users 38278856 Oct 20 00:06 base.tar.xz
-rw-r--r-- 1 tom users 10913096 Oct 20 00:05 base32.tar.xz
-rw-r--r-- 1 tom users 82581160 Oct 20 00:10 comp.tar.xz
-rw-r--r-- 1 tom users 510172 Oct 20 00:06 etc.tar.xz
-rw-r--r-- 1 tom users 2550444 Oct 20 00:06 games.tar.xz
-rw-r--r-- 1 tom users 4143972 Oct 20 00:06 gpufw.tar.xz
-rw-r--r-- 1 tom users 8910200 Oct 19 23:58 kern-GENERIC.tar.xz
-rw-r--r-- 1 tom users 9135068 Oct 19 23:58 kern-GENERIC_KASLR.tar.xz
-rw-r--r-- 1 tom users 6645472 Oct 19 23:58 kern-XEN3_DOM0.tar.xz
-rw-r--r-- 1 tom users 2533980 Oct 19 23:58 kern-XEN3_DOMU.tar.xz
-rw-r--r-- 1 tom users 5705380 Oct 20 00:07 man.tar.xz
-rw-r--r-- 1 tom users 3488776 Oct 20 00:07 manhtml.tar.xz
-rw-r--r-- 1 tom users 4167024 Oct 20 00:07 misc.tar.xz
-rw-r--r-- 1 tom users 10104552 Oct 20 00:07 modules.tar.xz
-rw-r--r-- 1 tom users 3517528 Oct 20 00:07 rescue.tar.xz
-rw-r--r-- 1 tom users 15288428 Oct 20 00:08 tests.tar.xz
-rw-r--r-- 1 tom users 1921928 Oct 20 00:08 text.tar.xz
linveo#
I think I could just go extract base, comp, etc, games, man, misc, modules, rescue, tests, and text from /. Then run postinstall and etcupdate. But, what happened to build.sh in /home/tom/src? That's the same build.sh that was used to build the kernel. Maybe the loss of the build.sh file had something to do with running out of filesystem space? Maybe something to do with the inode issue? Did I make a mistake?
It's getting about time to reinstall. But it's fun to mess around with stuff!
So . . . the NetBSD-current VM still reboots successfully. I want to work more on etcupdate. And everything else. But maybe we have arrived at the goal of building and running NetBSD-current. Maybe it's time to reinstall with @cmeerw's newer image that has the number of inodes fixed.
If I rebuild NetBSD-current a few dozen more times, maybe I can start to remember to build as tom instead of root. It seems also that I can use "-j 2" with build.sh, and maybe that will decrease the build times.
If anybody catches other mistakes that I am making, I'd be glad to learn more!
Thanks to @linveo for a very fast, very nice VPS! Thanks to NetBSD! Thanks to the guys here in the LES BSD thread and to everyone else at LES!
linveo# shutdown -r now
Shutdown NOW!
shutdown: [pid 19887]
linveo#
*** FINAL System shutdown message from [email protected] ***
System going down IMMEDIATELY
System shutdown time has arrived
About to run shutdown hooks...
Stopping cron.
Stopping inetd.
Saved entropy to /var/db/entropy-file.
Forcibly unmounting /tmp
Forcibly unmounting /var/shm
Removing block-type swap devices
swapctl: removing /dev/dk1 as swap device
Mon Oct 28 17:14:44 UTC 2024
Done running shutdown hooks.
Connection to xxx.xxx.xxx.xxx closed by remote host.
Connection to xxx.xxx.xxx.xxx closed.
chronos@penguin:~/servers/linveo$ sleep 60
chronos@penguin:~/servers/linveo$ `head -n 1 login` | tee -a 20241027-build.sh-usr-src
Last login: Mon Oct 28 03:57:04 2024 from xxx.xxx.xxx.xxx
NetBSD 10.99.12 (GENERIC) #1: Sat Oct 19 23:52:58 UTC 2024
Welcome to NetBSD!
We recommend that you create a non-root account and use su(1) for root access.
linveo# date
Mon Oct 28 17:16:24 UTC 2024
linveo#
make distribution started at: Mon Oct 28 03:21:47 UTC 2024
make distribution finished at: Mon Oct 28 08:04:55 UTC 2024
The procedure for investigating was that /usr/src, /usr/tools, and /usr/obj were moved aside and replaced by new versions. The new /usr/src was extracted from the same tar file as previously used without the -j 2 option.
This time the job was run by user tom instead of by root.
Here is the full command:
linveo$ date; time ./build.sh -j 2 -O ../obj -T ../tools -U distribution; date
And here is the result summary showing that make distribution took 2 hours and 57 minutes:
make distribution started at: Mon Oct 28 22:27:46 UTC 2024
make distribution finished at: Tue Oct 29 01:24:08 UTC 2024
===> Successful make distribution
===> build.sh ended: Tue Oct 29 01:24:08 UTC 2024
===> Summary of results:
build.sh command: ./build.sh -j 2 -O ../obj -T ../tools -U distribution
build.sh started: Mon Oct 28 22:27:46 UTC 2024
NetBSD version: 10.99.12
MACHINE: amd64
MACHINE_ARCH: x86_64
Build platform: NetBSD 10.99.12 amd64
HOST_SH: /bin/sh
share/mk MAKECONF: /etc/mk.conf
MAKECONF file: /etc/mk.conf (File not found)
TOOLDIR path: /usr/src/../tools
DESTDIR path: /usr/src/../obj/destdir.amd64
RELEASEDIR path: /usr/src/../obj/releasedir
Updated makewrapper: /usr/src/../tools/bin/nbmake-amd64
Successful make distribution
build.sh ended: Tue Oct 29 01:24:08 UTC 2024
===> .
10582.23 real 7028.72 user 1949.24 sys
Tue Oct 29 01:24:08 UTC 2024
linveo$
So, -j 2 saved 1 hour and 46 minutes or 37%.
Below is the output of top during the -j 2 compile.
@linveo Thanks again for the fast VPS! Are you happy to see this usage sustained for three of four hours at a time multiple times per week or even per day? If this usage is too much, I can go run it on a dedi, no problem at all.
@linveo Thanks again for the fast VPS! Are you happy to see this usage sustained for three of four hours at a time multiple times per week or even per day? If this usage is too much, I can go run it on a dedi, no problem at all.
I am happy when people are using the VMs for a good purpose like this!
Hi bsd guys, new freebsd user here, taking advantage of Linveo's offer to learn a little FreeBSD and soon OpenBSD.
I have a question: which firewall should I use when learning freebsd? I have read the documentation and I see that there are at least 3 firewalls available. However, I don't know which one is recommended for FreeBSD, or which one is used by most people who use FreeBSD. At the moment I am learning PF firewall since I see that if I am not mistaken it is the one used by PFSense and the one that seems to me to be the most used among the FreeBSD community, however, since it is developed for OpenBSD, I do not know if it is the most indicated to use in FreeBSD.
My doubt here is really which is the firewall that integrates better with the operating system?
Comments
Thanks so much. The NetBSD 10 image works as expected.
The NetBSD 9.4 image works, except that it doesn't see any configuration data, so no network or SSH keys are configured. I am not sure if this an issue with NetBSD 9.4 or if that config data is missing from the VirtFusion side (on my local tests, NetBSD 9.4 seemed to see the same drives as NetBSD 10).
I have made my scripts available here - this script just needs to be run as root on a NetBSD 10 host (it should work without any additional dependencies on a minimal NetBSD 10 installation)
will create a
netbsd-10.0.rawfile.That can then be converted with
qemu-img -c -O qcow2 netbsd-10.0.raw netbsd-10.0.qcow2to a qcow2 image (qemu-imgis not installed on the minimal NetBSD 10 image, but you could do that step on Linux).(for NetBSD 9.4 I use
./build.sh -v 9.4without the-e(EFI boot) option as that didn't seem to work, so the 9.4 image is BIOS boot only - which is what VirtFusion is using anyway).Finally getting around to trying this.
The above linked guide says to make an unprivileged user account to use for building.
Documentation for how to make a user account is in Part III, Section 5.6 of the NetBSD Guide.
I followed the above linked steps in the Guide, Now I have my unprivileged user, tom, who successfully can connect via
sshand usesu.I hope that my posting reference links and steps I followed might encourage others who haven't yet tried building their entire NetBSD system to actually try it.
Next up will be making the build directories and downloading and updating the sources.
Thanks to @linveo for providing the fun test VM for free and to the BSD guys here for watching my back and catching my mistakes.
Now we have the directory for the sources. Reference: https://www.netbsd.org/docs/guide/en/chap-fetch.html#chap-fetch-dirs
linveo# cd /usr
linveo# pwd
/usr
linveo# ls /home
tom
linveo# mkdir /usr/src
linveo# chown tom /usr/src
linveo# ls -l | grep src
drwxrwxr-x 53 600 125 1024 Sep 28 01:01 pkgsrc
-rw-r--r-- 1 root wheel 84995861 Sep 28 01:15 pkgsrc.tar.gz
-rw-r--r-- 1 root wheel 64 Sep 28 01:15 pkgsrc.tar.gz.SHA1
drwxr-xr-x 2 tom wheel 512 Oct 14 00:11 src
linveo#
Setting the CVS environment variables as instructed in https://www.netbsd.org/docs/guide/en/chap-fetch.html#chap-fetch-cvs
linveo# pwd
/usr
linveo# export CVSROOT="[email protected]:/cvsroot"
linveo# echo $CVSROOT
[email protected]:/cvsroot
linveo# export CVS_RSH="ssh"
linveo# echo $CVS_RSH
ssh
linveo#
We have options to download tarballs of the sources, but, even though it is slower, let's try CVS as specified at https://www.netbsd.org/docs/guide/en/chap-fetch.html#chap-fetch-cvs-netbsd-current
Switching to
tom, checking the environment variables, then switching totomwithsuand double checking the environment variables. Skipping the X Window System sources for now.Now checking out the sources with CVS as per https://www.netbsd.org/docs/guide/en/chap-fetch.html#chap-fetch-cvs-netbsd-current
Haha, since it wasn't running inside
tmux, I borked the above checkout by shutting my Chromebook when I wanted to go to sleep. Probably all that was necessary was to rerun the checkout command, which I did, and the results seemed okay.For the sake of completeness, I moved
srctosrc-old, made a newsrcdirectory owned bytom, and reran the checkout from the beginning, insidetmux. A few lines of terminal output from new checkout are shown below. It looks like the full checkout took around 3.5 hours.I think that the final checkout "Updating" lines which mention X are not a problem. If I remember right, NetBSD together with pkgsrc offer X.org and XFree86.org versions of X. There also seem to be some X libraries in the main NetBSD sources.
Next up might be to see whether the newly checked out sources build successfully. In the newly checked out README.md file, it says:
./build.sh -U -u -j4 -m amd64 -O ~/obj releaseI think I'd have to use
-j 2on my two vCore VPS. I'm not sure if~/objneeds to exist. I should skimbuild.sh.I made a tar archive of the newly checked out
/srcso it's easy to revert if / when the build fails or for any other reason.Building NetBSD-current on a Linveo 2 vCore 2 GB RAM VPS
Reference: https://www.netbsd.org/docs/guide/en/chap-build.html
Wondering about whether the VPS has enough memory. Wondering about the effect of the inode issue mentioned above. Wondering how long the build will take, if it completes.
Decided to try a compile, because, why not? Following nia's procedure in the above linked page, I extracted the tar file into /home/tom and made a /home/tom/obj directory.
Ran this build.sh command inside tmux:
time ./build.sh -U -u -j 2 -m amd64 -O ~/obj releaseSince starting the build, it's been maybe half an hour while I have been writing this post, and the build seems to be still running.
@linveo Linveo control panel shows CPU utilization ranging from about 88% to about 102.3%.
Here's the current output of
topin case anybody might be interested.The build still seems like it's going strongly.
I just grabbed the following output lines as they flashed by:
Time for sleep soon.
I wonder if we are going to run out if disk space.
Not out of disk space yet, but it's getting close.
Uh . . . File system is full.
Sleep now for me. . . .
Anybody have a genius idea? Or two or three?
Looks like you didn't start with the new NetBSD template, which would have given you maybe 4 GB more space on the file system (not sure if that would have been enough space).
Just out of interest, how many inodes are you using?
df -hi /Right. I went with what I already had, just for fun, to see if it would work. Now I have had some fun, and learned a little. So, all good!
Yes.
16853 vs 795510
One of the reasons why I went ahead with the old image was that I wondered whether the large number of inodes would cause any kind of a problem. Looks like, until we ran out of space, everything went fine despite the large number of inodes.
When things started looking like space was going to get tight, I deleted pkgsrc and also my previously mentioned initial src checkout which had been moved to src-old.
If @linveo wants to bump my VM's disk size, I am happy to rebuild NetBSD, this time beginning by reinstalling with @cmeerw's latest image. @linveo Rather than increasing disk size, if it's easier to wipe the existing VM and provision a replacement, that's fine. If increasing my VM's disk size causes any issues, it's also fine to leave it as is. I really like this VM! It has a fast processor, and it has only 23 ms ping from my current location in Sonora.
Was actually wondering how many inodes you had used compared to the new max of 3100457 - your 795510 will still comfortably fit into that max.
Google ""NetBSD" inode max 3100457" says "Your search did not match any documents".
@cmeerw If you do not mind, may I please ask, what is the best way to find out where, when, and why the inode max was increased? Why was 3100457 selected?
Sorry... that's just the (max) number of inodes you will get when re-installing NetBSD from my template on that particular VM.
See the newfs man page:
Note that when creating the NetBSD image I am creating a 512 MB root filesystem, but as I am expecting the file system to be resized during first boot, I am setting "-b 16384 -f 2048" (so we get more reasonable defaults for the final filesystem size).
So the initial 512 MB file system will get approximately 65 thousand inodes, and when the filesystem gets resized to 24.5 GB, the inode density stays the same and you'll end up with roughly 3 million inodes.
BTW, the actual maximum number of inodes is
3117310(3100457was the number of available inodes on my system), but I thinkdfonly shows that number when usingdf -G /(and the values are currently swapped around on 10.0).You got it! I upped your VM to 50GB, just need a reboot. You might need to manually increase the FS too if cloud-init is not able to interact.
linveo.com | Shared Hosting | KVM VPS | Dedicated Servers
@linveo The pending changes show up on the Control Panel at my end. Before I do anything, though, I just want to double check with @cmeerw because he's been working on the NetBSD images. If there is anything that @cmeerw might want to do or check or ask me to do or check, I want to make sure that whatever @cmeerw wants is done.
My inclination might be to reinstall from @cmeerw's latest NetBSD image. I might just do Debian for a moment first, simply to exercise the system and to double check that everything works with an OS that VF might more likely be expecting. But I could proceed differently if @cmeerw or anybody else wants. Thanks!
@Crab What's up in the FreeBDSD world?
@FrankCastle And in the OpenBSD world?
I believe your version of the image might actually just resize the partition and file system on the next reboot (that's because in the first version I didn't mount the filesystem with the "log" option).
Later versions now use the "log" option - this is preferable, but it means that resizing (after the first boot) gets more complicated (or even dangerous).
So I think, just try a restart of the machine and see if the filesystem gets resized.
After that, I think it still makes sense to move to the new image to get some more useful block sizes and inode counts for the filesystem (it should get resized to the full capacity on the first boot, but further resizes won't be automatic).
BTW, I actually tried booting into the OpenBSD 7.6 installer (via netboot.xyz) when it was released, but it did hang just before getting to user space.
A new version just came out a couple weeks ago so I've been slowing upgrading all of my VMs to this latest version.
I've upgraded a handful of my 7.5 instances to 7.6 thus far with no issues. I haven't done my Linveo VM yet but when I do if I run into any problems I'll let you know. I didn't have any problems getting 7.5 installed using netboot.xyz though so I don't expect any issues with 7.6 either.
Rebooting from the Linveo panel worked!
Inode check for @cmeerw
How come only 40880 1M-blocks total when @linveo increased the size to 50GB? Does using so many extra inodes somehow take up a substantial portion of the "missing" 9120 blocks?
Since there now is more disk space and since the system still seems to be working okay, it might be fun to continue the system build, just to see whether it breaks, and, if so, where and how and why. Of course, after retrying the system build, we can go ahead and reinstall.
Yes, that's the inodes/filesystem overhead (and there is a 512 MB swap partition).
BTW, you can also look at the partitions with
gpt show ld0anddkctl ld0 listwedgesFinally I got around to retrying build.sh from where the file system became full, above.
The rebuild seems to be running okay. I will post again when it either breaks or completes.
Thanks to @linveo for a nice VPS and for additional disk space!
Thanks to @cmeerw for his NetBSD images, help, and encouragement! 
Wow!
Um . . . what now?
It looks like we only needed a little more space than we had.
When we ran out of file space before the size increase:
Now, after the filesystem size increase and after finishing the build:
It's probably very simple, but I have to figure out how to install the newly built distribution and then see if the VM will reboot.
Then we can reinstall from @cmeerw's new image to fix the inode issue.
Hmm. I wonder if there might be a way to fix the inode issue without reinstalling the VM from the new image. Like, for example, create and move critical parts of the OS into a memory resident filesystem, then wipe and reinstall the disk based filesystem, download full NetBDSD to the new disk based filesystem, get sources, and rebuild.
Since everything "just works" every day and night, there's not much to report but 100% uptime across the board!
FreeBSD 14.2 is going to arrive in early December, so that'll bring some new excitement and bunch of work to get all the systems updated, but not expecting any hiccups there.
People have been very busy with this thread and I am super happy to see that. Thank you @linveo for providing these great resources to the community and @cmeerw for the excellent work on NetBSD!
Also I wanted to add a tip of the day, check out this great command line utility with an awesome modern approach: https://github.com/aristocratos/bpytop
Risky:
In stasis until the shitposting stops/abates.
Than=compare;then=sequence:brought=bring;bought=buy:staffs=pile of sticks:informations/infos=no plural.
It wisnae me! A big boy done it and ran away. || NVMe2G for life! until death (the end is nigh).
So it would have worked with the new filesystem settings with the original disk space allocation.
Well... there is a 512 MB swap partition you could use as a temporary alternative root filesystem.
One of our awesome customers put together a guide on installing OpenBSD via ISO on our services. Take a look!
https://btxx.org/posts/openbsd-linveo/
linveo.com | Shared Hosting | KVM VPS | Dedicated Servers
Thanks @linveo! The OpenBSD install post looks great! From the above linked overview, there are links to a step-by-step walkthrough, which links to a page about setting up a desktop environment.
I hope to try the OpenBSD install after I get NetBSD to rebuild itself as -current (done), install the newly rebuilt kernel (upcoming), and reboot (upcoming), and then install (upcoming) the already built, -current userland (done).
@linveo If you haven't already, maybe you could kindly pass on to your customer our invitation to join us here in the LES BSD thread? Thanks again!
Interesting, I only ever get as far as this with OpenBSD:
Not that I really want to use OpenBSD, was just curious...
Here is moving our previous faithful and hard working kernel to "retired" status, copying the newly compiled kernel from the
/objdirectory to/, and adjusting the mode to match the previous kernel.Now to reboot and hopefully come back up running our self-compiled NetBSD-current GENERIC kernel.
Did it work?
Next up is to install and, if changes are needed, to adjust the configuration of our newly compiled NetBSD-current userland. When the new userland has been installed, we will have reached our goal of running self-compiled NetBSD-current! All the sources will be really handy -- we can look at or change anything. If we get the userland up, then maybe we can try @AlwaysSkint's risky method of restructuring the filesystem to fix the inodes issue, or just reinstall from an ISO where the inodes issue already is fixed.
Thanks to @linveo for the really nice VPS!
Thanks to @cmeerw, @AlwaysSkint and the other guys here for suggestions and for catching my mistakes. 
You mean the
tunefs -m? That won't do anything to fix the inodes issue - it only allows non-root users to use a bit more of the filesystem space.My bad, I'm old skool:
I tweak the superblock allocation on some filesystems to eke out a little more space
egs. tune2fs -m0 /dev/system-backups and tune2fs -m1 /dev/system-tmp
Typically, I don't reduce root, however.
In stasis until the shitposting stops/abates.
Than=compare;then=sequence:brought=bring;bought=buy:staffs=pile of sticks:informations/infos=no plural.
It wisnae me! A big boy done it and ran away. || NVMe2G for life! until death (the end is nigh).
This is the problem with x86_64 image. Regular x86 works fine.
https://btxx.org/posts/openbsd-linveo/ actually seems to use the amd64 image.
BTW, a few weeks ago I mentioned the cpuid tool - as of cpuid 20241023 my changes for NetBSD have been integrated, so it now also works on NetBSD.
For convenient reference, here is a link to the mention of cpuid by @cmeerw on September 24:
https://lowendspirit.com/discussion/comment/186451/#Comment_186451
where @cmeerw said:
There are several more mentions of cpuid in additional posts which can be seen by scrolling further down on September 24.
Congrats to @cmeerw on getting his patch accepted!
Reference: https://www.netbsd.org/docs/guide/en/chap-updating.html
After what a few days ago seemed like a successful build and reboot of a new kernel,
finally, today, I got around to trying to install the a-few-days-ago compiled userland.
I think I am in the same /home/tom/src directory that I used to build the distribution, but some things don't seem quite right. For example, build.sh is empty.
I do have
I think I could just go extract base, comp, etc, games, man, misc, modules, rescue, tests, and text from /. Then run postinstall and etcupdate. But, what happened to build.sh in /home/tom/src? That's the same build.sh that was used to build the kernel. Maybe the loss of the build.sh file had something to do with running out of filesystem space? Maybe something to do with the inode issue? Did I make a mistake?
It's getting about time to reinstall. But it's fun to mess around with stuff!
Just for fun, trying again, as root, to build and install the userland.
Reference: https://www.netbsd.org/docs/guide/en/chap-updating.html
Build
Install
Example Results In /bin
Run postinstall
Run etcupdate
Tried etcupdate for the first time, as far as I recall. It has lots of output! Left it for later.
So . . . the NetBSD-current VM still reboots successfully. I want to work more on etcupdate. And everything else. But maybe we have arrived at the goal of building and running NetBSD-current. Maybe it's time to reinstall with @cmeerw's newer image that has the number of inodes fixed.
If I rebuild NetBSD-current a few dozen more times, maybe I can start to remember to build as tom instead of root. It seems also that I can use "-j 2" with build.sh, and maybe that will decrease the build times.
If anybody catches other mistakes that I am making, I'd be glad to learn more!
Thanks to @linveo for a very fast, very nice VPS!
Thanks to NetBSD! 
Thanks to the guys here in the LES BSD thread and to everyone else at LES! 
How effective was adding
-j 2at reducingbuild.shcompile time?From above at https://lowendspirit.com/discussion/comment/188830/#Comment_188830 we can see that the time taken to compile the NetBSD-current distribution without any "jobs"
-jflag was about 4 hours and 43 minutes:The procedure for investigating was that
/usr/src,/usr/tools, and/usr/objwere moved aside and replaced by new versions. The new/usr/srcwas extracted from the same tar file as previously used without the-j 2option.This time the job was run by user
tominstead of byroot.Here is the full command:
linveo$ date; time ./build.sh -j 2 -O ../obj -T ../tools -U distribution; dateAnd here is the result summary showing that make distribution took 2 hours and 57 minutes:
So,
-j 2saved 1 hour and 46 minutes or 37%.Below is the output of
topduring the-j 2compile.@linveo Thanks again for the fast VPS!
Are you happy to see this usage sustained for three of four hours at a time multiple times per week or even per day? If this usage is too much, I can go run it on a dedi, no problem at all.
I am happy when people are using the VMs for a good purpose like this!
linveo.com | Shared Hosting | KVM VPS | Dedicated Servers
Reinstalled with NetBSD 10 image.
Thanks to @cmeerw for creating the image!
Thanks to @linveo for making @cmeerw's NetBSD 10 image available! 
Note the significant change in inodes available.
Formerly:
After NetBSD 10 reinstall:
Hi bsd guys, new freebsd user here, taking advantage of Linveo's offer to learn a little FreeBSD and soon OpenBSD.
I have a question: which firewall should I use when learning freebsd? I have read the documentation and I see that there are at least 3 firewalls available. However, I don't know which one is recommended for FreeBSD, or which one is used by most people who use FreeBSD. At the moment I am learning PF firewall since I see that if I am not mistaken it is the one used by PFSense and the one that seems to me to be the most used among the FreeBSD community, however, since it is developed for OpenBSD, I do not know if it is the most indicated to use in FreeBSD.
My doubt here is really which is the firewall that integrates better with the operating system?
Backend Ruby Dev and Linux user