Does VirtFusion have CPU topology settings hidden somewhere? When I look at the server creation API (https://docs.virtfusion.com/api/#api-Servers-Build), it mentions cpuCores in the hypervisor section, but I don't see API to manipulate them.
I was also trying to find some settings for the CPU model. BuyVM's panel for example allows to change CPU model between "AMD/Intel" and "QEMU".
@cmeerw is doing excellent investigation work at the kernel level, but I was just wondering whether the issue could actually be in the virtualization settings rather than in the kernel itself.
@Crab said:
Does VirtFusion have CPU topology settings hidden somewhere? When I look at the server creation API (https://docs.virtfusion.com/api/#api-Servers-Build), it mentions cpuCores in the hypervisor section, but I don't see API to manipulate them.
I was also trying to find some settings for the CPU model. BuyVM's panel for example allows to change CPU model between "AMD/Intel" and "QEMU".
@cmeerw is doing excellent investigation work at the kernel level, but I was just wondering whether the issue could actually be in the virtualization settings rather than in the kernel itself.
Just throwing some ideas into the air.
Yes, CPU topology is available by VM and also by package to assign the resources.
For CPU model, I don't believe this is user facing. I typically have this as host passthrough, but I am happy to change it for anyone that wants to try another setting such as QEMU.
@Crab said: @cmeerw is doing excellent investigation work at the kernel level, but I was just wondering whether the issue could actually be in the virtualization settings rather than in the kernel itself.
As I said here earlier
But it looks like something is not being reported correctly (by the CPU/virtualised CPU) here.
but NetBSD should probably be able to handle this situation better (by continuing with sensible settings instead of asserting), particularly if it's a somewhat common issue in virtualised environments (and other OSes like Linux and FreeBSD a happy with it).
Now on Ryzen - yahoo! Thanks @linveo (I thought it might have been a more lowly AMD.)
I booted from a manually mounted NetBSD ISO, setup partitions and tried autoconfigure/DHCP. DHCP wouldn't assign the public IP etc., so needed to manually add the network configuration (IPv4) in /etc/rc.conf, within the console.
I really like how compact NetBSD is - the VF backup only consumes 799MB. I wonder if I can make something useful out of this *nix. A great opportunity!
Thanks again Linveo.
In stasis until the shitposting stops/abates.
Than=compare;then=sequence:brought=bring;bought=buy:staffs=pile of sticks:informations/infos=no plural. It wisnae me! A big boy done it and ran away. || NVMe2G for life! until death (the end is nigh).
There doesn't appear to be a NetBSD package for qemu-guest-agent - seeing if I can build one from source - don't hold yer breath!
1st stumbling block: the basics!
On NetBSD, you need to install the “comp” and the “text” distribution sets.
Though one needs to guess the procedure, other than starting from the boot installation!
In stasis until the shitposting stops/abates.
Than=compare;then=sequence:brought=bring;bought=buy:staffs=pile of sticks:informations/infos=no plural. It wisnae me! A big boy done it and ran away. || NVMe2G for life! until death (the end is nigh).
^ Figured just to wget then untar from online binary/sets
In stasis until the shitposting stops/abates.
Than=compare;then=sequence:brought=bring;bought=buy:staffs=pile of sticks:informations/infos=no plural. It wisnae me! A big boy done it and ran away. || NVMe2G for life! until death (the end is nigh).
BTW, https://www.etallen.com/cpuid.html is a very useful tool to get all that detailed CPU information - just sent the author a patch to make it compile under NetBSD.
Not sure how many of you have seen https://cybersecuritynews.com/critical-unauthenticated-rce-flaw/amp/ but appears to be a pretty bad vulnerability in Linux on the horizon (isn't there always). No operating system is immune from vulnerabilities, even all of the BSDs have had their share in the past. But with Windows having market share of all desktops when there is a major exploit it is a big problem for the world and with Linux having a big chunk of the server market something like this could also become a huge deal. Those of us on BSD can hopefully breathe easy for this one.
Used pfsense for ~3 years on my homelab until the whole drama with netgate started affecting the platform (cough, wireguard, cough), opnsense and haven't looked back.
@cmeerw said: Just as a little experiment, I thought I'd give Haiku a try (as Beta 5 has just been released), and that also doesn't boot.
Haiku does actually work in the VM, I think I was just a bit too impatient last time as it needs quite a bit time to boot from the ISO image (as I believe VirtFusion is streaming the image data on demand via HTTP from the remote server).
I have put together a first NetBSD 10.0 image (somewhat based on the script from bsd-cloud-image.org, but without any cloud-init so far, so will need manual network configuration via VNC). @linveo do you want to add that as an installation image (it should boot on the AMD VMs)?
^ interesting; even I got it to boot under Proxmox but unfortunately I can't seem to emulate a 64-bit Intel on my lowend AMD FX..
In stasis until the shitposting stops/abates.
Than=compare;then=sequence:brought=bring;bought=buy:staffs=pile of sticks:informations/infos=no plural. It wisnae me! A big boy done it and ran away. || NVMe2G for life! until death (the end is nigh).
Seems that I got NetBSD installed inside Qemu running on Debian 12 with a Ryzen processor.
This is not on my Linveo VPS, which is happily running FreeBSD. But I will try NetBSD on the Linveo VPS soon. Probably I also should try an Intel processor. And I should try OpenBSD.
I like installig with the NetBSD boot-com.iso because I don't have to use a graphical install environment.
Thu Sep 26 03:08:00 UTC 2024
NetBSD/amd64 (netbsd.metalvps.com) (constty)
login: root
Password:
Sep 26 03:08:10 netbsd login: ROOT LOGIN (root) on tty constty
Copyright (c) 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003,
2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013,
2014, 2015, 2016, 2017, 2018, 2019, 2020, 2021, 2022, 2023,
2024
The NetBSD Foundation, Inc. All rights reserved.
Copyright (c) 1982, 1986, 1989, 1991, 1993
The Regents of the University of California. All rights reserved.
NetBSD 10.99.12 (GENERIC) #0: Mon Sep 23 07:35:44 UTC 2024
Welcome to NetBSD!
This is a development snapshot of NetBSD for testing -- user beware!
Bug reports: https://www.NetBSD.org/support/send-pr.html
Donations to the NetBSD Foundation: https://www.NetBSD.org/donations/
We recommend that you create a non-root account and use su(1) for root access.
netbsd#
I copied the above described NetBSD files previously installed on the Ryzen VPS to another VPS which is running on an Intel processor. Beyond copying the files, all I did on the second VPS was apt-get install qemu-system and bash boot-installed-netbsd.
SeaBIOS (version 1.16.2-debian-1.16.2-1)
iPXE (http://ipxe.org) 00:03.0 CA00 PCI2.10 PnP PMM+7EFCF060+7EF0F060 CA00
Booting from Hard Disk...
NetBSD MBR boot
NetBSD/x86 ffsv2 Primary Bootstrap
\\-__,------,___.
\\ __,---` NetBSD/x86 BIOS Boot
\\ `---,_. Revision 5.11 (Thu Sep 5 19:12:04 UTC 2024)
\\-,_____,.---`
\\
\\
\\
1. Boot normally
2. Boot single user
3. Drop to boot prompt
Choose an option; RETURN for default; SPACE to stop countdown.
Option 1 will be chosen in 0 seconds.
21687216+795904+1301248 [947522+1418328+1040172]=0x1b403c8
Loading /var/db/entropy-file
[ 1.0000000] WARNING: system needs entropy for security; see entropy(7)
[ 1.0000000] cpu_rng: rdrand/rdseed
[ 1.0000000] entropy: ready
[ 1.0000000] entropy: entering seed from bootloader with 256 bits of entropy
[ 1.0000000] Copyright (c) 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003,
[ 1.0000000] 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013,
[ 1.0000000] 2014, 2015, 2016, 2017, 2018, 2019, 2020, 2021, 2022, 2023,
[ 1.0000000] 2024
[ 1.0000000] The NetBSD Foundation, Inc. All rights reserved.
[ 1.0000000] Copyright (c) 1982, 1986, 1989, 1991, 1993
[ 1.0000000] The Regents of the University of California. All rights reserved.
[ 1.0000000] NetBSD 10.99.12 (GENERIC) #0: Mon Sep 23 07:35:44 UTC 2024
[ 1.0000000] [email protected]:/usr/src/sys/arch/amd64/compile/GENERIC
[ 1.0000000] total memory = 2047 MB
[ 1.0000000] avail memory = 1953 MB
[ 1.0000040] mainbus0 (root)
[ 1.0000040] ACPI: RSDP 0x00000000000F59D0 000014 (v00 BOCHS )
[ 1.0000040] ACPI: RSDT 0x000000007FFE1AC6 000034 (v01 BOCHS BXPC 00000001 BXPC 00000001)
[ 1.0000040] ACPI: FACP 0x000000007FFE197A 000074 (v01 BOCHS BXPC 00000001 BXPC 00000001)
[ 1.0000040] ACPI: DSDT 0x000000007FFE0040 00193A (v01 BOCHS BXPC 00000001 BXPC 00000001)
[ 1.0000040] ACPI: FACS 0x000000007FFE0000 000040
[ 1.0000040] ACPI: APIC 0x000000007FFE19EE 000078 (v01 BOCHS BXPC 00000001 BXPC 00000001)
[ 1.0000040] ACPI: HPET 0x000000007FFE1A66 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)
[ 1.0000040] ACPI: WAET 0x000000007FFE1A9E 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)
[ 1.0000040] ACPI: 1 ACPI AML tables successfully acquired and loaded
[ 1.0000040] ioapic0 at mainbus0 apid 0
[ 1.0000040] cpu0 at mainbus0 apid 0
[ 1.0000040] cpu0: Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz, id 0x406f1
[ 1.0000040] cpu0: node 0, package 0, core 0, smt 0
[ 1.0000040] acpi0 at mainbus0: Intel ACPICA 20240827
[ 1.0000040] acpi0: fixed power button present
[ 1.0260721] hpet0 at acpi0: high precision event timer (mem 0xfed00000-0xfed00400)
[ 1.0413615] qemufwcfg0 at acpi0 (FWCF, QEMU0002): io 0x510-0x51b
[ 1.0413615] pckbc1 at acpi0 (KBD, PNP0303) (kbd port): io 0x60,0x64 irq 1
[ 1.0413615] pckbc2 at acpi0 (MOU, PNP0F13) (aux port): irq 12
[ 1.0413615] fdc0 at acpi0 (FDC0, PNP0700): io 0x3f2-0x3f5,0x3f7 irq 6 drq 2
[ 1.0413615] lpt0 at acpi0 (LPT1, PNP0400-1): io 0x378-0x37f irq 7
[ 1.0413615] com0 at acpi0 (COM1, PNP0501-1): io 0x3f8-0x3ff irq 4
[ 1.0413615] com0: ns16550a, 16-byte FIFO
[ 1.0413615] com0: console
[ 1.0413615] ACPI: Enabled 2 GPEs in block 00 to 0F
[ 1.0413615] pckbd0 at pckbc1 (kbd slot)
[ 1.0413615] pckbc1: using irq 1 for kbd slot
[ 1.0413615] wskbd0 at pckbd0 mux 1
[ 1.0413615] pms0 at pckbc1 (aux slot)
[ 1.0413615] pckbc1: using irq 12 for aux slot
[ 1.0413615] wsmouse0 at pms0 mux 0
[ 1.0413615] pci0 at mainbus0 bus 0: configuration mode 1
[ 1.0413615] pchb0 at pci0 dev 0 function 0: Intel 82441FX (PMC) PCI and Memory Controller (rev. 0x02)
[ 1.0413615] pcib0 at pci0 dev 1 function 0: Intel 82371SB (PIIX3) PCI-ISA Bridge (rev. 0x00)
[ 1.0413615] piixide0 at pci0 dev 1 function 1: Intel 82371SB IDE Interface (PIIX3) (rev. 0x00)
[ 1.0413615] piixide0: primary channel interrupting at ioapic0 pin 14
[ 1.0413615] atabus0 at piixide0 channel 0
[ 1.0413615] piixide0: secondary channel interrupting at ioapic0 pin 15
[ 1.0413615] atabus1 at piixide0 channel 1
[ 1.0413615] piixpm0 at pci0 dev 1 function 3: Intel 82371AB (PIIX4) Power Management Controller (rev. 0x03)
[ 1.0413615] piixpm0: 24-bit timer
[ 1.0413615] piixpm0: interrupting at ioapic0 pin 9
[ 1.0413615] iic0 at piixpm0 port 0: I2C bus
[ 1.0413615] vga0 at pci0 dev 2 function 0: vendor 1234 product 1111 (rev. 0x02)
[ 1.0413615] wsdisplay0 at vga0 kbdmux 1
[ 1.0413615] drm at vga0 not configured
[ 1.0413615] wm0 at pci0 dev 3 function 0: Intel i82540EM 1000BASE-T Ethernet (rev. 0x03)
[ 1.0413615] wm0: interrupting at ioapic0 pin 11
[ 1.0413615] wm0: Ethernet address 52:54:00:12:34:56
[ 1.0413615] makphy0 at wm0 phy 1: Marvell 88E1011 Gigabit PHY, rev. 0
[ 1.0413615] makphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-FDX, auto
[ 1.0413615] isa0 at pcib0
[ 1.0413615] attimer0 at isa0 port 0x40-0x43
[ 1.0413615] pcppi0 at isa0 port 0x61
[ 1.0413615] spkr0 at pcppi0: PC Speaker
[ 1.0413615] wsbell at spkr0 not configured
[ 1.0413615] midi0 at pcppi0: PC speaker
[ 1.0413615] sysbeep0 at pcppi0
[ 1.0413615] attimer0: attached to pcppi0
[ 1.0413615] acpicpu0 at cpu0: ACPI CPU
[ 1.0413615] vmt0 at cpu0
[ 1.0413615] vmware: open failed, eax=0xffffffff, ecx=0x1e, edx=0x5658
[ 1.0413615] vmt0: failed to open backdoor RPC channel (TCLO protocol)
[ 1.5900700] fd0 at fdc0 drive 0: 1.44MB, 80 cyl, 2 head, 18 sec
[ 4.6008251] wd0 at atabus0 drive 0
[ 4.6117617] wd0: <QEMU HARDDISK>
[ 4.6231547] wd0: 51200 MB, 104025 cyl, 16 head, 63 sec, 512 bytes/sect x 104857600 sectors
[ 4.6608406] atapibus0 at atabus1: 2 targets
[ 4.6807660] cd0 at atapibus0 drive 0: <QEMU DVD-ROM, QM00003, 2.5+> cdrom removable
[ 4.6931056] swwdog0: software watchdog initialized
[ 4.7208087] WARNING: 1 error while detecting hardware; check system log.
[ 4.7371525] boot device: wd0
[ 4.7412069] root on wd0a dumps on wd0b
[ 4.7507688] root file system type: ffs
[ 4.7649995] kern.module.path=/stand/amd64/10.99.12/modules
Thu Sep 26 05:25:30 UTC 2024
Starting root file system check:
/dev/rwd0a: file system is clean; not checking
Setting sysctl variables:
ddb.onpanic: 1 -> 0
swapctl: setting dump device to /dev/wd0b
swapctl: adding /dev/wd0b as swap device at priority 0
Starting file system checks:
Loaded entropy from /var/db/entropy-file.
Waiting for entropy...done
Setting tty flags.
Starting network.
Hostname: netbsd.metalvps.com
IPv6 mode: host
Configuring network interfaces: wm0.
Adding interface aliases:.
Waiting for duplicate address detection to finish...
Starting dhcpcd.
Building databases: dev, utmp, utmpx.
Starting syslogd.
Setting date via ntp.
Mounting all file systems...
Clearing temporary files.
Checking quotas: done.
swapctl: setting dump device to /dev/wd0b
Starting virecover.
Checking for core dump...
savecore: no core dump
Starting local daemons:.
Updating motd.
Starting powerd.
Starting postfix.
postfix/postlog: starting the Postfix mail system
Starting inetd.
Starting cron.
Thu Sep 26 05:25:59 UTC 2024
NetBSD/amd64 (netbsd.metalvps.com) (constty)
login: root
Password:
Sep 26 05:26:11 netbsd login: ROOT LOGIN (root) on tty constty
Last login: Thu Sep 26 04:17:44 2024 on constty
Copyright (c) 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003,
2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013,
2014, 2015, 2016, 2017, 2018, 2019, 2020, 2021, 2022, 2023,
2024
The NetBSD Foundation, Inc. All rights reserved.
Copyright (c) 1982, 1986, 1989, 1991, 1993
The Regents of the University of California. All rights reserved.
NetBSD 10.99.12 (GENERIC) #0: Mon Sep 23 07:35:44 UTC 2024
Welcome to NetBSD!
This is a development snapshot of NetBSD for testing -- user beware!
Bug reports: https://www.NetBSD.org/support/send-pr.html
Donations to the NetBSD Foundation: https://www.NetBSD.org/donations/
We recommend that you create a non-root account and use su(1) for root access.
netbsd#
Just for fun, here are, from the Intel machine, the boot messages from the previous boot on the Ryzen machine.
If I add -smp 2 \ to the boot-installed-netbsd command frrom the Ryzen VPS but now copied to the Intel VPS, the next Qemu boot produces the full output shown below, including the two cores:
@Not_Oles said: If I add -smp 2 \ to the boot-installed-netbsd command frrom the Ryzen VPS but now copied to the Intel VPS, the next Qemu boot produces the full output shown below, including the two cores:
I wonder if it could be an old qemu or Linux kernel version running at linveo that could be responsible for the inconsistent cpuid information provided in their Intel VMs?
Apologies for marketing, but what @cmeerw said just above is a great example of the itch that MetalVPS was born to scratch. No one-click MetalVPS installs as yet, but MetalVPS neighbors can see what's going on inside the Node, including, as @cmeerw mentioned, the Node's kernel and Qemu versions.
@linveo Just having a quick look around the installation options on the VM dashboard. From the FreeBSDs only FreeBSD 13.2 seems to install (but only shows a single CPU core), with the other FreeBSDs I get an error at the "Creating configuration." stage.
^ Thought it was just me: I ended up template installing Debian, then restoring my FreeBSD backup. I did intend to start FreeBSD from scratch, using ext4 rather than ZFS 'cos one can shrink/resize ext4.
In stasis until the shitposting stops/abates.
Than=compare;then=sequence:brought=bring;bought=buy:staffs=pile of sticks:informations/infos=no plural. It wisnae me! A big boy done it and ran away. || NVMe2G for life! until death (the end is nigh).
@linveo I would like to surrender my BSD VPS as I am not able to spend enough time into it, please consider my request and let someone else have it if needed.
@cmeerw said:
I have put together a first NetBSD 10.0 image (somewhat based on the script from bsd-cloud-image.org, but without any cloud-init so far, so will need manual network configuration via VNC). @linveo do you want to add that as an installation image (it should boot on the AMD VMs)?
@Otus9051 said: @linveo I would like to surrender my BSD VPS as I am not able to spend enough time into it, please consider my request and let someone else have it if needed.
@Not_Oles said: Is anyone else interested in trying this cool NetBSD install method?
Absolutely! Gosh, not to expose how little I know, but I have never even heard of systemctl isolate
Hi @hornet! Yeah, same here! Also, it's been a few days since I last posted about From Linux to NetBSD, with SSH only, but I haven't tried it yet. I will post again when I do. If you try it sooner than I do, please let us know how it goes! Thanks! Tom
In stasis until the shitposting stops/abates.
Than=compare;then=sequence:brought=bring;bought=buy:staffs=pile of sticks:informations/infos=no plural. It wisnae me! A big boy done it and ran away. || NVMe2G for life! until death (the end is nigh).
Yes, for sure, if I am not mistaken, doesn't the method you mention require a graphical console via VNC or maybe HTML5? But the cloudbsd.xyz method does not require a graphical console. Lots of times a graphical console isn't provided, and the cloudbsd.xyz method nevertheless hopefully still works.
@Not_Oles said: the method you mention require a graphical console..
Fair point, though a text console is fine.
In stasis until the shitposting stops/abates.
Than=compare;then=sequence:brought=bring;bought=buy:staffs=pile of sticks:informations/infos=no plural. It wisnae me! A big boy done it and ran away. || NVMe2G for life! until death (the end is nigh).
@Not_Oles said: If I add -smp 2 \ to the boot-installed-netbsd command frrom the Ryzen VPS but now copied to the Intel VPS, the next Qemu boot produces the full output shown below, including the two cores:
I wonder if it could be an old qemu or Linux kernel version running at linveo that could be responsible for the inconsistent cpuid information provided in their Intel VMs?
The nodes are on QEMU 7.2.13 and Debian 12, so the kernel should be fairly up to date.
@cmeerw said: @linveo Just having a quick look around the installation options on the VM dashboard. From the FreeBSDs only FreeBSD 13.2 seems to install (but only shows a single CPU core), with the other FreeBSDs I get an error at the "Creating configuration." stage.
Do you mean none of the FreeBSD 14 versions work? I see a handful of servers running those templates at the moment.
@linveo said: I see a handful of servers running those templates at the moment.
I did at one point have issues with the 14.1 template but since have managed to boot it up. I had to manually configure the network, as DHCP wasn't working (from the cloud-init template?). Seems to be OK at the moment apart from locking myself out, setting up a firewall.;)
In stasis until the shitposting stops/abates.
Than=compare;then=sequence:brought=bring;bought=buy:staffs=pile of sticks:informations/infos=no plural. It wisnae me! A big boy done it and ran away. || NVMe2G for life! until death (the end is nigh).
@cmeerw said: @linveo Just having a quick look around the installation options on the VM dashboard. From the FreeBSDs only FreeBSD 13.2 seems to install (but only shows a single CPU core), with the other FreeBSDs I get an error at the "Creating configuration." stage.
Do you mean none of the FreeBSD 14 versions work? I see a handful of servers running those templates at the moment.
Seems to work today. Yesterday, only the FreeBSD 13.2 one worked (the others failed at the "Creating configuration." stage when rebuilding the server on VirtFusion).
I have put together another NetBSD 10.0 image. With some luck that should now even apply the network configuration and add SSH keys (if the configuration information is provided by VirtFusion).
@cmeerw said:
I have put together another NetBSD 10.0 image. With some luck that should now even apply the network configuration and add SSH keys (if the configuration information is provided by VirtFusion).
@cmeerw said:
I have put together another NetBSD 10.0 image. With some luck that should now even apply the network configuration and add SSH keys (if the configuration information is provided by VirtFusion).
@cmeerw said:
I have put together another NetBSD 10.0 image. With some luck that should now even apply the network configuration and add SSH keys (if the configuration information is provided by VirtFusion).
NetBSD 10.0 (GENERIC) #0: Thu Mar 28 08:33:33 UTC 2024
Welcome to NetBSD!
We recommend that you create a non-root account and use su(1) for root access.
linveo# date -u
Sat Sep 28 16:07:32 UTC 2024
linveo#
~~~
linveo# ping6 -c 2 ipv6.google.com
PING6(56=40+8+8 bytes) 2605:xxxx:xxxx:xxxx::xxxx --> 2607:f8b0:402a:80b::200e
16 bytes from 2607:f8b0:402a:80b::200e, icmp_seq=0 hlim=118 time=1.553 ms
16 bytes from 2607:f8b0:402a:80b::200e, icmp_seq=1 hlim=118 time=0.902 ms
--- ipv6.l.google.com ping6 statistics ---
2 packets transmitted, 2 packets received, 0.0% packet loss
round-trip min/avg/max/std-dev = 0.902/1.228/1.553/0.460 ms
linveo# ping -c 2 google.com
PING google.com (192.178.49.174): 56 data bytes
64 bytes from 192.178.49.174: icmp_seq=0 ttl=118 time=1.257350 ms
64 bytes from 192.178.49.174: icmp_seq=1 ttl=118 time=1.317900 ms
----google.com PING Statistics----
2 packets transmitted, 2 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 1.257350/1.287625/1.317900/0.042815 ms
linveo#
~~~~
@cmeerw said: @linveo Just having a quick look around the installation options on the VM dashboard. From the FreeBSDs only FreeBSD 13.2 seems to install (but only shows a single CPU core), with the other FreeBSDs I get an error at the "Creating configuration." stage.
Do you mean none of the FreeBSD 14 versions work? I see a handful of servers running those templates at the moment.
Seems to work today. Yesterday, only the FreeBSD 13.2 one worked (the others failed at the "Creating configuration." stage when rebuilding the server on VirtFusion).
I went through all FreeBSD templates on my Intel instance and they all got installed and worked fine. 13.3 did take some extra time bringing the network interface up, but eventually it started as well. When it comes to FreeBSD, virtio network drivers have pretty significant degradation on performance as far as I have seen on multiple providers and e1000 works generally much better. Some panels provide access to user to change them, but VirtFusion seems to lack that option.
@cmeerw said:
I have put together another NetBSD 10.0 image. With some luck that should now even apply the network configuration and add SSH keys (if the configuration information is provided by VirtFusion).
@Crab said: I tried it out with my Intel instance and I still see the kernel crash/reboot cycle like before. Please check how it works on AMD.
Yes, that's expected as I used the unpatched NetBSD kernel for that image. I can probably build another image with a patched kernel, now that the basics seem to be working.
Comments
@linveo Wow, that was easy!
Thank you so much!
Does VirtFusion have CPU topology settings hidden somewhere? When I look at the server creation API (https://docs.virtfusion.com/api/#api-Servers-Build), it mentions cpuCores in the hypervisor section, but I don't see API to manipulate them.
"cpuCores": { "units": "#", "max": 0, "allocated": 4, "free": -4, "percent": null }I was also trying to find some settings for the CPU model. BuyVM's panel for example allows to change CPU model between "AMD/Intel" and "QEMU".
@cmeerw is doing excellent investigation work at the kernel level, but I was just wondering whether the issue could actually be in the virtualization settings rather than in the kernel itself.
Just throwing some ideas into the air.
Yes, CPU topology is available by VM and also by package to assign the resources.
For CPU model, I don't believe this is user facing. I typically have this as host passthrough, but I am happy to change it for anyone that wants to try another setting such as QEMU.
linveo.com | Shared Hosting | KVM VPS | Dedicated Servers
Yes please change my VM's settings, let's give it a try!
I have changed yours over to QEMU 64bit.
linveo.com | Shared Hosting | KVM VPS | Dedicated Servers
It seems to be no-go. Powered the VM off and it doesn't even boot anymore. Control panel just says FAILED after trying for 49 seconds.
Changed it back. I wonder if it needs to be from scratch and this can't be changed mid install.
linveo.com | Shared Hosting | KVM VPS | Dedicated Servers
If you want to setup another VM with those settings, I can give it a try too.
As I said here earlier
but NetBSD should probably be able to handle this situation better (by continuing with sensible settings instead of asserting), particularly if it's a somewhat common issue in virtualised environments (and other OSes like Linux and FreeBSD a happy with it).
Now on Ryzen - yahoo!
Thanks @linveo (I thought it might have been a more lowly AMD.)
A great opportunity!
I booted from a manually mounted NetBSD ISO, setup partitions and tried autoconfigure/DHCP. DHCP wouldn't assign the public IP etc., so needed to manually add the network configuration (IPv4) in /etc/rc.conf, within the console.
I really like how compact NetBSD is - the VF backup only consumes 799MB. I wonder if I can make something useful out of this *nix.
Thanks again Linveo.
In stasis until the shitposting stops/abates.
Than=compare;then=sequence:brought=bring;bought=buy:staffs=pile of sticks:informations/infos=no plural.
It wisnae me! A big boy done it and ran away. || NVMe2G for life! until death (the end is nigh).
There doesn't appear to be a NetBSD package for qemu-guest-agent - seeing if I can build one from source - don't hold yer breath!
1st stumbling block: the basics!
Though one needs to guess the procedure, other than starting from the boot installation!
In stasis until the shitposting stops/abates.
Than=compare;then=sequence:brought=bring;bought=buy:staffs=pile of sticks:informations/infos=no plural.
It wisnae me! A big boy done it and ran away. || NVMe2G for life! until death (the end is nigh).
^ Figured just to wget then untar from online binary/sets
In stasis until the shitposting stops/abates.
Than=compare;then=sequence:brought=bring;bought=buy:staffs=pile of sticks:informations/infos=no plural.
It wisnae me! A big boy done it and ran away. || NVMe2G for life! until death (the end is nigh).
Could you maybe switch me over to AMD as well?
I have now created a bug report for the Intel issue https://gnats.netbsd.org/cgi-bin/query-pr-single.pl?number=58693 so it doesn't get lost, but not sure when/if that might get applied to the kernel sources.
I think it will be a lot easier not having to worry about having to use a patched kernel.
I have already started looking at the build scripts for the NetBSD images from https://bsd-cloud-image.org/
to build updated NetBSD 10.0 images: https://github.com/virt-lightning/netbsd-cloud-images/ (haven't looked at the cloud-init integration yet)
BTW, https://www.etallen.com/cpuid.html is a very useful tool to get all that detailed CPU information - just sent the author a patch to make it compile under NetBSD.
Not sure how many of you have seen https://cybersecuritynews.com/critical-unauthenticated-rce-flaw/amp/ but appears to be a pretty bad vulnerability in Linux on the horizon (isn't there always). No operating system is immune from vulnerabilities, even all of the BSDs have had their share in the past. But with Windows having market share of all desktops when there is a major exploit it is a big problem for the world and with Linux having a big chunk of the server market something like this could also become a huge deal. Those of us on BSD can hopefully breathe easy for this one.
Absolutely, you have been a huge help. I have migrated your VM over to Ryzen.
linveo.com | Shared Hosting | KVM VPS | Dedicated Servers
Thanks so much.
So looking at the CPU information on AMD, the CPUs show up as:
[ 1.000004] cpu0 at mainbus0 apid 0[ 1.000004] cpu0: Use mfence to serialize rdtsc[ 1.000004] cpu0: AMD Ryzen 9 7950X 16-Core Processor , id 0xa60f12[ 1.000004] cpu0: node 0, package 0, core 0, smt 0[ 1.000004] cpu1 at mainbus0 apid 1[ 1.000004] cpu1: AMD Ryzen 9 7950X 16-Core Processor , id 0xa60f12[ 1.000004] cpu1: node 0, package 1, core 0, smt 0which probably isn't really ideal either. I think they should show up as
core 0andcore 1(with both beingpackage 0).But looking at the information I get from
cpuid, there isn't much else the code could do here:hyper-threading / multi-core supported = falsenumber of threads = 0x1 (1)Other OSes probably do the same here, and as long as it works, nobody cares, I guess.
Used pfsense for ~3 years on my homelab until the whole drama with netgate started affecting the platform (cough, wireguard, cough), opnsense and haven't looked back.
Haiku does actually work in the VM, I think I was just a bit too impatient last time as it needs quite a bit time to boot from the ISO image (as I believe VirtFusion is streaming the image data on demand via HTTP from the remote server).
@linveo have the cloud-init settings changed on the VM with the move to AMD?
I used to see a small hard disk with the VM meta data on the Intel VM:
Sep 24 15:51:37 linveo /netbsd: [ 2.2808864] sd0 at scsibus3 target 0 lun 2: <QEMU, QEMU HARDDISK, 2.5+> disk fixedSep 24 15:51:37 linveo /netbsd: [ 2.2909223] sd0: 368 KB, 2 cyl, 16 head, 23 sec, 512 bytes/sect x 736 sectorsSep 24 15:51:37 linveo /netbsd: [ 2.2909223] sd0: async, 8-bit transfers, tagged queueingBut I no longer see that disk now on the AMD VM.
I have put together a first NetBSD 10.0 image (somewhat based on the script from bsd-cloud-image.org, but without any cloud-init so far, so will need manual network configuration via VNC). @linveo do you want to add that as an installation image (it should boot on the AMD VMs)?
^ interesting; even I got it to boot under Proxmox but unfortunately I can't seem to emulate a 64-bit Intel on my lowend AMD FX..
In stasis until the shitposting stops/abates.
Than=compare;then=sequence:brought=bring;bought=buy:staffs=pile of sticks:informations/infos=no plural.
It wisnae me! A big boy done it and ran away. || NVMe2G for life! until death (the end is nigh).
Seems that I got NetBSD installed inside Qemu running on Debian 12 with a Ryzen processor.
This is not on my Linveo VPS, which is happily running FreeBSD. But I will try NetBSD on the Linveo VPS soon. Probably I also should try an Intel processor. And I should try OpenBSD.
I like installig with the NetBSD boot-com.iso because I don't have to use a graphical install environment.
Here are some of the steps I followed:
Run the following two commands inside tmux or else kill the qemu process after shutdown of NetBSD.
I copied the above described NetBSD files previously installed on the Ryzen VPS to another VPS which is running on an Intel processor. Beyond copying the files, all I did on the second VPS was
apt-get install qemu-systemandbash boot-installed-netbsd.Just for fun, here are, from the Intel machine, the boot messages from the previous boot on the Ryzen machine.
If I add
-smp 2 \to theboot-installed-netbsdcommand frrom the Ryzen VPS but now copied to the Intel VPS, the next Qemu boot produces the full output shown below, including the two cores:Full output:
I wonder if it could be an old qemu or Linux kernel version running at linveo that could be responsible for the inconsistent cpuid information provided in their Intel VMs?
Apologies for marketing, but what @cmeerw said just above is a great example of the itch that MetalVPS was born to scratch. No one-click MetalVPS installs as yet, but MetalVPS neighbors can see what's going on inside the Node, including, as @cmeerw mentioned, the Node's kernel and Qemu versions.
From https://metalvps.com (emphasis added):
Thanks to YetiNode Free for hosting MetalVPS.com!
@linveo Just having a quick look around the installation options on the VM dashboard. From the FreeBSDs only FreeBSD 13.2 seems to install (but only shows a single CPU core), with the other FreeBSDs I get an error at the "Creating configuration." stage.
^ Thought it was just me: I ended up template installing Debian, then restoring my FreeBSD backup. I did intend to start FreeBSD from scratch, using ext4 rather than ZFS 'cos one can shrink/resize ext4.
In stasis until the shitposting stops/abates.
Than=compare;then=sequence:brought=bring;bought=buy:staffs=pile of sticks:informations/infos=no plural.
It wisnae me! A big boy done it and ran away. || NVMe2G for life! until death (the end is nigh).
@linveo I would like to surrender my BSD VPS as I am not able to spend enough time into it, please consider my request and let someone else have it if needed.
youtube.com/watch?v=k1BneeJTDcU
BTW, the script to build the image is here: https://bitbucket.org/cmeerw/netbsd-cloud-image/src/main/
Don’t leave us, only a few left. 😀 #Kidding…
Absolutely! Gosh, not to expose how little I know, but I have never even heard of systemctl isolate
Hi @hornet! Yeah, same here! Also, it's been a few days since I last posted about From Linux to NetBSD, with SSH only, but I haven't tried it yet. I will post again when I do. If you try it sooner than I do, please let us know how it goes! Thanks! Tom
^Thought I had/have mentioned a simpler way!
In stasis until the shitposting stops/abates.
Than=compare;then=sequence:brought=bring;bought=buy:staffs=pile of sticks:informations/infos=no plural.
It wisnae me! A big boy done it and ran away. || NVMe2G for life! until death (the end is nigh).
Yes, for sure, if I am not mistaken, doesn't the method you mention require a graphical console via VNC or maybe HTML5? But the cloudbsd.xyz method does not require a graphical console. Lots of times a graphical console isn't provided, and the cloudbsd.xyz method nevertheless hopefully still works.
Fair point, though a text console is fine.
In stasis until the shitposting stops/abates.
Than=compare;then=sequence:brought=bring;bought=buy:staffs=pile of sticks:informations/infos=no plural.
It wisnae me! A big boy done it and ran away. || NVMe2G for life! until death (the end is nigh).
The nodes are on QEMU 7.2.13 and Debian 12, so the kernel should be fairly up to date.
linveo.com | Shared Hosting | KVM VPS | Dedicated Servers
Do you mean none of the FreeBSD 14 versions work? I see a handful of servers running those templates at the moment.
linveo.com | Shared Hosting | KVM VPS | Dedicated Servers
I did at one point have issues with the 14.1 template but since have managed to boot it up. I had to manually configure the network, as DHCP wasn't working (from the cloud-init template?). Seems to be OK at the moment apart from locking myself out, setting up a firewall.;)
In stasis until the shitposting stops/abates.
Than=compare;then=sequence:brought=bring;bought=buy:staffs=pile of sticks:informations/infos=no plural.
It wisnae me! A big boy done it and ran away. || NVMe2G for life! until death (the end is nigh).
Seems to work today. Yesterday, only the FreeBSD 13.2 one worked (the others failed at the "Creating configuration." stage when rebuilding the server on VirtFusion).
I have put together another NetBSD 10.0 image. With some luck that should now even apply the network configuration and add SSH keys (if the configuration information is provided by VirtFusion).
@linveo could we give that image a try?
@cmeerw Great news! Congrats!
@linveo I'm eager to try @cmeerw's image! Thanks so much!
Sure thing! I have added this to the available templates. The network interface is vioif0. Let me know if that looks good to you.
linveo.com | Shared Hosting | KVM VPS | Dedicated Servers
I tried it out with my Intel instance and I still see the kernel crash/reboot cycle like before. Please check how it works on AMD.
Do you want switched over to an AMD Ryzen node?
linveo.com | Shared Hosting | KVM VPS | Dedicated Servers
~~~
linveo# ping6 -c 2 ipv6.google.com
PING6(56=40+8+8 bytes) 2605:xxxx:xxxx:xxxx::xxxx --> 2607:f8b0:402a:80b::200e
16 bytes from 2607:f8b0:402a:80b::200e, icmp_seq=0 hlim=118 time=1.553 ms
16 bytes from 2607:f8b0:402a:80b::200e, icmp_seq=1 hlim=118 time=0.902 ms
--- ipv6.l.google.com ping6 statistics ---
2 packets transmitted, 2 packets received, 0.0% packet loss
round-trip min/avg/max/std-dev = 0.902/1.228/1.553/0.460 ms
linveo# ping -c 2 google.com
PING google.com (192.178.49.174): 56 data bytes
64 bytes from 192.178.49.174: icmp_seq=0 ttl=118 time=1.257350 ms
64 bytes from 192.178.49.174: icmp_seq=1 ttl=118 time=1.317900 ms
----google.com PING Statistics----
2 packets transmitted, 2 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 1.257350/1.287625/1.317900/0.042815 ms
linveo#
~~~~
I went through all FreeBSD templates on my Intel instance and they all got installed and worked fine. 13.3 did take some extra time bringing the network interface up, but eventually it started as well. When it comes to FreeBSD, virtio network drivers have pretty significant degradation on performance as far as I have seen on multiple providers and e1000 works generally much better. Some panels provide access to user to change them, but VirtFusion seems to lack that option.
Accidental double post.
At least for me, everything seems to be working as expected.
It did set up the network correctly (both IPv4 and IPv6) and also added the SSH key to the root user.
Yes, that's expected as I used the unpatched NetBSD kernel for that image. I can probably build another image with a patched kernel, now that the basics seem to be working.