@reisenpai said: And we got scheduled reboot/shutdown on 039 again. This time, we can't even boot it manually.
Can you issue a boot up for everyone on 039?
If I am correct users on Node39, which was a beta test node, are being migrated to a different node.
Nothing scheduled like that and the node is up. I get a lot of reports by a lot of people that that node keeps going down and it's just been up the whole time, I have no idea why people are saying all these things honestly. I guess because it was called beta test node or reported as having issues at some point in time everyone immediately assumes that as the default explanation for everything related to their VPS going offline or even SolusVM running into issues?
Anyway, my guess is that the "we can't even boot it manually" part is related to the fact that SolusVM is down.
Edit:
When I says offline or shutdown or reboot, I'm talking about the VM, not the node.
Whenever I reboot the VM within the OS, there is a fair chance for it to go offline until manually booted in the panel.
The reason I said automated or scheduled shutdown is the fact this always happens on 08:00 PST.
I know the node itself is running alright, it's the VMs being offline.
People mistakenly reported it as Node being down because they didn't realize they can just boot the VM up.
@reisenpai said: And we got scheduled reboot/shutdown on 039 again. This time, we can't even boot it manually.
Can you issue a boot up for everyone on 039?
If I am correct users on Node39, which was a beta test node, are being migrated to a different node.
Nothing scheduled like that and the node is up. I get a lot of reports by a lot of people that that node keeps going down and it's just been up the whole time, I have no idea why people are saying all these things honestly. I guess because it was called beta test node or reported as having issues at some point in time everyone immediately assumes that as the default explanation for everything related to their VPS going offline or even SolusVM running into issues?
Anyway, my guess is that the "we can't even boot it manually" part is related to the fact that SolusVM is down.
Whenever I reboot the server within the OS, there is a fair chance for it to be offline in the panel until manually booted.
The reason I said automated or scheduled shutdown is the fact this always happens on 08:00 PST.
I know the node itself is running alright, it's the VMs being offline.
People mistakenly reported it as Node being down because they didn't realize they can just boot the VM up.
And I highly doubt it's a problem within the VM, multiple people with different templates and ISO are having the same thing.
Edit:
And just to be clear, we have two problems:
1. VMs goes offline from time to time, but it's always at 08:00 PST. All we need to do is manually booting it in the panel.
2. Whenever we reboot the VMs within OS, there is a chance for it to go offline. And we boot it in the panel.
@VirMach said: The billing area should be working fine, I think it's just not connecting to SolusVM since that's down, and it may cause it to throw errors or not load your service page.
Yeah, not a huge deal. I rebooted an idler today and forgot it doesn't have automatic encryption unlock yet. I'll take care of it when they give up.
@reisenpai said: Ms goes offline from time to time, but it's always at 08:00 PST. All we need to do is manually booting it in the panel.
Who is we? I need to know exactly what list of VMs are facing this exact issue to be able to better diagnose the problem and it needs to actually be confirmed as being widespread.
@reisenpai said: Ms goes offline from time to time, but it's always at 08:00 PST. All we need to do is manually booting it in the panel.
Who is we? I need to know exactly what list of VMs are facing this exact issue to be able to better diagnose the problem and it needs to actually be confirmed as being widespread.
Personally, I am not seeing this. My 2.5GB VPS on Node 39 has not gone down and needed to be rebooted from the panel at all since activation. My 1.5GB VPS on Node 39 has not gone down and stayed down before today at 5:56AM PDT. I have not been able to check why this went down because of the SolusVm issues, but I do not expect it to be related to the issue others are talking about.
@reisenpai said: Ms goes offline from time to time, but it's always at 08:00 PST. All we need to do is manually booting it in the panel.
Who is we? I need to know exactly what list of VMs are facing this exact issue to be able to better diagnose the problem and it needs to actually be confirmed as being widespread.
Besides the guy replied to this thread, it's all on OGF. I don't want go back there so you gotta check it yourself.
Edit:
And at p70, quote from me: "Alma seems to be alright so far, you might need manually boot it in the panel whenever it goes off."
I think this is the first time anybody talked about unexpected offline.
@reisenpai said: Ms goes offline from time to time, but it's always at 08:00 PST. All we need to do is manually booting it in the panel.
Who is we? I need to know exactly what list of VMs are facing this exact issue to be able to better diagnose the problem and it needs to actually be confirmed as being widespread.
Besides the guy replied to this thread, it's all on OGF. I don't want go back there so you gotta check it yourself.
@VirMach said: I wouldn't be able to help anyone here until SolusVM comes back online but I can assure you that the node is online and also not running into the same disk issues it did in the past.
Is there any chance for Buffalo to move to Japan now?
@VirMach said: I wouldn't be able to help anyone here until SolusVM comes back online but I can assure you that the node is online and also not running into the same disk issues it did in the past.
Is there any chance for Buffalo to move to Japan now?
This was going to be automated/activated and offered but I'm considering against it now with all the events that have transpired. I may restrict further migrations/creations on Tokyo after we get these done. We're just getting way too many of these tickets and also we won't have room to do every single person that wants to migrate and it will just end up upsetting some or make them think they've been wronged because the rollout absolutely will not be smooth.
Then I'll have to deal with who gets to go when, etc. I think I will at least try to offer it for those people who completed the survey about a year ago. Outside of that, I don't know.
@VirMach said: Then I'll have to deal with who gets to go when, etc. I think I will at least try to offer it for those people who completed the survey about a year ago. Outside of that, I don't know.
< joke > I think you should let higher end migration specials migrate. You don't charge transfer fees, right? Papa needs to pimp his ride.< /joke >
Is there any chance I could pick up a 4GB Chicago pre-order? I'm definitely in no rush to get it activated — idling is the best form of usage — but Chicago's rare enough that I want to jump on the opportunity to grab one .
@drunekndog said:
Is there any chance I could pick up a 4GB Chicago pre-order? I'm definitely in no rush to get it activated — idling is the best form of usage — but Chicago's rare enough that I want to jump on the opportunity to grab one .
Chicago's rare? That's the first time I've heard that. We're going to have extra space left over in Chicago, as in excessive power and bandwidth so it won't be a problem. I'm just not doing any preorders though right now, sorry.
thank you for this detailed explanation, and I will always try to understand this and it is clear in the sense that it is still in beta and the development stage for ryzen. I am patient to wait. good luck for @VirMach
If you're speaking of rDNS on Ryzen, you can assume long waits and potentially for the request to not be filled for now. None of the systems are built for that yet. With CC, we coded it out using their API. We do not consider rDNS critical and those tickets will most likely not be answered until we clear out the entire ticket queue.
Wow, client area is really suffering under the ddos. I had to do 5+ captchas plus a password reset in order to log in, and then I can't reach the screens for individual servers. Will try again later I guess. Sympathy.
@edenten said: So now how do I know the results of the Russian roulette.
No migration happened. What did happened was...
I prematurely posted a tag that was for a sale that has not happened yet in the OGF VirMach pre-sale thread.
This was not the right thing to do, was a poor attempt at humor on my part, and I regret it.
I commented at the time that I was playing Russian roulette with VirMach. Then things spiraled out of control.
It ended up more like this Clint Eastwood scene from The good, the bad, and the ugly
Then VirMach shot my Buffalo VPS dead.
So I suggest thinking very carefully before trying to outwit VirMach.
Because you may not be able to live with yourself after being responsible for the death of an innocent Buffalo VPS.
@edenten said: So now how do I know the results of the Russian roulette.
No migration happened. What did happened was...
I prematurely posted a tag that was for a sale that has not happened yet in the OGF VirMach pre-sale thread.
This was not the right thing to do, was a poor attempt at humor on my part, and I regret it.
I commented at the time that I was playing Russian roulette with VirMach. Then things spiraled out of control.
It ended up more like this Clint Eastwood scene from The good, the bad, and the ugly
Then VirMach shot my Buffalo VPS dead.
So I suggest thinking very carefully before trying to outwit VirMach.
Because you may not be able to live with yourself after being responsible for the death of an innocent Buffalo VPS.
Too many words, didn't read or understand any of it. All I heard was "post the tag" and "Buffalo dead" which I like, I'm posting it now.
I'm looking into that node right now to see what's up. Some guy's complaining about it on LET but not providing any information and just getting offended. If you have any information for me let me know.
I'm looking into that node right now to see what's up. Some guy's complaining about it on LET but not providing any information and just getting offended. If you have any information for me let me know.
my personal feeling is that the VM CPU/IO is currently fast, and only the network is "severely limited", and the latency is very high. some packet restriction? (im no network geek)
yum -y update is downloading at 30k/s
since network is slow i have not run any YABS yet.
tried to reinstall Debian 11 using client panel, but it doesnt work.
tried to access SolusVM from client panel, but it asks for login details (which I dont have)
@cybertech said: my personal feeling is that the VM CPU/IO is currently fast, and only the network is "severely limited", and the latency is very high. some packet restriction? (im no network geek)
CPU is important for networking and pretty much everything. I've had cases where customers complain of networking being slow and it turns out it's because their VM is extremely limited in CPU (one Intel core in the past) and they're doing to do some massive network operations that basically cause it to run out of processing power. Change it to two cores, and it's fixed.
In this case the entire node is potentially having CPU spikes in a way where it's limiting the network performance. Certain types of CPU usage can cause this to happen and I suspect that.
To be clear, Tokyo is still waiting for 40Gbps uplink but other servers are not having as much issue as this node right now, they're getting normal/much better speeds. I'm working on this now and already making some progress, there's a few more of these kernel panic VMs that need to be powered down.
@cybertech said: i feel that apart from slow network, executing install commands are fast. my steal is 0.2st.
Yeah it's weird the CPU usage isn't causing it to "overload" in the sense that there would be a lot of steal, it's just a few of these going into kernel panic and high system interrupts as a result and that's what's causing networking to be terrible. I'm testing it now after the powerdowns to see if there's an improvement.
@cybertech said: i feel that apart from slow network, executing install commands are fast. my steal is 0.2st.
Yeah it's weird the CPU usage isn't causing it to "overload" in the sense that there would be a lot of steal, it's just a few of these going into kernel panic and high system interrupts as a result and that's what's causing networking to be terrible. I'm testing it now after the powerdowns to see if there's an improvement.
yes "interrupt" sounds like it.
i'm just gonna leave it here first while i do actual work, feel free to tag or PM me if you need any other tests, or if you wanna double my bandwidth that i will never use.
I'm on SJCZ003 and it's just blindingly fast. The NVMe disk is so much faster than the desktop SSD's in other VPS, that it makes it feel like everything is already cached in ram. I'm feeling remorse over not having chosen a bigger VM. I vaguely remember having seen an upgrade button in the web console somewhere, but I couldn't find it just now, so must have been imagining it. Oh well. I could even imagine getting another one after I install my app and see how it goes. Any idea how long this promo will be running?
I've had a few other Virmach VM's which were perfectly fine, but this one is by far the best.
@willie said:
I'm on SJCZ003 and it's just blindingly fast. The NVMe disk is so much faster than the desktop SSD's in other VPS, that it makes it feel like everything is already cached in ram. I'm feeling remorse over not having chosen a bigger VM. I vaguely remember having seen an upgrade button in the web console somewhere, but I couldn't find it just now, so must have been imagining it. Oh well. I could even imagine getting another one after I install my app and see how it goes. Any idea how long this promo will be running?
I've had a few other Virmach VM's which were perfectly fine, but this one is by far the best.
Don't worry much about it being too good, it'll get worse from here once you're not the only one on there! One day I might get on here and see a bunch of messages getting flamed about how SJCZ003 is awful.
Actually yours YABS was concerningly high as well when I skimmed over it again today (I was trying to grab one real quick for a frame of reference.) I'd need to look into it more to see what's even happening because I didn't even get something that high during testing. I guess I need to take a look and see what the specifications are on that particular node anyway. It's possible it's some weird build I did one or two of and it ended up there.
I just took a look though and you're not even the only one on here anymore, there's a dozen 30+ others. At a glance though I don't see anything spectacular in terms of model numbers for these. I'll have another look later.
Edit -- Looks like it was someone else's YABS but that one was insanely good as well.
You're on an SK Hynix SHGP31-2000GM. I remember in testing these were my favorite, even above Gen4 NVMe because of the terrific real-world performance. I also did a lot of research on them and vaguely remember these having an insanely low latency, second to Intel Optane.
This also explains the insanely high number on YABS now. I also vaguely remember some benchmarks showing abnormally high performance because of this low latency, but extremely vaguely. I'm sure I wrote a long post about it somewhere maybe half a year ago.
@willie said: Any idea how long this promo will be running?
We have a quantity cap somewhere in my head. I usually end them when I'm overwhelmed with work for the next 3 months.
Comments
Edit:
When I says offline or shutdown or reboot, I'm talking about the VM, not the node.
Whenever I reboot the VM within the OS, there is a fair chance for it to go offline until manually booted in the panel.
The reason I said automated or scheduled shutdown is the fact this always happens on 08:00 PST.
I know the node itself is running alright, it's the VMs being offline.
People mistakenly reported it as Node being down because they didn't realize they can just boot the VM up.
Edit: grammar
And I highly doubt it's a problem within the VM, multiple people with different templates and ISO are having the same thing.
Edit:
And just to be clear, we have two problems:
1. VMs goes offline from time to time, but it's always at 08:00 PST. All we need to do is manually booting it in the panel.
2. Whenever we reboot the VMs within OS, there is a chance for it to go offline. And we boot it in the panel.
Yeah, not a huge deal. I rebooted an idler today and forgot it doesn't have automatic encryption unlock yet. I'll take care of it when they give up.
Who is we? I need to know exactly what list of VMs are facing this exact issue to be able to better diagnose the problem and it needs to actually be confirmed as being widespread.
Sounds like a scheduled job killing your server
“Technology is best when it brings people together.” – Matt Mullenweg
Personally, I am not seeing this. My 2.5GB VPS on Node 39 has not gone down and needed to be rebooted from the panel at all since activation. My 1.5GB VPS on Node 39 has not gone down and stayed down before today at 5:56AM PDT. I have not been able to check why this went down because of the SolusVm issues, but I do not expect it to be related to the issue others are talking about.
Besides the guy replied to this thread, it's all on OGF. I don't want go back there so you gotta check it yourself.
At page 76 people are starting to talk about it.
https://lowendtalk.com/discussion/177704/virmach-ryzen-nvme-8-88-yr-384mb-21-85-yr-2-5gb-instant-japan-pre-order-more/p76
Edit:
And at p70, quote from me: "Alma seems to be alright so far, you might need manually boot it in the panel whenever it goes off."
I think this is the first time anybody talked about unexpected offline.
btw mine would be id=652752
IPv6 rDNS should be delegated to a name server specified by the customer.
Only allowing updates may not enough.
See what you can do with IPv6 rDNS delegation:
https://devpost.com/software/traceart-8gsn5o
https://yoursunny.com/t/2016/HackArizona/
We accept Karma donations for the last flan. 🍮 affbrr
Is there any chance for Buffalo to move to Japan now?
This was going to be automated/activated and offered but I'm considering against it now with all the events that have transpired. I may restrict further migrations/creations on Tokyo after we get these done. We're just getting way too many of these tickets and also we won't have room to do every single person that wants to migrate and it will just end up upsetting some or make them think they've been wronged because the rollout absolutely will not be smooth.
Then I'll have to deal with who gets to go when, etc. I think I will at least try to offer it for those people who completed the survey about a year ago. Outside of that, I don't know.
< joke > I think you should let higher end migration specials migrate. You don't charge transfer fees, right? Papa needs to pimp his ride.< /joke >
Is there any chance I could pick up a 4GB Chicago pre-order? I'm definitely in no rush to get it activated — idling is the best form of usage — but Chicago's rare enough that I want to jump on the opportunity to grab one
.
SolusVM back up.
Chicago's rare? That's the first time I've heard that. We're going to have extra space left over in Chicago, as in excessive power and bandwidth so it won't be a problem. I'm just not doing any preorders though right now, sorry.
Someone migrated because of your eggs. Will not continue?
thank you for this detailed explanation, and I will always try to understand this and it is clear in the sense that it is still in beta and the development stage for ryzen. I am patient to wait. good luck for @VirMach
@VirMach
Hi, My order number is 6488962051
Thank you.
I don't even think they migrated. I'm pretty sure @FrankZ just signed up for his service being fully broken and stuck in limbo.
Yep, stuck in migration since yesterday. No worries. It's what I get for playing Russian roulette with VirMach.
Hello,
My Order Number is: 8247662033
Thank you.
I tried to play Russian roulette as a joke too, but it seems that many people thought of it as a real lottery.
Sure happy to see you here!
And my order ID is: #560703 and my order number is 9906668560
@FrankZ now I really bus spooked you, didn’t I?
Here’s one frm my city
The closest and cleanest shot I could find…
blog archives
So now how do I know the results of the Russian roulette. It seems that both people who got migrated only posted the first hashtags.
@vyas - Other then joking around here and there, it really was a VirMach BF thread thing.
Complements on the very nice bus post.
Indeed but since a point was raised (no bus shaming) I thought would clear the air.
Some day, they may return.
Till then 🥂
meanwhile... still mulling AMS or Tokyo.
blog archives
Wow, client area is really suffering under the ddos. I had to do 5+ captchas plus a password reset in order to log in, and then I can't reach the screens for individual servers. Will try again later I guess. Sympathy.
oh man im together with mjj on TYC040?
network is real slow now,
is going at 30k/s
I bench YABS 24/7/365 unless it's a leap year.
No migration happened. What did happened was...
I prematurely posted a tag that was for a sale that has not happened yet in the OGF VirMach pre-sale thread.
This was not the right thing to do, was a poor attempt at humor on my part, and I regret it.
I commented at the time that I was playing Russian roulette with VirMach. Then things spiraled out of control.
It ended up more like this Clint Eastwood scene from The good, the bad, and the ugly
Then VirMach shot my Buffalo VPS dead.
So I suggest thinking very carefully before trying to outwit VirMach.
Because you may not be able to live with yourself after being responsible for the death of an innocent Buffalo VPS.
Too many words, didn't read or understand any of it. All I heard was "post the tag" and "Buffalo dead" which I like, I'm posting it now.
I'm looking into that node right now to see what's up. Some guy's complaining about it on LET but not providing any information and just getting offended. If you have any information for me let me know.
The node might just be having CPU issues due to a bunch of kernel panics, let me see if this'll fix it.
@VirMach
Invoice #1413535
Thanks.
my personal feeling is that the VM CPU/IO is currently fast, and only the network is "severely limited", and the latency is very high. some packet restriction? (im no network geek)
yum -y update is downloading at 30k/s
since network is slow i have not run any YABS yet.
tried to reinstall Debian 11 using client panel, but it doesnt work.
tried to access SolusVM from client panel, but it asks for login details (which I dont have)
I bench YABS 24/7/365 unless it's a leap year.
well i might be wrong, YABS is stuck on FIO, so disk is also an issue for me
which is odd because IOWait is 0.0
I bench YABS 24/7/365 unless it's a leap year.
CPU is important for networking and pretty much everything. I've had cases where customers complain of networking being slow and it turns out it's because their VM is extremely limited in CPU (one Intel core in the past) and they're doing to do some massive network operations that basically cause it to run out of processing power. Change it to two cores, and it's fixed.
In this case the entire node is potentially having CPU spikes in a way where it's limiting the network performance. Certain types of CPU usage can cause this to happen and I suspect that.
To be clear, Tokyo is still waiting for 40Gbps uplink but other servers are not having as much issue as this node right now, they're getting normal/much better speeds. I'm working on this now and already making some progress, there's a few more of these kernel panic VMs that need to be powered down.
What OS and plan?
you could be right on the CPU, it probably had to wait for processing power to perform as below:
fio Disk Speed Tests (Mixed R/W 50/50): --------------------------------- Block Size | 4k (IOPS) | 64k (IOPS) ------ | --- ---- | ---- ---- Read | 405.26 MB/s (101.3k) | 960.79 MB/s (15.0k) Write | 406.33 MB/s (101.5k) | 965.85 MB/s (15.0k) Total | 811.59 MB/s (202.8k) | 1.92 GB/s (30.1k) | | Block Size | 512k (IOPS) | 1m (IOPS) ------ | --- ---- | ---- ---- Read | 948.41 MB/s (1.8k) | 893.29 MB/s (872) Write | 998.80 MB/s (1.9k) | 952.78 MB/s (930) Total | 1.94 GB/s (3.8k) | 1.84 GB/s (1.8k)i feel that apart from slow network, executing install commands are fast. my steal is 0.2st.
Almaxlinux 8.3
2560mb pre-order plan
looks like like network is on
I bench YABS 24/7/365 unless it's a leap year.
Yeah it's weird the CPU usage isn't causing it to "overload" in the sense that there would be a lot of steal, it's just a few of these going into kernel panic and high system interrupts as a result and that's what's causing networking to be terrible. I'm testing it now after the powerdowns to see if there's an improvement.
yes "interrupt" sounds like it.
i'm just gonna leave it here first while i do actual work, feel free to tag or PM me if you need any other tests, or if you wanna double my bandwidth that i will never use.
I bench YABS 24/7/365 unless it's a leap year.
I just posting here needs to know how many weeks does it take to reinstall windows os. Las time I check (2018), it tooks only 2 days.
I'm on SJCZ003 and it's just blindingly fast. The NVMe disk is so much faster than the desktop SSD's in other VPS, that it makes it feel like everything is already cached in ram. I'm feeling remorse over not having chosen a bigger VM. I vaguely remember having seen an upgrade button in the web console somewhere, but I couldn't find it just now, so must have been imagining it. Oh well. I could even imagine getting another one after I install my app and see how it goes. Any idea how long this promo will be running?
I've had a few other Virmach VM's which were perfectly fine, but this one is by far the best.
Don't worry much about it being too good, it'll get worse from here once you're not the only one on there! One day I might get on here and see a bunch of messages getting flamed about how SJCZ003 is awful.
Actually yours YABS was concerningly high as well when I skimmed over it again today (I was trying to grab one real quick for a frame of reference.) I'd need to look into it more to see what's even happening because I didn't even get something that high during testing. I guess I need to take a look and see what the specifications are on that particular node anyway. It's possible it's some weird build I did one or two of and it ended up there.
I just took a look though and you're not even the only one on here anymore, there's a dozen 30+ others. At a glance though I don't see anything spectacular in terms of model numbers for these. I'll have another look later.
Edit -- Looks like it was someone else's YABS but that one was insanely good as well.
You're on an SK Hynix SHGP31-2000GM. I remember in testing these were my favorite, even above Gen4 NVMe because of the terrific real-world performance. I also did a lot of research on them and vaguely remember these having an insanely low latency, second to Intel Optane.
This also explains the insanely high number on YABS now. I also vaguely remember some benchmarks showing abnormally high performance because of this low latency, but extremely vaguely. I'm sure I wrote a long post about it somewhere maybe half a year ago.
We have a quantity cap somewhere in my head. I usually end them when I'm overwhelmed with work for the next 3 months.
Wow, new site for Virmach public ticket queue!
Could you double something?
Ontario Dildo Inspector
Double green queue
Hello, please help me with it
Invoice 1413470
order #560309
Thank you!
Hi, how can I help you?
Warm regards.
dnscry.pt - Public DNSCrypt resolvers hosted by LowEnd providers • Need a free NAT LXC? -> https://microlxc.net/
Nice, glad to see you moved over here. Guess I might as well pick up another one, surely I'll put it to use for something. Order #7177636556
Awesome. Please activate Invoice #1398758.