@Clouvider said: Of course Intel is in defence now, but it’s not like AMD is an ultimate solution in this particular segment yet.
I guess in your position it does not pose as much risk to your business, for us little guys this is truly horrible, sorry if I got your back up I am not trying to, I am just trying to get across in good humor how serious I am about wanting an alternative to build on.
The difference between standing up a hand full of new E3 based nodes and a few EPYC nodes is night and day in terms of cost and commitment.
The only host I can find that quickly outside of OVH and hetzner is reliablesite they offer:
12/24 Ryzen 3900X, 128GB DDR4, 150TB BW @ 10gbit, 4 x NVMe slots, IPMI/KVM at $258 in NewYork Metro.
That seems like a good fair price to me, that can probably be brought down 20 - 30% with a contract, not that I am particularly interested in a whole new location and I believe reliablesite had some drama recently but they are old and known so is probably a fair price guide.
I don't think there is anything intel comparable that wont require 4 microcode updates a year, no HT and slowly degrading performance.
So I don't mean to piss you off @clouvider I just assumed you would be able to come up with an option because I actually want to spend money with you first and thought you simply would not entertain it for no other reason than its a desktop CPU which frankly in isolation is nonsense now, but I accept that not in isolation there is more to it.
Everyone should disable Hyper-Threading. It causes the two threads in a core to compete with each other on L1 and L2 caches. Today the bottleneck is memory access, not number of instructions. Thus, Hyper-Threading makes programs run slower.
We accept Karma donations for the last flan. 🍮 affbrr
Everyone should disable Hyper-Threading. It causes the two threads in a core to compete with each other on L1 and L2 caches. Today the bottleneck is memory access, not number of instructions. Thus, Hyper-Threading makes programs run slower.
But it means less vCPU then? Then per node there will be less VPS can be made?
Everyone should disable Hyper-Threading. It causes the two threads in a core to compete with each other on L1 and L2 caches. Today the bottleneck is memory access, not number of instructions. Thus, Hyper-Threading makes programs run slower.
Interesting. The bottleneck is still RAM for DDR4?
@AnthonySmith I think the only real option for running Ryzen at the moment is to colo, especially if you want to be competitive. The guys doing it like @Francisco and @MikeA are buying boards and CPUs and making it work in their own racks.
I can't blame @Clouvider for not jumping in head first at the moment. To do Ryzen in the DC it requires manually assembling boxes with parts from a variety of manufacturers, with IPMI software that doesn't integrate with his automation, non-standard warranty processes if something breaks, etc. Just crazy amounts of time and overhead in bare metal where the goal is to touch the gear as little as possible to turn a profit.
The day Supermicro launches an AM4 microcloud I'm sure he'll be lined up though, something that's supported well and comes with all the management features needed to work with his setup.
I think your best bet if you want to make a Ryzen move to stay competitive would be to get a business account with a credit line from Amazon/Newegg/whoever and assemble your own boxes and rent a rack to stick them in.
@Harambe said: @AnthonySmith I think the only real option for running Ryzen at the moment is to colo, especially if you want to be competitive. The guys doing it like @Francisco and @MikeA are buying boards and CPUs and making it work in their own racks.
I can't blame @Clouvider for not jumping in head first at the moment. To do Ryzen in the DC it requires manually assembling boxes with parts from a variety of manufacturers, with IPMI software that doesn't integrate with his automation, non-standard warranty processes if something breaks, etc. Just crazy amounts of time and overhead in bare metal where the goal is to touch the gear as little as possible to turn a profit.
The day Supermicro launches an AM4 microcloud I'm sure he'll be lined up though, something that's supported well and comes with all the management features needed to work with his setup.
I think your best bet if you want to make a Ryzen move to stay competitive would be to get a business account with a credit line from Amazon/Newegg/whoever and assemble your own boxes and rent a rack to stick them in.
Yep fair enough, I just have a very bad intel taste in my mouth right now.
Yep fair enough, I just have a very bad intel taste in my mouth right now.
Oh I get it man. Providers have it tough at the moment, your gear still costs the same to keep up but every month it gets slower. I do not envy you over here in the in-house sysadmin world where you're not dealing with multiple tenants and can just boot with mitigations=off #yolo
ASRR boards have IPMI and cooling is nothing, I run a 16 core Ryzen on air and it's only 42*c under 70% load right now. ASRR has barebones (1U) servers for Ryzen. They are great.
@MikeA said:
ASRR boards have IPMI and cooling is nothing, I run a 16 core Ryzen on air and it's only 42*c under 70% load right now. ASRR has barebones (1U) servers for Ryzen. They are great.
Anyway, good day.
I’m genuinely interested in the positive way. I saw two barebones only from ASRR, one with and one with no hot swap. Neither supported NVMe on M.2 nor U.2, possibly one could put the daughter card to split NVMe out on this PCIe slot (but then sacrificing ability to upgrade to 10G as there’s only one slot..), assuming there is enough space to cool them. Would you be happy to share what chassis you run? I’m not judging anything. I’ve been attacked for buying Intel, my reasons is the platform of lack of it, I’m happy to re-evaluate if that’s not the case
@Cyder, this is lowend world. Ghetto setup works. Average users want Ryzen because of its ryzing fame. Gotta give kids toys that they want after all.
Now, I don't think you've been specifically attacked for using intel. The trigger for some is your statement of, "Desktop CPU is not meant to be used for servers." Which is technically true. But then again, this is lowend segment. Kids want their toys.
On a side note, doesn't Ryzen support ECC as long as mobo supports it?
♻ Amitz day is October 21.
♻ Join Nigh sect by adopting my avatar. Let us spread the joys of the end.
@MikeA said:
ASRR boards have IPMI and cooling is nothing, I run a 16 core Ryzen on air and it's only 42*c under 70% load right now. ASRR has barebones (1U) servers for Ryzen. They are great.
Anyway, good day.
I’m genuinely interested in the positive way. I saw two barebones only from ASRR, one with and one with no hot swap. Neither supported NVMe on M.2 nor U.2, possibly one could put the daughter card to split NVMe out on this PCIe slot (but then sacrificing ability to upgrade to 10G as there’s only one slot..), assuming there is enough space to cool them. Would you be happy to share what chassis you run? I’m not judging anything. I’ve been attacked for buying Intel, my reasons is the platform of lack of it, I’m happy to re-evaluate if that’s not the case
Well the stuff I run doesn't need 4 or more NVMe drives, and the drives I use are just 1 or 2 M.2 drive configs with adapters (or 1x onboard + 1x adapter) with the rest being SATA for boot and stuff. I don't hotswap, but you can technically hotswap without a sas card or whatever anyway?
But of course if you're wanting a huge NVMe array or hotswap Epyc is obviously the way to go, there's no doubt.
@AnthonySmith said:
Ryzen is simply a better choice and ticks every box in this market segment especially and out performs/delivers over any intel server CPU in the same ballpark, I hate to say it especially as everyone knows how hard I ride the @clouvider train but sadly that train only seems to stop at intel stations now and I may need to consider another route.
1 intel bug... ok, 2 hmm, 3, are you shitting me, 4,..... FOUR!!! ... oh fuck off intel.
Ryzen shows better performance, but AFAIK it is not generally more secure. This particular vulnerability affects Intel specifically, but speculative execution attack vector is present in AMD as well.
Also, are there platforms with Ryzen and IPMI?
Having spoken to a few people who research these sorts of bugs, they all seem to be of the opinion that while AMD isn't perfect, there are far, far less cut corners than with Intel, and so these sorts of bugs are much less likely.
Cooling becomes a problem if you're rimming the cores 24/7, but then you'll just throttle anyway.
You can disable turbo (at least in Linux) and that seems to keep it from throttling, but you aren't going to see past the stock clock.
A 2U would probably be better, but that's a lot of rack space if you're in premium markets.
For us and our CPU usage, 1U's are fine. The projections on our Ryzen nodes is they'll run at < 50% total CPU utilization with it being a good bit lower than that on quieter ones. It isn't going to help with the hosts that redline their nodes all the damn time, but that's life.
Yeah, the risk here I suppose, when offering them as dedicated, is that someone actually might and then might expect the provider to get them to such a state they are usable at full load. You’re too using these Asrock Rack barebones, if you don’t mind me asking?
Even under full load mine never got hot enough to be in throttling range, but I use 2U for all of my existing stuff. Anything in the future will be 1U though.
Surely some ASRock boards support Ryzen and multiple NVMe drives, as Hetzner offers Ryzens with NVMe? That's not to say they not building their own boxes though?
@Mr_Tom said:
Surely some ASRock boards support Ryzen and multiple NVMe drives, as Hetzner offers Ryzens with NVMe? That's not to say they not building their own boxes though?
See I’d ideally want to see a barebone when they already validated it will be able to cool what’s inside.
@Clouvider your Epyc servers are listed under the "unmetered" category. I presume it's not worth capping the bandwidth as it wouldn't reduce the cost that much?
@yoursunny said:
Everyone should disable Hyper-Threading. It causes the two threads in a core to compete with each other on L1 and L2 caches. Today the bottleneck is memory access, not number of instructions. Thus, Hyper-Threading makes programs run slower.
But it means less vCPU then? Then per node there will be less VPS can be made?
If you need more cores, choose a CPU with more cores or a dual-socket machine. Don't use the fake cores created by Hyper-Threading.
Also, disable Virtualization as it's another performance bottleneck.
@poisson said:
Interesting. The bottleneck is still RAM for DDR4?
I'm building a router with content caching capability.
The bottleneck was Ethernet adapter, so we get 100Gbps adapters.
The bottleneck moves to kernel, so we use user-space drivers.
Now the bottleneck is RAM even at 2933MHz speeds.
Another bottleneck is PCIe 3.
Each Xeon can only support three PCIe 3 x16 slots.
EPYC supposedly can have 128 lanes of PCIe 4, but I can't find a single-socket motherboard with eight PCIe 4 x16 slots.
We accept Karma donations for the last flan. 🍮 affbrr
@Mr_Tom said: @Clouvider your Epyc servers are listed under the "unmetered" category. I presume it's not worth capping the bandwidth as it wouldn't reduce the cost that much?
It wouldn’t be terribly substantial but yeah, it could save some smallish number of £ :-).
@debaser said:
This is true as well of course. These providers don’t just own a DC but own vast complexes. That being said, I wouldn’t know if their Ryzen dedis are 1U, 2U or maybe even 4U. Maybe @Hetzner_OL or @ninzo59 will share this information (probably not, but can’t blame me for trying).
We use our own customer build cases and racks. You can see our custom-built racks/cases on our 360 degree tour. These aren't Ryzens, but they're similar. The image in one of our social media posts for Ryzen is pretty much what it looks like on the inside. --Katie
We're Katie and Lea and we'll do our best to answer questions you have about Hetzner Online. We and not our employer are responsible for any horrible puns and dated cultural references.
@AnthonySmith said: The issue with EPYC is long contracts and poor single thread performance, there is no good config combination that works if you want to compete on price with the likes of Vultr, Do, hetzner, ovh clouds etc.
EPYC actually makes it easier to compete.
RamNode: High Performance Cloud VPS
NYC - LA - ATL - SEA - NL - DDoS Protection
That’s the general opinion and the one I share. I understand the problem here is the CPU frequency @AnthonySmith seems to need high clock and less cores rather than a lot more cores but a lower clock.
Well you know what it costs me and the commit to stand up an E3 and you know what it costs me to stand up and the commit to EPYC and its a low end EPYC at that, if I was a big cash rich business then I know it makes sense, but if I was I would not be having this conversation i would be getting a colo price
Ryzen stacks up against an E3 cost wise EPYC simply does not.
Anyway, I dont really want to push it any more, there are things from your side of the business that make it not appealing, I get it, if you dont want that target market any more sticking with intel is your best bet, that's not meant to be personal, that is just how I see it from my side of the business.
Comments
I guess in your position it does not pose as much risk to your business, for us little guys this is truly horrible, sorry if I got your back up I am not trying to, I am just trying to get across in good humor how serious I am about wanting an alternative to build on.
The difference between standing up a hand full of new E3 based nodes and a few EPYC nodes is night and day in terms of cost and commitment.
The only host I can find that quickly outside of OVH and hetzner is reliablesite they offer:
12/24 Ryzen 3900X, 128GB DDR4, 150TB BW @ 10gbit, 4 x NVMe slots, IPMI/KVM at $258 in NewYork Metro.
That seems like a good fair price to me, that can probably be brought down 20 - 30% with a contract, not that I am particularly interested in a whole new location and I believe reliablesite had some drama recently but they are old and known so is probably a fair price guide.
I don't think there is anything intel comparable that wont require 4 microcode updates a year, no HT and slowly degrading performance.
So I don't mean to piss you off @clouvider I just assumed you would be able to come up with an option because I actually want to spend money with you first and thought you simply would not entertain it for no other reason than its a desktop CPU which frankly in isolation is nonsense now, but I accept that not in isolation there is more to it.
https://inceptionhosting.com
Please do not use the PM system here for Inception Hosting support issues.
@AnthonySmith do all providers on Intel have HT disabled?
Recommend: SmallWeb|BuyVM|Linode|RamNode
Everyone should disable Hyper-Threading. It causes the two threads in a core to compete with each other on L1 and L2 caches. Today the bottleneck is memory access, not number of instructions. Thus, Hyper-Threading makes programs run slower.
We accept Karma donations for the last flan. 🍮 affbrr
But it means less vCPU then? Then per node there will be less VPS can be made?
I bench YABS 24/7/365 unless it's a leap year.
Interesting. The bottleneck is still RAM for DDR4?
Deals and Reviews: LowEndBoxes Review | Avoid dodgy providers with The LEBRE Whitelist | Free hosting (with conditions): Evolution-Host, NanoKVM, FreeMach, ServedEZ | Get expert copyediting and copywriting help at The Write Flow
1U Supermicro chassis (via the FT discord)
🦍🍌
Ryzen forever.
The same for your dicks as well.
Edit: No wait, forever-Ryzen dicks can be a huge issue. Nevermind my second line.
♻ Amitz day is October 21.
♻ Join Nigh sect by adopting my avatar. Let us spread the joys of the end.
@AnthonySmith I think the only real option for running Ryzen at the moment is to colo, especially if you want to be competitive. The guys doing it like @Francisco and @MikeA are buying boards and CPUs and making it work in their own racks.
I can't blame @Clouvider for not jumping in head first at the moment. To do Ryzen in the DC it requires manually assembling boxes with parts from a variety of manufacturers, with IPMI software that doesn't integrate with his automation, non-standard warranty processes if something breaks, etc. Just crazy amounts of time and overhead in bare metal where the goal is to touch the gear as little as possible to turn a profit.
The day Supermicro launches an AM4 microcloud I'm sure he'll be lined up though, something that's supported well and comes with all the management features needed to work with his setup.
I think your best bet if you want to make a Ryzen move to stay competitive would be to get a business account with a credit line from Amazon/Newegg/whoever and assemble your own boxes and rent a rack to stick them in.
🦍🍌
Yep fair enough, I just have a very bad intel taste in my mouth right now.
https://inceptionhosting.com
Please do not use the PM system here for Inception Hosting support issues.
Oh I get it man. Providers have it tough at the moment, your gear still costs the same to keep up but every month it gets slower. I do not envy you over here in the in-house sysadmin world where you're not dealing with multiple tenants and can just boot with
mitigations=off#yolo🦍🍌
ASRR boards have IPMI and cooling is nothing, I run a 16 core Ryzen on air and it's only 42*c under 70% load right now. ASRR has barebones (1U) servers for Ryzen. They are great.
Anyway, good day.
ExtraVM - KVM NVMe VPS in USA, EU, APAC -|- RackColo - Find Colo
Wow!
I’m genuinely interested in the positive way. I saw two barebones only from ASRR, one with and one with no hot swap. Neither supported NVMe on M.2 nor U.2, possibly one could put the daughter card to split NVMe out on this PCIe slot (but then sacrificing ability to upgrade to 10G as there’s only one slot..), assuming there is enough space to cool them. Would you be happy to share what chassis you run? I’m not judging anything. I’ve been attacked for buying Intel, my reasons is the platform of lack of it, I’m happy to re-evaluate if that’s not the case
Clouvider Limited - VPS in 11 datacenters - Intel Xeon/AMD Epyc with NVMe and 10G uplink! | Dedicated Servers
@Cyder, this is lowend world. Ghetto setup works. Average users want Ryzen because of its ryzing fame. Gotta give kids toys that they want after all.
Now, I don't think you've been specifically attacked for using intel. The trigger for some is your statement of, "Desktop CPU is not meant to be used for servers." Which is technically true. But then again, this is lowend segment. Kids want their toys.
On a side note, doesn't Ryzen support ECC as long as mobo supports it?
♻ Amitz day is October 21.
♻ Join Nigh sect by adopting my avatar. Let us spread the joys of the end.
Well the stuff I run doesn't need 4 or more NVMe drives, and the drives I use are just 1 or 2 M.2 drive configs with adapters (or 1x onboard + 1x adapter) with the rest being SATA for boot and stuff. I don't hotswap, but you can technically hotswap without a sas card or whatever anyway?
But of course if you're wanting a huge NVMe array or hotswap Epyc is obviously the way to go, there's no doubt.
ExtraVM - KVM NVMe VPS in USA, EU, APAC -|- RackColo - Find Colo
Wow!
The majority of the boards support ECC, even many of the cheap $60-80 boards. There's no point of doing that with a super cheap board though really..
ExtraVM - KVM NVMe VPS in USA, EU, APAC -|- RackColo - Find Colo
Wow!
@aMike, you are giving kids (Spring-break hosts) ideas...
♻ Amitz day is October 21.
♻ Join Nigh sect by adopting my avatar. Let us spread the joys of the end.
Look at the scrub running a 5.x kernel
"It's a hard life- to be a stick insect." - Karl Pilkington
Having spoken to a few people who research these sorts of bugs, they all seem to be of the opinion that while AMD isn't perfect, there are far, far less cut corners than with Intel, and so these sorts of bugs are much less likely.
Nothing is ever perfect.
♻ Amitz day is October 21.
♻ Join Nigh sect by adopting my avatar. Let us spread the joys of the end.
I'm using them in 1U's.
Cooling becomes a problem if you're rimming the cores 24/7, but then you'll just throttle anyway.
You can disable turbo (at least in Linux) and that seems to keep it from throttling, but you aren't going to see past the stock clock.
A 2U would probably be better, but that's a lot of rack space if you're in premium markets.
For us and our CPU usage, 1U's are fine. The projections on our Ryzen nodes is they'll run at < 50% total CPU utilization with it being a good bit lower than that on quieter ones. It isn't going to help with the hosts that redline their nodes all the damn time, but that's life.
Francisco
Yeah, the risk here I suppose, when offering them as dedicated, is that someone actually might and then might expect the provider to get them to such a state they are usable at full load. You’re too using these Asrock Rack barebones, if you don’t mind me asking?
D
Clouvider Limited - VPS in 11 datacenters - Intel Xeon/AMD Epyc with NVMe and 10G uplink! | Dedicated Servers
Even under full load mine never got hot enough to be in throttling range, but I use 2U for all of my existing stuff. Anything in the future will be 1U though.
ExtraVM - KVM NVMe VPS in USA, EU, APAC -|- RackColo - Find Colo
Wow!
What about this stuff then?
https://www.asrockrack.com/general/productdetail.asp?Model=1U4LW-X470
Hey teamacc. You're a dick. (c) Jon Biloh, 2020.
Interesting
https://inceptionhosting.com
Please do not use the PM system here for Inception Hosting support issues.
They're good, but if I recall that one is like $750.
ExtraVM - KVM NVMe VPS in USA, EU, APAC -|- RackColo - Find Colo
Wow!
Surely some ASRock boards support Ryzen and multiple NVMe drives, as Hetzner offers Ryzens with NVMe? That's not to say they not building their own boxes though?
That’s the one I was saying has severely limited storage options (no NVMe, or one maybe 4 on a daughter, and no option to upgrade to 10G then)
Clouvider Limited - VPS in 11 datacenters - Intel Xeon/AMD Epyc with NVMe and 10G uplink! | Dedicated Servers
See I’d ideally want to see a barebone when they already validated it will be able to cool what’s inside.
Clouvider Limited - VPS in 11 datacenters - Intel Xeon/AMD Epyc with NVMe and 10G uplink! | Dedicated Servers
It's got one of these in it according to dmidecode, which seems slightly different to the one linked above: https://www.asrockrack.com/general/productdetail.asp?Model=X470D4U#Specifications
Now this has 2x m.2 and could be at least sellable - but no barebone with this one as far as I can find
Clouvider Limited - VPS in 11 datacenters - Intel Xeon/AMD Epyc with NVMe and 10G uplink! | Dedicated Servers
That's the board we use. They have a 2T version which has dual 10gig copper ports.
For us, we use the D4U since our Infiniband 3X VPI's have a 2nd port on them that can operate in 10gig & 40gig ethernet simultaneously.
Francisco
A quick question.
Does "copper" part really make a difference?
♻ Amitz day is October 21.
♻ Join Nigh sect by adopting my avatar. Let us spread the joys of the end.
Yes, it uses a lot more power. It's something like 3W+ per side.
Francisco
That, long ago switches were less available with J but i guess this now changed. And fibre looks better on pictures ;-)
Not a deal breaker though, would just need an entire rack of those if that was to make sense.
Clouvider Limited - VPS in 11 datacenters - Intel Xeon/AMD Epyc with NVMe and 10G uplink! | Dedicated Servers
@Clouvider your Epyc servers are listed under the "unmetered" category. I presume it's not worth capping the bandwidth as it wouldn't reduce the cost that much?
If you need more cores, choose a CPU with more cores or a dual-socket machine. Don't use the fake cores created by Hyper-Threading.
Also, disable Virtualization as it's another performance bottleneck.
I'm building a router with content caching capability.
The bottleneck was Ethernet adapter, so we get 100Gbps adapters.
The bottleneck moves to kernel, so we use user-space drivers.
Now the bottleneck is RAM even at 2933MHz speeds.
Another bottleneck is PCIe 3.
Each Xeon can only support three PCIe 3 x16 slots.
EPYC supposedly can have 128 lanes of PCIe 4, but I can't find a single-socket motherboard with eight PCIe 4 x16 slots.
We accept Karma donations for the last flan. 🍮 affbrr
That is not a router.
"It's a hard life- to be a stick insect." - Karl Pilkington
It wouldn’t be terribly substantial but yeah, it could save some smallish number of £ :-).
Clouvider Limited - VPS in 11 datacenters - Intel Xeon/AMD Epyc with NVMe and 10G uplink! | Dedicated Servers
It's normally called a "forwarder". "router" is an alias.
See here: https://github.com/usnistgov/ndn-dpdk
We accept Karma donations for the last flan. 🍮 affbrr
sounds like a cobalt raq 4
https://inceptionhosting.com
Please do not use the PM system here for Inception Hosting support issues.
Ah, damn, Cobalt RAQ. Sure brings back memories.
♻ Amitz day is October 21.
♻ Join Nigh sect by adopting my avatar. Let us spread the joys of the end.
We all had ideals then.
"It's a hard life- to be a stick insect." - Karl Pilkington
Butts.
Given your statements and ideology, I'd prefer not to use anything you endorse. All the same - butts.
"It's a hard life- to be a stick insect." - Karl Pilkington
I've got one of these
We use our own customer build cases and racks. You can see our custom-built racks/cases on our 360 degree tour. These aren't Ryzens, but they're similar. The image in one of our social media posts for Ryzen is pretty much what it looks like on the inside. --Katie
We're Katie and Lea and we'll do our best to answer questions you have about Hetzner Online. We and not our employer are responsible for any horrible puns and dated cultural references.
EPYC actually makes it easier to compete.
RamNode: High Performance Cloud VPS
NYC - LA - ATL - SEA - NL - DDoS Protection
That’s the general opinion and the one I share. I understand the problem here is the CPU frequency @AnthonySmith seems to need high clock and less cores rather than a lot more cores but a lower clock.
Clouvider Limited - VPS in 11 datacenters - Intel Xeon/AMD Epyc with NVMe and 10G uplink! | Dedicated Servers
Well you know what it costs me and the commit to stand up an E3 and you know what it costs me to stand up and the commit to EPYC and its a low end EPYC at that, if I was a big cash rich business then I know it makes sense, but if I was I would not be having this conversation i would be getting a colo price
Ryzen stacks up against an E3 cost wise EPYC simply does not.
Anyway, I dont really want to push it any more, there are things from your side of the business that make it not appealing, I get it, if you dont want that target market any more sticking with intel is your best bet, that's not meant to be personal, that is just how I see it from my side of the business.
https://inceptionhosting.com
Please do not use the PM system here for Inception Hosting support issues.
I said more than one time, Intel is the new Apple, people dont listen to me.