Cheapest storage ($0.5 - ) $2.5 / TB / Month

edited December 2024 in Help
  • Hetzner "storage box" 1TB 5TB 10TB 20TB
  • Backblaze personal backup (unlimited. Price per machine)

Cheapest & Future (Decentralized)

  • Filecoin & IPFS (SIA, Storj,...)
    • some latency, not designed for small files / databases
    • might comes with 3 country locations by default / always

More:

  • OVH kimsufi 4 x 2TB (not very convenient & only when on sale) - but you can set you can set ZFS deduplication and compression
  • Rare Hetzner auctions with clunky things like 15 x 6TB

Small companies:

  • Servarica ZFS

New:

  • bulkvm (since june) (testing, but unfortunately didnt answer me anymore and why not just a little more ram?)

Gone:

  • Google drive unlimited

@servarica_hani can you give me any coupon or custom offer <= $2 / TB of secure storage? (under NDA if you will) - When you sell ZFS double paritiy, what host systems are that? Can i get triple parity too? And will the zfs pool span a big server, so that customers profit from deduplication?
( https://lowendbox.com/blog/got-burned-on-crypto-invest-in-a-servarica-cheap-vps-your-storage-grows-every-day/ by @raindog308 )

I watch your dynamic offers including some RAM and SSD but then it gets much more expensive.
Currently i only use ~10 TB of HDD storage + backup and only ~5 mbit average.


  1. Harddrives are mostly only for backups and log data, while for sorting & refining data, of course one needs SSD & RAM sometimes (and some always, for caching and ZFS). - Yet I still work only on 64GB DDR4 (as avaiable for $5 from OVH at times. No SSD. And 100ms latency to my storage...)
    QUESTION OF THE DAY:
    WHY IS THERE NO COMPETITIVE SSD HOST?
    The SSD hosting prices are crazy by now, when actually buying SSD servers starts around $20/TB. (a 2020 Server with 24 x 8 TB NVMe, might costs $4000 and power consumption of SSDs is minimal. So despite the performance the prices are similar to HDD now. SSD can be even cheaper considering wear down, electricity and no hardware failures. (The 95th quantile of load or write speed could be limited therefore for low budget plans (almost nobody will constantly use the crazy high speed) - Yet as of now, my non-profit projects still can't afford hosted SSDs (like commerce can usually) (but my project are held back to HDD speeds, as if it was 20 or 30 years ago.
    So of course i wonder to buy an SSD server, but why would i really have to? isn't that strange? Why does Nobodies offers such machine for an acceptable rent?
  • Contabo gives 2400GB storage VPS "SSD" starting at 12,6€ ("-10% sales"), but they have 1 star on google and reviews and terms that these can be lost anytime with no backup (no raid even?) .

Comments

  • For those with parents or grandparents + fast internet:
    100€ Mini-PC with Intel N100 or some old laptop (practically free)
    50€ 4-Bay HDD enclosure on eBay or 100€ new
    100€ each refurbished 12TB drive on eBay from a reputable seller
    Buy everything using PayPal credit if you can't afford it in one go

    = 36TB + 12TB parity for 25€/month for 2 years, afterwards free forever

    • privacy + fast access, should you need large amounts of data quickly
      (Your parents could copy several TB of data onto an external drive and mail it to you)

    None of the cheap storage providers provide fast download/upload speeds, really sucks when transferring large amounts of data within a short period of time is necessary

  • @archivist said: the prices are similar to HDD now

    @archivist said: SSD can be even cheaper considering wear down, electricity and no hardware failures

    Seems like a parallel dimension.

  • dosaidosai OG தோசை
    edited December 2024

    Hosthatch storage deal is $1.67 per TB.

  • edited December 2024

    @archivist said: When you sell ZFS double paritiy, what host systems are that?

    Those are SANs, however I do not recall further details.

    Also Servarica already has offers with the cost of terabyte below $2.50 per month. Not by much, but it's still less than $2.5/m.

    Let me know if you need links.

    VDS & StorageBudget AMD EPYC VDS (ref) from $4/m / Big HDD VPS (ref) from $2/TB/mo / EPYC/Gold/Platinum dedis (ref) w/ GPU support

  • I went from Googles unlimited cloud (or may it was their 100TB offer) to uloz.to before switching to a selfhosted NAS. Uloz has been around for a while, I think they started as a filesharing site and switched to private cloud somewhere later. Their service used to be really good (may still is, cant tell).

    5 TB Storage $6.5/mo
    30 TB for data transfer/month

    25 TB Storage $16/mo
    150 TB for data transfer/month

    50 TB Storage $32/mo
    300 TB for data transfer/month

    100 TB Storage $64/mo
    600 TB for data transfer/month

  • @cayc said: uloz.to

    Do their paid plans allow mounting the volumes SMB/CIFS or otherwise?

  • @hornet said:

    @cayc said: uloz.to

    Do their paid plans allow mounting the volumes SMB/CIFS or otherwise?

    Rclone seems to support it
    https://rclone.org/ulozto/

  • Is mega still good now? They're offering 16tb at 25EUR/mon.

  • It's like I don't exist, really.

    Z Plus's default rates are $2.5/m/tb with our cheapest plan starting at $5 / month for 2TB volumes attached to VPS's. USA hosted, all equipment owned, arrays are 3x7 raidz-2 so there's enough IOPS to go around.

    Our stuff's built on enterprise hardware, and I come from a background of running long term archives. I'm a strong believer you get what you pay for, so no my service isn't ideal - but I've done the math to ensure it's sustainable.

    There will be a LES exclusive deal going up this Wednesday for 3.5TB of storage volume for $7/month from Z+

    I also offer slot hosting, just mail in a 20tb drive and I'll host it.

  • @fan said: Is mega still good now?

    I don't know how they are in other places, but it's hella slow here. I uploaded the same file to Hostbrr storagebox and Mega, and I was able to get files to the storagebox about 3 times quicker. the file was around 5 gigs or so. also might be because i am using the free storage.

  • @ZizzyDizzyMC said:
    It's like I don't exist, really.

    Z Plus's default rates are $2.5/m/tb with our cheapest plan starting at $5 / month for 2TB volumes attached to VPS's. USA hosted, all equipment owned, arrays are 3x7 raidz-2 so there's enough IOPS to go around.

    Our stuff's built on enterprise hardware, and I come from a background of running long term archives. I'm a strong believer you get what you pay for, so no my service isn't ideal - but I've done the math to ensure it's sustainable.

    There will be a LES exclusive deal going up this Wednesday for 3.5TB of storage volume for $7/month from Z+

    I also offer slot hosting, just mail in a 20tb drive and I'll host it.

    You need better marketing. You spent time writing that post but failed to include a single link to your company website (which does not contain the word "plus" in the domain name at all making it especially difficult to locate). Also on your website your "About Us" page is well buried and could inspire more confidence in your company. Best wishes.

  • @Newyawker said:

    @ZizzyDizzyMC said:
    It's like I don't exist, really.

    Z Plus's default rates are $2.5/m/tb with our cheapest plan starting at $5 / month for 2TB volumes attached to VPS's. USA hosted, all equipment owned, arrays are 3x7 raidz-2 so there's enough IOPS to go around.

    Our stuff's built on enterprise hardware, and I come from a background of running long term archives. I'm a strong believer you get what you pay for, so no my service isn't ideal - but I've done the math to ensure it's sustainable.

    There will be a LES exclusive deal going up this Wednesday for 3.5TB of storage volume for $7/month from Z+

    I also offer slot hosting, just mail in a 20tb drive and I'll host it.

    You need better marketing. You spent time writing that post but failed to include a single link to your company website (which does not contain the word "plus" in the domain name at all making it especially difficult to locate). Also on your website your "About Us" page is well buried and could inspire more confidence in your company. Best wishes.

    This is a help thread, figured it's not a place for me to advertise. I only posted what I do because it's topic relevant, if it was a requests' thread I'd have dropped a link directly to the order page and a YABS. I appreciate the feedback though. I'm always open to suggestions, feel free to DM me or call me out if I need to change things on the site etc.

  • HostBrrr had pretty good storage prices

  • @ZizzyDizzyMC said:
    There will be a LES exclusive deal going up this Wednesday for 3.5TB of storage volume for $7/month from Z+

    Yaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaas!

  • edited January 2025

    Thank you!

    uloz.to ... been around for a while, I think they started as a filesharing site
    ..
    50 TB Storage $32/mo
    300 TB for data transfer/month

    thanks! @cayc

    • Will the latency be better than FileCoin?
    • isn't it too cheap to be sustainable? what storage-redundancy?
    • rclone looks good. So i can simply use sshfs or any distributed / HDFS?
    • Do these profit from deduplication btw? With a large or decentralized systems the percentage could be high, and their average effective income also >$1.5 TB/month?
    • (Any Benchmarks? VS. hetzner storage box, mega, backblaze, gdrive, etc?)

    100€ Mini-PC with Intel N100 or some old laptop (practically free)
    50€ 4-Bay HDD enclosure on eBay or 100€ new
    100€ each refurbished 12TB drive on eBay from a reputable seller
    Buy everything using PayPal credit if you can't afford it in one go

    = 36TB + 12TB parity for 25€/month for 2 years, afterwards free forever

    thanks @lowendspiritxdax, Used HDD might still be spooky/uncertain(?)
    n100 is 10nm ;) - Can i combine a Ryzen Z1 extreme (most efficient) with SAS HBA? ;) got a lucky 16 TB SSD

    None of the cheap storage providers provide fast download/upload speeds

    how much?

    @archivist said: the prices are similar to HDD now

    @archivist said: SSD can be even cheaper considering wear down, electricity and no hardware failures

    Seems like a parallel dimension.

    hi @tetech, got used to HDDs? ;)
    $10/TB or $25/TB are similar, because SSDs can break even in a year or two (given wear-down and electricity. 10+ times faster than conservative investments) - so it just is unnecessary torture to wait 50 times longer for my 4 x 2 HDDs at OVH Kimsufi for example and less economical for them not to give us that (why @OVHcloud_APAC ? not one 8TB SATA SSD or 2x2TB or anything.) ( Replacement is expensive work too. And my latest OVH HDDs came with 40000? operating hours.

    Fully using HDDs for random reads (linux load up to 280), mine broke every few years, i had to replace7 HDDs yet. 6? were new/unused at the start ) (while usually you wont not need much speed but maybe just twice of HDD,bBut for example to replace 10 x old 8 TB SSDs you might want least 20 x 4 TB HDD.
    While SSD consumes ~ 3.5 watt (but usually be 99% idle), while HDD can consume 12 watt and be the bottle neck

    just mail in a 20tb drive and I'll host it

    thanks @ZizzyDizzyMC how much do you host? can i get z-3? - 16TB SSD too (and 5070 TI) (and 4TB DDR3? ;) )

    Hosthatch storage deal is $1.67 per TB.

    hi @dosai, currenty $3.8+

    HostBrrr had pretty good storage prices

    hi @corbpie "Our budget storage starts at just $2.5 per TB", seems only when paying yearly

  • @lowendspiritxdax said: = 36TB + 12TB parity for 25€/month for 2 years, afterwards free forever

    What is the expected lifespan of these drives?

  • edited January 2025

    What is the expected lifespan of these drives?

    (to be more complete, for pure backups you could have the HDD only run a few % of the time and be off mostly, so it would be cheaper for many years. Besides the latency to start a HDD is a few seconds. And Filecoin is $0.5/tb/month.)

  • edited January 2025

    (i'd be updating my list above, yet not allowed to.)

  • FrankZFrankZ ModeratorOG
    edited January 2025

    @archivist said:
    (i'd be updating my list above, yet not allowed to.)

    I expect that there is a time limit to edit the OP. If you want to update you can:
    1. Start a new thread with the updated info and ask me to merge the old and the new.
    2. Send me what you would like changed in the OP by DM. [preferably in code box] and I will modify the OP.
    3. Post what you want the new OP to be in a comment and I will edit the OP accordingly.

  • "How much do I host" - As much as people are willing to pay for. Bandwidth is limited but storage space is limited only by my bank account...
    "Can I get 16TB SSD in raidz3?" Yes, how much are you willing to pay for me to purchase 7.68tb disks? Would probably be more efficient in a raid 10 though speed isn't a concern here. Z2 on 6-8 disks would be sufficient in most cases.
    4TB of DDR3 - not sure if that's possible, however I do have a system capable of 3TB of (slow) ddr4. For that bundle I could probably do $0.25/GB/M. I know you were probably joking, but I'll happily take a clients money and deliver the service they paid for.

  • @ZizzyDizzyMC said:
    There will be a LES exclusive deal going up this Wednesday for 3.5TB of storage volume for $7/month from Z+

    Which Wednesday

  • @grasple said:

    @ZizzyDizzyMC said:
    There will be a LES exclusive deal going up this Wednesday for 3.5TB of storage volume for $7/month from Z+

    Which Wednesday

    Supposed to be last but I looked at my schedule and thought "I could have a good weekend if I didn't launch anything and minimized the risk of me breaking things"
    I had a good weekend. Working on setting up a slightly different storage infrastructure for future builds - wanted to test it to see if it will work as good / better as current. I'll still drop my offer, but eventually will have to migrate everyone to the new stuff if it works well.

  • how much do you host?

    hi @ZizzyDizzyMC how many machines and locations?

    just mail in

    joking

    16TB SSD, 5070 TI and TB Ram is what i was sending you too , since i'm tired of HDDs-only at OVH and not yet feeling too high tech ambitious with my low $1XXX budget in 2025 (just almost.)

    4TB of DDR3 - not sure if that's possible,

    a 96 x 64GB DDR3 4U 4-CPU is almost beautiful e-waste now (CPUs will waste electricity)

    I expect that there is a time limit to edit the OP

    1 hour. Aren't you the forum? Les go wiki

  • host_chost_c Provider
    edited January 2025

    @archivist said: SSD can be even cheaper considering wear down, electricity and no hardware failures.

    This is Wrong on so many levels, that I cannot even say.

    High Performance SSD will consume more power then a Magnetic SAS Drive.

    fail Rate on consumer drives used 24/7/365 is much higher then Enterprise drives used 24/7/365, especially after the first year.

    While Consumer SSD is cheap~ish, Enterprise is not and Enterprise drives with capacity of 6.4 TB or grater cost a fortune even on e-bay.

    Consumer SSD used in Enterprise/Hosting is like shooting your own balls, reliability wise. - This is a NoGo.

    A setup of 24 Enterprise drives in a raid60 is astronomical cost wise; Raid 10 on that would max out the 12GBPS SAS Link, will also give you enormous IOPS but it will cost a few K in USD to set up, I doubt that it would be feasible on the low-end-market for anyone.

    So spinning Rust is still the GO TO for Large Storage. Raid 10 can speed things up, but that will end up in a ~5USD / TB by today prices.

    EDIT:

    SSD VS HDD Power usage:

    6.4 TB SAS SSD

    18 TB SAS HDD

    Host-C | Storage by Design | AS211462

    “If it can’t guarantee behavior under load, it doesn’t belong in production.”

  • @archivist said:

    how much do you host?

    hi @ZizzyDizzyMC how many machines and locations?

    just mail in

    joking

    16TB SSD, 5070 TI and TB Ram is what i was sending you too , since i'm tired of HDDs-only at OVH and not yet feeling too high tech ambitious with my low $1XXX budget in 2025 (just almost.)

    4TB of DDR3 - not sure if that's possible,


    a 96 x 64GB DDR3 4U 4-CPU is almost beautiful e-waste now (CPUs will waste electricity)

    I expect that there is a time limit to edit the OP

    1 hour. Aren't you the forum? Les go wiki

    I don't currently offer GPU colo, this may change in the future.
    I don't colo ram either, mostly because all of my machines use matched sets.

    If you wanted to colo all three I'd recommend setting up a 2U GPU server and seeking colo rates from providers.

  • edited March 2025

    hi @host_c! Show me a single consumer SSD with at least 2TB from a server with at least 8 storage devices, that has worn out too much. And from read-only traffic? ;). SSDs have come a long way. DC versions go with TLC cells since several years, QLC too i suppose?

    We don't need to compare with NVMe's, which only happens when making it pump too many GB per second.
    Depending exact age and model, DC sata SSDs might idle at 0.8 watt and not be able to consume more than 4 watts dynamically.
    While HDD will consume much from spinning at all.

    HDD 7200rpm can do at most 120 requests per second. (not 100 000 like SSDs or "unlimited" like RAM).
    Thus its good for backups and cold archives but why would i run a HDD 24-7?
    And if 24-7 data ever needs many requests per second at rush hour you'd rather run one SSD than multiple HDDs.

    Say you had to host a 1TB DB with up to 500 000 small requests per second. You might go with some mirrored used DC or consumer SSDs. Rather than 2000 pieces of 15000 rpm drives

  • host_chost_c Provider

    @archivist said:
    hi @host_c! Show me a single consumer SSD with at least 2TB from a server with at least 8 storage devices, that has worn out too much. And from read-only traffic? ;). SSDs have come a long way. DC versions go with TLC cells since several years, QLC too i suppose?

    We don't need to compare with NVMe's, which only happens when making it pump too many GB per second.
    Depending exact age and model, DC sata SSDs might idle at 0.8 watt and not be able to consume more than 4 watts dynamically.
    While HDD will consume much from spinning at all.

    HDD 7200rpm can do at most 120 requests per second. (not 100 000 like SSDs or "unlimited" like RAM).
    Thus its good for backups and cold archives but why would i run a HDD 24-7?
    And if 24-7 data ever needs many requests per second at rush hour you'd rather run one SSD than multiple HDDs.

    Say you had to host a 1TB DB with up to 500 000 small requests per second. You might go with some mirrored used DC or consumer SSDs. Rather than 2000 pieces of 15000 rpm drives

    Those DC TLC QLC sata drives you mentioned, yes good for Hypervisor OS, for extensive data read and write - joke.

    Have you ever seen a raid 10 Array of 8 ssd "DC branded" sata drives fail? we had, not once and not only in hosting.

    IDLE does not exist in Hosting Environment, you cannot have 20 VM's idle, they never will.

    I will sustain my above as it is not from what I read, it is from what we saw while doing it.

    Also, if you do not believe me, get a few servers with at least 8 drives in each, start a hosting business for the next 3 years with consumer drives and I will wait here for the findings you had. - after the first 12 moths you will move away from consumer line as fast as you can run / do it. =)

    As a hosting company, you will never know what the customer will do on the service. That is why some have limits imposed, either by IOPS, max Read/Write speed or other.

    Host-C | Storage by Design | AS211462

    “If it can’t guarantee behavior under load, it doesn’t belong in production.”

  • edited March 2025

    good day @Host-C
    Unlike HDDs, SSDs dont wear down from reading.

    IDLE does not exist in Hosting

    dynamic chip load. Not spinning. Over 99% idle (or HDD is not an alternative) and 1000 times faster, thus up to 1000s of times more efficient.

    raid 10

    no double parity? i meant, did you wear down an SSD 2TB or larger? with what application?

  • host_chost_c Provider
    edited March 2025

    @archivist said: dynamic chip load. Not spinning. Over 99% idle (or HDD is not an alternative) and 1000 times faster, thus up to 1000s of times more efficient.

    Math wise yes, real world usage, depends on a ton of factors, brand of ssd, nand memory type and other. Consumer products are not meant to... last, as in that case, when will the next sale be? - this is totally counter retail business goals.

    @archivist said: no double parity? i meant, did you wear down an SSD 2TB or larger? with what application?

    Yes, raid 10 and wear level hit max, drive went into read only mode it's pair left the raid array for heck knows the reason, so the whole array went to shit. - and this not once with consumer ssd, other time some drives magically disappeared from the back-plane ( unit removed ) and I swear they were in the cady and the cady was in the back-plane =)

    AAA, just remembered, so a few drives had wear level max and some barely 5-7%, wtf??? so consumer = no-go for me.

    I could not care less if this was a FW issue or not, It was my stupidity to be blamed believing that consumer stuff can last, I learned my lesson, the hard way.

    I will let others experiment, it is all fun and specs until you loose customer data, then it is not that fun, trust those that went thru this shit.

    Here is one of our oldest Data-center DELL AIC NVME cards, it still works as in day 1:

    PM1725 - PCI-EX gen 3.0 - it can be considered a dinosaur by today NVME standards.

    SMART/Health Information (NVMe Log 0x02)
    
    Critical Warning:                   0x00
    Temperature:                        44 Celsius
    Available Spare:                    100%
    Available Spare Threshold:          10%
    Percentage Used:                    1%
    Data Units Read:                    7,735,851,372 [3.96 PB]
    Data Units Written:                 2,698,705,744 [1.38 PB]
    Host Read Commands:                 327,362,451,060
    Host Write Commands:                170,195,617,701
    Power Cycles:                       19
    Power On Hours:                     42,774
    Unsafe Shutdowns:                   10
    Media and Data Integrity Errors:    0
    Error Information Log Entries:      0
    

    Powered on hours = about 4 years and 10.5 months

    Consumer NVME in a customers Server soon to be decommissioned:

    SMART/Health Information (NVMe Log 0x02)
    Critical Warning:                   0x00
    Temperature:                        24 Celsius
    Available Spare:                    100%
    Available Spare Threshold:          5%
    Percentage Used:                    12%
    Data Units Read:                    584,345,351 [299 TB]
    Data Units Written:                 288,358,905 [147 TB]
    Host Read Commands:                 6,174,855,476
    Host Write Commands:                11,849,004,723
    Power Cycles:                       84
    Power On Hours:                     33,307
    Media and Data Integrity Errors:    0
    Error Information Log Entries:      64
    Warning  Comp. Temperature Time:    0
    Critical Comp. Temperature Time:    0
    

    Powered on hours = about 3 years and 9.6 months

    So about the same lifetime used, 10 times less read and write and it is cooked. ( ok 1 year difference, take into account that this NVME runs 2 VM's on it and that is all, the above... oh boy what it does since day 1 )

    Customer runs an accounting software + some measly static website from it.

    PS: the QLC are even worse

    found some in another customer server, ±3 years powered on and are 44% wear-out, but yes, those were in raid 10 and had 5 vm's on it, wow, really reliable, not to mention that out of 10pcs, 3 or 4 died in the first year. :+1: . Good that he had backups by the way, he only lost 12 hours of work when the R630 went off the grid.

    Host-C | Storage by Design | AS211462

    “If it can’t guarantee behavior under load, it doesn’t belong in production.”

  • hey @host_c would you stop?

    the last thing i need is more competition on the correct hardware. :s

  • FrankZFrankZ ModeratorOG

    It's been kinda nice seeing a drive conversation around here. I don't have much to add, but I expect like me, others are following the conversation.

  • I've had some good luck with Teamgroup MP44's in zfs raid 1, plenty fast, run fairly cool and I've been running one with a stress test and have plopped a few hundred TB on it without too much issues.

    All of Z+'s ZFS stuff runs on either 6 or 7 wide RaidZ2 or 12 wide RaidZ3 for the "idc how long it takes to access it" datasets. Customer facing is almost always 2-4 vdevs of 6 wide raidz2, gives us 250-500 raw IOPS and then it's cached by large enterprise NVMe drives.

    Admittedly I put a lot of effort into making sure the large data arrays is what's going to be left if anything were to happen. But at the end of the day, I'm not an iron mountain and I really hope any client who chooses to backup data with my company and service is doing so as a secondary backup.

    I'm getting to a volume of data where the power usage of the HDD's is actually a concern - due to using a bunch of 3 and 6TB drives for my own personal arrays I'm burning 400-500w of power for only 180tb of usable volume. That's pretty bad.

    Switching consumer side from enterprise 6TBs to enterprise 18TB's in July / Aug which will let me store data 3x more efficiently. Unfortunately it won't save me money because it's the same number of disks, but my usable volumes will 3x. Small chance I take the leap and go for 24TB disks.

    Since I work with such large datasets, I've actually started investing in tape, and now have the ability to store data on tape. Still working out how I'd like to offer this as a service, but there's already competition out there for ultra slow access data on the cheap.

  • AdvinAdvin Provider
    edited March 2025

    @host_c said:

    @archivist said: SSD can be even cheaper considering wear down, electricity and no hardware failures.

    This is Wrong on so many levels, that I cannot even say.

    High Performance SSD will consume more power then a Magnetic SAS Drive.

    fail Rate on consumer drives used 24/7/365 is much higher then Enterprise drives used 24/7/365, especially after the first year.

    While Consumer SSD is cheap~ish, Enterprise is not and Enterprise drives with capacity of 6.4 TB or grater cost a fortune even on e-bay.

    Consumer SSD used in Enterprise/Hosting is like shooting your own balls, reliability wise. - This is a NoGo.

    A setup of 24 Enterprise drives in a raid60 is astronomical cost wise; Raid 10 on that would max out the 12GBPS SAS Link, will also give you enormous IOPS but it will cost a few K in USD to set up, I doubt that it would be feasible on the low-end-market for anyone.

    So spinning Rust is still the GO TO for Large Storage. Raid 10 can speed things up, but that will end up in a ~5USD / TB by today prices.

    EDIT:

    SSD VS HDD Power usage:

    6.4 TB SAS SSD

    18 TB SAS HDD

    To be fair, I found that enterprise NVMe disks were cheaper than consumer NVMe disks at large storage capacities, specifically 3.84TB and above. At the moment, 7.68TB Enterprise NVMe is ~$450-550, but it depends on how the market is doing. I'm buying used 3.84TB Gen4 Enterprise NVMe drives for ~$230 each. I agree though, generally they are more expensive than HDD.

    Storage VMs have always interested us as a product to potentially offer, but we ultimately decided that using HDDs for such a product would not be worth the headache of having to deal with the generally slow performance of those HDDs. SSDs, while they are at a relatively low price, the upfront cost is still much more expensive than selling HDD VMs. I looked into it, and to make SSD VMs make sense, I would either need to sell it at ~$15/TB+ or use some sort of setup with deduplication and slightly overcommit space, which is both complex and really expensive to do correctly.

    A 24 x 7.68TB NVMe server would cost at least $10,800 for the drives alone, maybe add an extra +$3k for the server hardware itself (assume maybe an AMD EPYC system), and the margins just aren't really there. The bandwidth usage and I/O usage from clients would likely be absurd too.

    Best scenario I could assume, buy 12 x 15.36TB used NVMe for $900 each and run it in RAID5. You will get 168.96 TB in capacity. At $10/TB, you could make ~$1000/month. Then once someone starts using the write speed, you will have 1000 "Exposing XXX host for using hard drive!!!" posts because their Yabs.sh write speed is not 21,939,123 MB/s.

    I'll look into it again likely in a few months, to see if it's something we can offer. I'm hoping that we can come up with something that we could rent out at $10/TB/month, or if NAND flash pricing drops to that point. $5/TB/month would be the dream, but I know that it's basically impossible without a heavy ROI, even if you didn't have any redundancy at all.

    I am a representative of Advin Servers

  • AdvinAdvin Provider
    edited March 2025

    Also regarding Contabo and data loss: I’ve seen data loss incidents at pretty much every provider. You should be taking your own backups no matter what, regardless of the provider that you’re using. Even providers with billions, like Google, have lost data before. RAID does not always protect you and is not a backup, and many VPS hosts either don’t do backups or charge extra since it’s a huge logistical challenge. We do backups, but it’s only once a day, and not something we would do on a storage-focused product.

    Realistically, it would not be a bad idea to take a 1.8TB SSD Contabo VPS and then a 2TB HDD VPS elsewhere and sync them together, if you are happy with Contabo’s SSD performance.

    I am a representative of Advin Servers

  • host_chost_c Provider
    edited March 2025

    @ZizzyDizzyMC said: Switching consumer side from enterprise 6TBs to enterprise 18TB's in July / Aug which will let me store data 3x more efficiently. Unfortunately it won't save me money because it's the same number of disks, but my usable volumes will 3x. Small chance I take the leap and go for 24TB disks.

    If you plan to do 12 x18TB in Z2 ( ZFS ) = Respect for burning ~100 GB ram for that.

    When fragmentation will be above 20%, performance will be..... I cannot say the word as @AuroraZero will ban my ass, I will go with "low =) "

    Raid 6/60/Z2 or stripped Z2:

    I know that I am gonna get my ass whooped for trashing in SW raid from some/most of you ( it is not that it does not work, it works, but not as HW raid in large setups , also I said it does not work I mean hitting it whit a ton of IOPS, even for backup), I can say this, at 360 TB in stripped Z2 with 512 GB DDR4 only for ZFS after ~1 and a half hear year performance was barely saturating a 10GBPS link in sequential read / write. HW raid will also struggle with it ( fragmentation ), but at least you will not waste CPU Cycles for math and RAM for cache and also rely on a kernel not to freeze fuck up.

    In small numbers, up to 8 drives we also do ZFS sometimes ( well the only case is if I cannot fit in a HW raid controller )

    Yes, re-silver on ZFS is much faster then on HW raid at large capacity drives ( whatever is above 10TB ), sincerely, on 18 TB drives it finished ~5 days before the HW raid, while performance was rubbish on both during re-silver.

    And yes, we did not loose a single BIT with ZFS for more than 10 years, :3

    In our case RAID6/60 or any double parity raid ( HW or SW ) the bottleneck will be the math. Now I will trust an ASIC to do this rather then to introduce latency's from the CPU to do it.

    In raid 10 - whatever works, either HW or SW.

    The above for spinning rust.

    on NVME, or NVME RAID 1/10/5/6/60/z1/z2.... , caching and compression actually helps as data blocks are "evenly/full" distributed/filled and that little optimization you get helps the life of the NAND a LOT! , yet, if using Enerprise NVME, you saw my post, it can take PB read/write and just go on and on and on - you get the idea.

    From my point of view, if you want true performance from HDD, RAID10 and a minimum of 12 drives ( the more, the better ) but you will loose 50% of total capacity, and the outcome is $$$$$$$$$$$ ( ran out of $$$ =) )

    Budget/Backup/storage optimized raid 6/z2 up to 12 drives and 60 whatever is above. ( RAID60 = stripped raid 6 )

    There is no magic formula to have speed and redundancy and max capacity at the same time other then raid 10 and that is quite a fiasco if the underneath hardware is not "strong" quality wise.

    Waiting for others to share more info / insight on this. :3

    Host-C | Storage by Design | AS211462

    “If it can’t guarantee behavior under load, it doesn’t belong in production.”

  • My host nodes for disks currently have 1TB of ram, and I have 1 that's in use and another with identical specs as a backup that's powered off. Next host node setup will have 2TB of ram per node. Finding a 2u 2node system with 32 dimm slots per node is hard on a budget though. (Xeon Scalable gen2 or epyc gen 2/3)

    But yes, ZFS ram usage will just use the whole ram amount it's allowed by default on TrueNAS Scale. The reason I go for multiple vdevs of ZFS raidz2 is because I'm paranoid about disk loss. It's not about the capacity or the speed, it's about not risking client data if I can at all help it.

  • @host_c above 20% frag and the diks will be ass!!

    The Yeti has left the building.

  • AdvinAdvin Provider

    I am a representative of Advin Servers

Sign In or Register to comment.