Contemplating Dallas colo for storage, anyone interested
Lately I'm toying with the idea of setting up a server for cold storage (i.e. backups of backups) in one of the Dallas DCs. I'm wondering if anyone wants to do some cost-sharing. If I start with 48TB, it is probably around $1/TB/mo, but if I pack in 192TB (which I cannot use by myself) it goes down to more like $0.60-0.70.
Pros: new enterprise drives in RAID 5/6, KVM, pretty cheap (per TB), no exorbitant BW costs, flexible on config (this means you can use your own drives, send encrypted data via mail if you time it right, have your own RAID array or whatever).
Cons: this would be a bit older hardware (think 2x E5-2630 v3). So this is NOT for anyone who wants to run Plex with transcoding, CPU-based encryption, etc. No fancy SolusVM or DA control panel. No SLA - I am not your hosting company so it is not for you if you need 24/7 support or "five-nines" type uptime or will squeal if the ping is over 10 msec to your location.
Thoughts? Interest? At the moment still just contemplating the idea and haven't crunched it in detail, so no decision to do it yet.
Comments
I'm interested. Crunch some numbers please. Right now I have a box from Shock Hosting in Dallas I added 2x4tb that I'm paying $35 for. Maybe we can split this one as my storage needs have gone down? This box has a total of 10TB so that's like $3.5 per TB which isn't great as I would prefer something for $2 per tb or less. But it's got a dedicated 1270v3 with 32 GB of ram and I'm sure Shock will let me send them some different drives to swap out.
Definitely interested in about 6tb ish raid 6 (or raid z1/2 preferably) . Lmk final pricing if you proceed.
Recommended hosts:
Letbox, Data ideas, Hetzner
I am sure interested if I can get 4+TB storage with decent bandwith for <6$. Will be nice if I can run plex (will not be transcoding)
Need to see how much people want to store (on average). Putting 192TB into a HP Proliant DL380 in RAID5 works out to around $0.70/TB/mo. That colo plan has 5 usable IPs, so each person needs to take around 35TB on average (~$25/mo), otherwise there's an extra IP cost (~$1/IPv4/mo), IPv6-only, or some loss of efficiency.
If we go smaller, say a DL360 with 64TB, then we don't get the same scale and the per-TB cost becomes around $1.23/TB/mo, or (on average) around 10TB for $12/mo for 5 people.
That's all plus tax. Also depends what the arrangements are - I mean, I don't want to plonk down $600 for drives and then have someone cancel after a month leaving me stuck with unused drives, so we probably need to talk some type of setup cost sharing rather than just "per month".
Currently looking Equinix DA1, but that's just the "straw man" for pricing.
What is your definition of "decent"?
I might be interested in this, depending on the DC that you end up with.
Might make sense to be ipv6 only/nat ipv4 as its only meant to be backup storage. Cutting that cost would allow more users.
Recommended hosts:
Letbox, Data ideas, Hetzner
>
That's one inherent risk when doing something like this. Would need to be an understanding that if someone bails, the rest of the group would have to absorb the cost until someone else can step in to take their place.
Who are using for the colo and what is the monthly? joesdatacenter has some nice colo pricing with a /29 in KC for like $50 a month if Dallas isn't a hard point.
I would ideally like to avoid the ones used by the other VPS providers in case I end up with backups in the same building, but not sure how realistic that is. So probably not ColoCrossing, Dedipath, QuadraNet, Psychz. Currently looking Equinix DA1 or CoreSpace Dallas, but early stages.
That's a good comment. On the other hand more users is more admin and I don't want to become a hosting provider. I certainly don't want people taking 1TB here and there, that's an admin nightmare for me.
Yes. Personally I don't mind carrying the colo risk if people pay the pro-rata share of the drive purchase. So 4TB becomes something like $65 up-front and then $2.50-3.50/mo (equivalent) with no obligation. For that price, you'd potentially get something like 4-5 dedicated cores, 2.5GB RAM, 4TB HDD (based on splitting a DL360 into 12), which seems decent . Not everyone is in a position to do that up-front, so there's probably room to discuss something in-between.
Nothing fixed, still looking, but prices generally seem $30-35/mo range for 1U (e.g. LevelOne have this in Carrier-1) and $60/mo for 2U. Going to 2U can get 1Gbps unmetered. Dallas is a fixed thing for me.
Hmm, maybe. It doesn't have to be vps: borg or rsync is fine. I don't see a need for ip addresses, especially v4. v6-only or shared v4 is fine for this type of thing. Raid level should not be 5. raid-6 or zfs or (depending on how many drives) software erasure code all work. The cpus you mention are not too bad. You'd be buying the drives new? Would you require up front payment for 3 years or anything like that? How many drives in the box?
Actually this could be nice for initially backing up my laptop, if there is a usb port on the box. It would be great if I could send you a 256gb usb stick or sd card to plug in for a while. My home internet upload speed is slow enough that uploading that much isn't feasible for me.
I have 5TB in Hetzner Storagebox right now so I guess I'd want a similar amount with you.
A missile will destroy multiple data centers.
It might be safer to colo at @DataIdeas that is in Texas but not Dallas.
Very few VPS provides are in that city.
We accept Karma donations for the last flan. 🍮 affbrr
Yeah, I suspect several people who only want to use borg would be happy sharing an IP address. But it would be good to have a shared IPv4 as not all the servers I want to back up have IPv6.
I'd be interested for borg, but I'd probably only need 1-2TB, so I'm not really in your target demographic anyway. I'd maybe take more if you could get the prices really cheap though.
If you can beat my 12Tb for $30 with HostHatch am in
Doesn't have to be a VPS: true, but the server still has a cost to be spread around even if the resources are idle. So either we allocate the resources, or leave them shared. Giving people the option (dedicated or shared) is also fine; just means that there's more need for some "AUP" (even if informal) to ensure everyone gets a fair go. Same with IPs. Open for discussion.
RAID: it is a backup of a backup of a backup for me, so RAID5 keeps the cost down. If there's 4 drives then going to RAID6 means 2 data drives instead of 3, so the "cost/usable TB" goes up 50%. If there's 12 drives then OK, it is pretty marginal. So it depends a bit on the demand. As said above, I'm not looking to become a host and certainly don't want to deal with 100 people (think 1/10 of that, absolute max) and based on the comments so far we're looking at less than 10TB/person.
Drives would be new enterprise drives from a known brand (WD, Seagate, Toshiba, ...) with a 5-year warranty (2.5M hr MTBF). I allowed for a few replacements at cost, but generally I'm assuming the manufacturer stays in business and would honor the warranty, thus averaged the cost over 60 months. I could get a refurb DL360 with 4- or 8-bays (1U), or a DL380 with 12 bays (2U). Don't necessarily have to populate all the bays up front (but will need 3-4 depending on RAID).
Payment: open for discussion, but see previous comment. Drives are the largest cost. For 4TB, the "share" of the drive cost would be around $80 (depends on RAID level etc.) and the drive life is 5 years. One option is an up-front cost like $80 then a much smaller (like $2.50-3.50) monthly cost. After 5 years the amortization is basically zero and do it again with the then-current specs. People could also potentially get some lesser amount of TB and then pay a further share as they add more (i.e. pay as the drives are purchased). I'm open to a lower up-front cost (within reason) but that kind of equates to everyone else giving financing for drives and taking on the risk.
I'm very open to this, assuming data is encrypted etc. From memory the DL360 has 3 USB slots. Would kind of prefer it to having everyone saturate the network 24/7 for the first X months. The main constraint would be timing - needs to be before the thing is taken to the DC.
Dallas is fixed, so if someone is worried about a missile then they should pass on this.
$2.50/TB is beatable. I guess HH got you to prepay for a year or two (which funded the drive purchase).
More generally (for everyone): As mentioned, this would be a "cost sharing" project - I'm not a hosting company and don't have any SLA or 24/7 support, so if you're comparing with your current host remember that is a cost for them. But in some cases I might not have a disadvantage
Nah it’s monthly, I want out though. Would be happy to pre pay x amount of months too
Probably a bit on the low side, but maybe keep an eye on how it develops. Do you have a target for "really cheap"? I suppose at some price point it 4TB becomes cheaper than 1TB somewhere else.
Given you don't want to become a host I'd probably reccomend not selling this as a vps and just selling straight storage. Easier to manage and you won't have to eventually deal with the headache of who did what download of kiddie porn or other questionable activities. Especially if there is shared risk involved where the group eats the startup cost of the drives having potential liabilities ain't great.
Recommended hosts:
Letbox, Data ideas, Hetzner
StorageBox is monthly and is about $2/TB. The SX dedi line is a little bit out of whack starting at €1.44/TB/m for raw storage, but the last refresh was in early 2021, right in the middle of the Chia lunacy iirc. So maybe another refresh is due.
I see, I'd want to get the USB thing back after the data is tranferred though? I guess the transfer could happen during pre-deployment testing.
Whenever I've done the math on this kind of thing in the past, it's been hard to beat Hetzner, especially counting avoiding the hassle of mucking around with the hardware. Right now Hetzner is a little worse but with any luck they will resync soon.
Where are you located (general region) if I can ask? Near Dallas?
Those concerns are real. I don't look at it as "selling" at all, I look at it as cost-sharing amongst a restricted/invite-only group. So to be clear there will be no "ordering" web site and yes, there will be discrimination based on things like having a LES account for some period of time/some type of reputation, and there will be a restriction on the number of users to reduce the statistical probability of some idiot slipping through. There's a bit of trust involved in both directions.
Yes, assuming you are in the US then I don't see much of a problem.
I'm not keen on doing this for fun, so if a better option comes up then I'd take it and encourage everyone else to do likewise.
The problem for me is that getting the $/TB to a reasonable point generally requires a higher volume than I need (I'm happy with 10TB, myself). I'm actually already running a DL360 with 2x4TB + 2xSSD at home which amortizes to about $2/TB/mo (obviously no colo charge, but also pretty low density) and my server load is like 0.00 - it is totally underutilized.
DFW area.
Small note, Hetzner Auction has much better storage deals with no setup fees starting at 1 euro per TB.
https://www.serverhunter.com/#sort=price_per_hdd_capacity&sort_dir=asc
I am a representative of Advin Servers
It is a good point. A few "gotchas": the price/TB does not take RAID into consideration as I've been doing, so if you take that 4x10TB server and put it in RAID6, the price per TB is doubled.
You also only get that price with 20TB+. If the question is asked differently, "what is the cheapest for 10TB (usable) in RAID1/6/10" then the answer with the Hetzner auctions is "over $40" which isn't so great.
But maybe the answer for a lot of people is to stick with Hetzner, or at least stick with them if you want 20TB+.
Seems like the interest level here (with limited samples) is more around the 4-8 TB level.
Based on what people have said so far, see if this "straw man" outline sounds interesting.
Start-up:
Ongoing:
Admin:
So if you add the up-front cost to the ongoing cost, it is in the range $0.83-$1.61/TB/mo for RAID6 depending how densely the server is configured, with some reasonable (4TB?) minimum.
That's just for a proposal to gauge the reaction. I've made assumptions which could turn out to be off the mark.
If you were doing this for fun, I would have researched and provided some options, but since you aren't, I'll just come out and say it: Don't do it.
It will be far more trouble than it's worth: you only need 10TB, and the difference between your rate (0.6$) and the market rate (2.0$, ServaRica) means you'll save a grand total of 14 dollars, which is roughly an okay McD meal.
If you need a storage VPS, go for ServaRica BF Offers (2TB at 4$ or 3.5TB at 7$, paid annually).
Otherwise, get decent cloud storage at a discount:
MS OneDrive: Get 4 * MS 365 Personal 15-month subscriptions at 4 * 60$, redeem on MS website, add a month of MS 365 Family at 10$. 250$ gets you 6TB of storage for 5 years and 1 month.
Google Drive: CryptoRefill/BitRefill, get Turkey gift card to use Turkey's discounted pricing.
Fair comments; I already have Servarica and that's great but don't want to put all my eggs in one basket with them. Not willing to use MS or Google. I'm also looking at shutting down several other VMs which I currently am spending a few hundred on per year. For me this isn't "just" cheap storage.
The Hetzner alternative that Advin mentioned might be feasible though. It isn't in the US which is a limitation.
I haven't been following happenings in HD costs but this impressed me:
https://www.newegg.com/red-plus-wd120efbx-12tb/p/N82E16822234466
WD Red 12TB drives for $220 minus a $40 promo code = $180, so $15/T.B (T.B separated by a period is my abbreviation for decimal terabytes, i.e. 1e12 bytes). They are consumer drives with 3 year warranty but that is still under $.50/T.B/month.
Using 12 drives with raid-6 (2 parity drives), I get: 12T.B=10.9 TiB. 12 drives in box minus 2 parity = 10x10.9TiB=109 TiB in box. $2160 drive costs split over 36 months = $60/month, so $0.55/TiB/m for disk drives. Obviously that ignores the server cost and colo cost. And it would be lower if computed by 5 years, but 3 years may be more realistic given the still ongoing decreases in drive costs.
I'm not sure why use enterprise drives. Backblaze and seemingly other big users have consumer drives and just swap them out when they have to.
The Hetzner auction servers (15x 10TB drives for $118/month) are actually very attractive compared to this, enough to put me close to the "don't do it" camp. With two parity drives that is $1/TiB/month but that includes the entire server with beefy cpu and lots of ram, hosting, no setup fees, etc. Maybe we should be talking about splitting up one of those servers instead of this colo idea. With THREE parity disks it's $1.08/TiB/m. Hard to resist. (Edit: fixed some wrong numbers).
More bandwith than storage. Double/Triple would be awesome. Ready to reduce a bit of storage space for more bandwith
@tetech
If you're set on direct access to the HDD, these seem to be your options:
A Hetzner Auction server (storage servers seem to start around 33 euros, and the biggest ones go below 1e/TB)
Server-Factory storage VPS (2e/TB for NAT on multi-year payments)
PulsedMedia and HostHatch offers, when available.
If cloud storage (with RClone encryption) is feasible, you have GDrive and OneDrive, and smaller options like 1Fichier, Mega.nz etc.
Do you mean monthly? Idk how the economics works out but this storage is imagined as pure cold backups, I think. So there's not really a way to use bw=3x storage for the intended purpose and even 2x seems hard.
For colo'ing something an LTO tape drive would be really nice if they weren't so durn expensive.
All the said providers aren’t doing storage in Dallas, that was my main interest point
My gut feeling would be to we wary of WD Red+ as the older ones weren't apparently not clearly labelled as SMR. This link claims the drive is CMR, but my suspicions are still high on it!
Purely because (a) they have a 5 year warranty and averaging the cost over the warranty period makes them less per TB/mo and (b) Backblaze have staff on hand to just swap them out whereas we have to get remote hands to do it.
Yes, the Hetzner option is the best alternative going and may make this unattractive to the point where there isn't enough interest. I fully agree with that and won't try to bang a square peg into a round hole. It is only ~24 hours so still just gauging interest.
The 2U pricing appears to include unmetered 1 Gbps. In that case I wouldn't care about the quantity of BW but more how the usage impacts other people being able to upload/retrieve their stuff. I guess that people would hammer it pretty good for a few weeks while uploading and after that people are going to get the full 1 Gbps most of the time.
Yea, that's a big selling point for me too. Transferring data to another continent and being subject to exchange rates and German electricity costs (I saw Hetzner just announced that prices even for current products will be increased) isn't ideal. Plus I'd ideally like to run some stuff like my [internal] VoIP server and cancel the some of the VMs I have in Dallas. Can't really do that from Europe.
Seems like about 10% from the FAQ, but not disclosed yet.
If your $1/TB is manageable then I would stretch to 4TB. I've no idea what the tax rate would be, so ideally $1 including tax, but if it's not too high I also might not care.
However, I think all the data I have, if de-duplicated, would be 6TB, including data I don't care about at all and I currently only back up off-site about 0.5TB important data, and 0.6TB of maybe useful data. Given how little data I have, I might well just be best served by continuing to use external disks for the "probably not useful data" and finding a new place to put them away from home (my off-site location used to be my office, but I now work for myself from home, so that's no longer practical).
I'd also echo some of the warnings. It could easily become a nightmare organisationally for you to maintain this server and for relatively modest savings, and I suspect my storage requirements are probably too low to make it worth the hassle for you.
Any idea how much the color space itself will cost? I'm having redouble seeing it much under $100/m (2 amps) so that's $1/tb right there.
Tax is 8.25% on most things. All warnings duly noted.
So that's $65 with tax for the colo. Maximum density I assume is 12x16TB drives, so 160TB usable which is $0.40/TB/mo. Minimum density is whatever we decide is viable, but for the sake of discussion assume it is 64TB which is $1/TB/mo, tax included.
If there's really not much demand, then we can drop to 1U for $30/mo+tax.
That is an amazing colo offer. 3 amps is 262 KWH/month if my math is right, so that's about $.23/KWH counting zero for the rack space and network. I'd worry about them trying to jack up the price after a while.
I do think for a 12 drive dual cpu server, 3 amps are probably needed. For a 4 drive 1U server, 1 amp may be just barely possible using a low powered processor. What do you expect to spend on the server hardware? That has to be counted too.
About raid-5, the idea of a 12-drive raid-5 array (11+1) seems doomed. At minimum the system should be taken offline while any rebuild is in progress, since user activity during a rebuid slows the rebuild a lot. I'd prefer 10+2 or even 9+3.
What would the timeframe for this be? Later is better for me since I want to avoid significant expenditures for the moment.
I have an old 1U dual cpu Opteron server with 4 drive slots that I can donate, but it weighs a ton and probably wants at least 2 amps, so may not be cost effective.
Another note, you don't need a Turkey gift card or credit card to get the discounted pricing. You don't even need a VPN. If you set your country to Turkey at checkout, you can use a regular Visa credit card without needing a VPN.
I am a representative of Advin Servers
Enterprise drives are becoming a lot cheaper, most drives 10-12TB+ are NAS/enterprise already.
If you're fine with refurbished, you can get 16TB's for as low as $180 and 14TB's for as low as $150
Refurb:
https://www.amazon.com/Arsenal-7200RPM-3-5-Inch-Drive-Renewed/dp/B08K3TR9HD/
Although, the deal you pointed out is quite good 🤝
I am a representative of Advin Servers
What is this thing about Turkey?
I see some amazing offers in Hetzner auction now, even at the relatively low end. Like Intel Xeon E3-1271V3, 32GB ECC ram, and 4x10TB enterprise drives in Helsinki for €42.70/month. I'm paying around €28 for a non-ECC i7-3770 and 2x3TB drives so maybe I should just upgrade. There are also some intermediate offers like 2x6TB etc.
They might (who knows what happens to future prices), but I'm not too worried. There are similar offers from others. They are going to assume that not everyone will max out their power allowance 100% of the time. Anyway, my retail cost is something like $0.15/kWh.
I think it is pretty clear there will not be enough demand for 12 drives at least for some time. This is my actual usage on a dual-Xeon DL360 with 4 drives over the past 24 hours (blue = average, red = peak):
The server itself, under $500 (refurb, obviously) for a 12-bay DL380 with 2x E5-2630 v3, 32GB RAM, redundant power supplies, P840 RAID controller. Was already counted in all earlier numbers. This is one of the smallest costs (like $8/mo over 5 years, but it will definitely be ready for retirement then). Just the two parity drives for RAID6 would cost that much again.
After the first couple of posts, numbers have all assumed RAID6. This really hurts the $/TB for 4 drives, so if there is low demand maybe it gets revisited.
I'd probably aim for a go/no-go decision in the next few weeks, but to actually put it in production maybe mid-October. It would be kind of nice to give people time to copy stuff off BF deals in time to cancel them in an orderly way.
It is a nice offer, thank you, but yeah it probably isn't worth trying to save too much on that.
If you're fine with Helsinki, then it is a good deal.
@willie, let's split up one of those hetzner boxes and proxmox/nat it up? what do you say?
Hmm, I have some interest in that. What do you have in mind?
I'm flexible. I can do anything from the 4x6tb up to the 4x16tb.. you can put it on your account or I can use mine. don't matter to me. The $/TB is amazing on all of those right now. trying to get some storage before the price hikes come along.
PS - location can be either HEL, NBG or FSN, I have no preference.
How much space do you want? Would it be just you and me? I have a 2x3tb dedi now that I'd want to cancel, but I do sometimes use it for computing. Mostly idle, but occasionally a long computation (hours to days) on all 4 cores. Would that get in your way?
I let this sit for a week or so, and it seemed to "die a death", which is fair enough.
I'm probably still going to do the colo for my own needs but will limit it to 1U without the same potential for expansion. I'm going to re-think what to put into it, and if there's still spare space I might just offer that after the thing is running.
That sounds like a reasonable plan: although, a 1U server (not counting drives) tends to cost at least as much as a 2U, and in some data centers, an extra 1U of rack space doesn't cost much either (it's electric power and bandwidth that cost). So you could leave yourself some headroom. Anyway please keep us updated about what you do, even if there is no offer of space to others. Some of us like to keep an eye on this sort of option. Up til now colo just hasn't kept up with hosted storage, surprisingly.
Yes, the server cost will basically be zero difference. The colo for 1U is approximately half 2U though, and that is the killer if I'm doing it for myself. I already have a (1U) DL360 at my house and I am tempted to move that to colo and replace it with a (2U) DL380 at the house, where space isn't a constraint.