whats the best way to backup 200TB of data - amazon cold storage? or buy a tape backup

Collapse
X
Collapse
 
  • Page of 1
  • Time
  • Show
Clear All
new posts
Previous template Next
  • necoeds
    Confirmed User
    • Nov 2002
    • 2453
    #1

    whats the best way to backup 200TB of data - amazon cold storage? or buy a tape backup

    what I need to do for a backup of my studio hard drives with all my raw content from the past 22 years of filming
    Currently I have 200TB locally here in Europe (on external drives)
    I have duplicate external drives at my office in the USA, I have been syncing them using a vpn and a sync program for years.
    BUT, this is becoming tedious, and i'm running into issues now with my RAW VR180 video files being like 250GB just one file, and my syncing software is fucking up
    ...so maybe you have some solution (I don't even know what to ask for specifically) ... I just want a cloud backup of my 200TB ... but this is COLD storage... I only need it, if one of my drives failshahere in Europe...
    I probably should have said that first, I'm looking for a huge amount of cold storage - on tape is would assume... 2-3 months to recover a 16tb drive is ok for me, if something fails
    I've searched and gotten a good idea of the cost of cold storage cloud storage - from amazon google etc... But I am hoping that just a tape system at mojo could be my solution
    OR maybe just a tape system at my office in nebraska
    Message me to promote nebraskacoeds + network sites...
    Tags: None
    • k0nr4d
      Confirmed User
      • Aug 2006
      • 9231
      #2
      Wasabi is much cheaper then S3.

      As for the cheapest way to archive data, and the safest - it's gonna be tapes...
      Mechanical Bunny Media
      Mechbunny Tube Script | Mechbunny Webcam Aggregator Script | Custom Web Development

        Comment

        • Major (Tom)
          So Fucking Banned
          • Nov 2003
          • 32492
          #3
          Depending on your electrical grid reliability and budget, buy an old dell r610, load it with windows server, and an lsi raid card, several “rackable systems” sas enclosures, put in 80 or so wd nas red 4tb drives (possibly larger) and run it in a raid 60. It’s about 300 tb of redundancy. Get a good battery backup too.

            Comment

            • Brad Mitchell
              Confirmed User
              • Nov 2001
              • 9813
              #4
              Check your email.. We have a new cold storage tier that is less expensive that Wasabi with more features so it's a better alternative to that and Amazon's glacier. We haven't put it up on the site yet or started marketing it... but I'm happy to put thousands of HDDs to work for ya

              Brad
              President at MojoHost | brad at mojohost dot com | Skype MojoHostBrad
              71 industry awards for hosting and professional excellence since 1999

                Comment

                • 2MuchMark
                  Mark of 2Much.net
                  • Aug 2004
                  • 50981
                  #5
                  Hi Necoeds,

                  What you want to do is balance your Recovery Time Objective (RTO), and Recovery Point Objective (RPO).

                  According to AWS at https://docs.aws.amazon.com/wellarch...bjectives.html, Your RTO would be defined as the maximum acceptable delay between the interruption of service and restoration of service. This determines what is considered an acceptable time window when service is unavailable.

                  Your RPO is the maximum acceptable amount of time since the last data recovery point. This determines what is considered an acceptable loss of data between the last recovery point and the interruption of service.

                  I recently attended a seminar by AWS and they have alot of backup solutions, not just S3. Even their cheaper solutions still give you fast access to your backups. I think their slowest one must be Tape though, and would expect much slower access to that.

                  I have been using AWS alot and am really happy with their prices, options and speed. I think Microsoft Azure and Google cloud may offer similar solutions but I have never used their backup / storage services.

                  Good luck!

                    Comment

                    • plsureking
                      bored
                      • Aug 2003
                      • 4904
                      #6
                      Originally posted by DukeSkywalker
                      Depending on your electrical grid reliability and budget, buy an old dell r610, load it with windows server, and an lsi raid card, several “rackable systems” sas enclosures, put in 80 or so wd nas red 4tb drives (possibly larger) and run it in a raid 60. It’s about 300 tb of redundancy. Get a good battery backup too.
                      yea keep it local. upfront cost but no monthly costs. if you choose one of these "cloud" solutions, you are married to them, paying monthly fees, and your data is at risk to any random issue.

                      you content and data only needs to be web accessible if you are serving it thru a website or node.

                      you can get 16TB drives on Amazon (prime not aws lol) for around $300.

                      #
                      Last edited by plsureking; 11-10-2022, 07:04 AM. Reason: not aws
                      PornCMS / low cost paysite management with hosting

                        Comment

                        • DVTimes
                          xxx
                          • Jun 2003
                          • 31658
                          #7
                          Personally I would buy a few 2T external hard drives.

                          Back up your content to them.

                          While tapes may be cheaper, I suspect they may degrade in time.

                          If you plan on uploading to an external system, how long will it take you to upload that amount of data?
                          XXX

                            Comment

                            • ravo
                              Confirmed User
                              • Jun 2001
                              • 5461
                              #8
                              Originally posted by DVTimes
                              Personally I would buy a few 2T external hard drives.
                              A few??? Try 100+
                              AdultAdBroker - Buy and Sell Your Flat Rate Banners, Links, Tabs, Pops, Email Clicks and Members' Area Traffic - updated May 2026

                                Comment

                                • Brad Mitchell
                                  Confirmed User
                                  • Nov 2001
                                  • 9813
                                  #9
                                  Certainly nothing could be cheaper long-term than just building some very large systems with raided 16TB drives - I could help you figure that out pretty easy. Having said that, if its the choice make sure you design a raid type with enough sparing. Also, any data that is just on a single system should have another copy somewhere else, for sure.

                                  Brad
                                  President at MojoHost | brad at mojohost dot com | Skype MojoHostBrad
                                  71 industry awards for hosting and professional excellence since 1999

                                    Comment

                                    • 2MuchMark
                                      Mark of 2Much.net
                                      • Aug 2004
                                      • 50981
                                      #10
                                      Originally posted by plsureking
                                      yea keep it local. upfront cost but no monthly costs. if you choose one of these "cloud" solutions, you are married to them, paying monthly fees, and your data is at risk to any random issue.
                                      Not exactly. Your data is always yours and you always have access to it. A good cloud backup strategy would include backups in 2 different physical locations, at both an East Coast and a West Coast data center of AWS for example, which you can manage. So this means that even in the event of a catastrophic disaster, you can still access your data. And like any good datacenter, built-in redundancy, backups, hardware replacements, UPS's and especially security are all a part of the service.


                                      Originally posted by plsureking
                                      you can get 16TB drives on Amazon (prime not aws lol) for around $300.

                                      #
                                      This might be a good idea, but the question to ask is how valuable is the data. If the owner cannot ever lose the data and must have it accessible, then a cloud backup needs to be considered as a part of the overall backup strategy, since hard drives can fail, power failures can occur, OS's can crash, Ransomware attacks, etc etc.

                                        Comment

                                        • plsureking
                                          bored
                                          • Aug 2003
                                          • 4904
                                          #11
                                          Originally posted by 2MuchMark
                                          Not exactly. Your data is always yours and you always have access to it.
                                          first of all, glad to hear you are still alive.

                                          second, related to my first point. in case of death or bankruptcy, that data is no longer yours because you can't pay to host it anymore. at that point, you will need to make a local backup, which you can't afford because of the whole death/bankruptcy thing.

                                          so even if you want to host it in the air (aka someone else's drives), make sure you have it locally backed up.

                                          side note - most porn sites are already backed up on all the tubes

                                          #
                                          PornCMS / low cost paysite management with hosting

                                            Comment

                                            • DVTimes
                                              xxx
                                              • Jun 2003
                                              • 31658
                                              #12
                                              It is just to back up files, so no point having some fancy system designed for constant uploading data, as in vid editing.

                                              As said, get a few hard rives, and back stuff up..

                                              Remember hard drives can fail It is why I would not opt for huge storage, but more smaller drives, that way, if one fails, you loose less data.

                                              I still would love to know how long it would take to upload all that data you have to something online, as you must have a fast connection. It is my big problem in the UK uploading 4k vids as it takes ages.

                                              In a few years they may have 500TB drives that cost next to nothing, and all you need do is get your drives and copy everything to it.

                                              Do not forget it is only a few years ago when 1gb SD cards costs over $100. Today they sell 250GB SD cards for next to nothing.

                                              The advantage of buying hard drives, is that you can buy them as you need them too, to save cash. As it will take some time to fill them with data.

                                              One thing too, is question if you do need RAW files, or can you convert them to MP4 or something? As that will reduce the file size. It may be worth having both RAW and say MP4 versions, in case of a crash.
                                              XXX

                                                Comment

                                                • zijlstravideo
                                                  Confirmed User
                                                  • Sep 2013
                                                  • 806
                                                  #13
                                                  What about the decentralized storage networks out there? They used to be a lot cheaper compared to the regular cloud storage options. Networks like Storj, Sia etc.
                                                  Contact: email

                                                    Comment

                                                    • SpicyM
                                                      Confirmed User
                                                      • Aug 2006
                                                      • 4575
                                                      #14
                                                      I would never rely on a third party service for backups, just like plsureking mentioned.

                                                      I would invest $4k, buy 10 drives, 20TB each and store them in a safe place. You can buy one for $390 on Amazon.
                                                      no sig, sorry

                                                        Comment

                                                        • wankawonk
                                                          Confirmed User
                                                          • Aug 2015
                                                          • 1018
                                                          #15
                                                          hard drives at home (even in RAID) are NOT suitable long-term storage devices

                                                          you'll get minimum 5 years before data loss, and realistically more like 10-20, but you will lose data within a couple decades. almost guaranteed.

                                                          Interestingly -- if you burn the data to write-only DVDs it will last at least 100 years on the DVDs. I believe this is the most robust long-term at-home data storage format.

                                                          that said....use amazon cold storage.

                                                            Comment

                                                            • NoWhErE
                                                              Too lazy to set a custom title
                                                              • Sep 2005
                                                              • 10583
                                                              #16
                                                              How much would 200TB cost on Amazon Glacier (or S3)?
                                                              skype: lordofthecameltoe

                                                                Comment

                                                                • plsureking
                                                                  bored
                                                                  • Aug 2003
                                                                  • 4904
                                                                  #17
                                                                  Originally posted by wankawonk
                                                                  hard drives at home (even in RAID) are NOT suitable long-term storage devices

                                                                  you'll get minimum 5 years before data loss, and realistically more like 10-20, but you will lose data within a couple decades. almost guaranteed.

                                                                  Interestingly -- if you burn the data to write-only DVDs it will last at least 100 years on the DVDs. I believe this is the most robust long-term at-home data storage format.

                                                                  that said....use amazon cold storage.
                                                                  sounds like a non-techie response. you only buy one computer or phone every 10-20 years? what happens when things break at your house? throw them in the backyard?

                                                                  all equipment has to be maintained and updated. nothing lasts forever. the price of a home storage solution will be a fraction of any cloud or cold storage - and you own it.

                                                                  #
                                                                  PornCMS / low cost paysite management with hosting

                                                                    Comment

                                                                    • plsureking
                                                                      bored
                                                                      • Aug 2003
                                                                      • 4904
                                                                      #18
                                                                      Originally posted by NoWhErE
                                                                      How much would 200TB cost on Amazon Glacier (or S3)?
                                                                      $737 USD / month

                                                                      https://calculator.aws/#/addService/S3Glacier

                                                                      #
                                                                      PornCMS / low cost paysite management with hosting

                                                                        Comment

                                                                        • Tubevideditor
                                                                          Confirmed User
                                                                          • Sep 2013
                                                                          • 352
                                                                          #19
                                                                          Also looking for something similar.

                                                                          Cloud storage is ok but without a fast internet connection, uploading tens or hundreds of terabytes won't be quick nor fun and you're tied into a monthly fee literally forever.
                                                                          If anything goes down, you're without access to it.

                                                                          Hard drives not really an option as they can fail over time. Will also need lots of them so will take up a lot of space.

                                                                          LTO tape back up seems the best option for me but the tape recorder machines are surprisingly expensive for what they actually are but the actual tapes are cheap.
                                                                          You also need an interface to use the player.

                                                                            Comment

                                                                            • plsureking
                                                                              bored
                                                                              • Aug 2003
                                                                              • 4904
                                                                              #20
                                                                              education is your friend, and Youtube is your teacher.



                                                                              .
                                                                              .
                                                                              but but but, hard drives fail over time!! ok...



                                                                              or you can pay that $700-1000 a month and own nothing.



                                                                              #
                                                                              PornCMS / low cost paysite management with hosting

                                                                                Comment

                                                                                • jamezon
                                                                                  Confirmed User
                                                                                  • Apr 2019
                                                                                  • 136
                                                                                  #21
                                                                                  lets say one of those 10 x 20 TB drives statistically fails within the next decade but 95% of the rest works fine for the next 20 years , will you survive that ?? or are you willing to pay 10times the price to archive the 99% level of security..then use tapes but deal with the issues (time, price, trickiness etc)..also please do avoid RAID, theres no need to have a system for offline storage where one disk depends on the other. also one thing, if you are in need to get back to some of the data, accessing the bare metal of the harddrive is much faster and easier then restoring it from tape. another thing with tapes, the bottleneck is the tape reading machine, they do make problems with outdated drivers or mechanical issues after 10- 15 years too sometimes,especially when urgently needed. a data center can cope with that but a single user.....

                                                                                    Comment

                                                                                    • Major (Tom)
                                                                                      So Fucking Banned
                                                                                      • Nov 2003
                                                                                      • 32492
                                                                                      #22
                                                                                      Originally posted by plsureking
                                                                                      yea keep it local. upfront cost but no monthly costs. if you choose one of these "cloud" solutions, you are married to them, paying monthly fees, and your data is at risk to any random issue.

                                                                                      you content and data only needs to be web accessible if you are serving it thru a website or node.

                                                                                      you can get 16TB drives on Amazon (prime not aws lol) for around $300.

                                                                                      #
                                                                                      The field narrows with which raid cards support 16tb drives

                                                                                        Comment

                                                                                        • Major (Tom)
                                                                                          So Fucking Banned
                                                                                          • Nov 2003
                                                                                          • 32492
                                                                                          #23
                                                                                          Originally posted by jamezon
                                                                                          lets say one of those 10 x 20 TB drives statistically fails within the next decade but 95% of the rest works fine for the next 20 years , will you survive that ?? or are you willing to pay 10times the price to archive the 99% level of security..then use tapes but deal with the issues (time, price, trickiness etc)..also please do avoid RAID, theres no need to have a system for offline storage where one disk depends on the other. also one thing, if you are in need to get back to some of the data, accessing the bare metal of the harddrive is much faster and easier then restoring it from tape. another thing with tapes, the bottleneck is the tape reading machine, they do make problems with outdated drivers or mechanical issues after 10- 15 years too sometimes,especially when urgently needed. a data center can cope with that but a single user.....
                                                                                          Raid 60 solves that. Each drive has parity

                                                                                            Comment

                                                                                            • Major (Tom)
                                                                                              So Fucking Banned
                                                                                              • Nov 2003
                                                                                              • 32492
                                                                                              #24
                                                                                              Originally posted by wankawonk
                                                                                              hard drives at home (even in RAID) are NOT suitable long-term storage devices

                                                                                              you'll get minimum 5 years before data loss, and realistically more like 10-20, but you will lose data within a couple decades. almost guaranteed.

                                                                                              Interestingly -- if you burn the data to write-only DVDs it will last at least 100 years on the DVDs. I believe this is the most robust long-term at-home data storage format.

                                                                                              that said....use amazon cold storage.
                                                                                              5 years? Going 7.5 and strong

                                                                                                Comment

                                                                                                • plsureking
                                                                                                  bored
                                                                                                  • Aug 2003
                                                                                                  • 4904
                                                                                                  #25
                                                                                                  Originally posted by DukeSkywalker
                                                                                                  Raid 60 solves that. Each drive has parity
                                                                                                  ya some people can't do fucking research

                                                                                                  1 drive fails in 20 years? you win!

                                                                                                  keep a swap in the closet...

                                                                                                  #
                                                                                                  PornCMS / low cost paysite management with hosting

                                                                                                    Comment

                                                                                                    • CurrentlySober
                                                                                                      Too lazy to wipe my ass
                                                                                                      • Aug 2002
                                                                                                      • 38946
                                                                                                      #26
                                                                                                      Originally posted by plsureking
                                                                                                      [SIZE="5"]
                                                                                                      or you can pay that $700-1000 a month and own nothing.
                                                                                                      you forgot to say '& be happy?'


                                                                                                      👁️ 👍️ 💩

                                                                                                        Comment

                                                                                                        • sandman!
                                                                                                          Icq: 14420613
                                                                                                          • Mar 2001
                                                                                                          • 15431
                                                                                                          #27
                                                                                                          Cheapest will be buying a bunch of 18+ tb hds and storing it yourself.

                                                                                                          Online backups will cost more but then you have less work so depends on what you want
                                                                                                          Need WebHosting ? Email me for some great deals [email protected]

                                                                                                            Comment

                                                                                                            • Freedom6995
                                                                                                              Friends of Venus founder
                                                                                                              • Jul 2010
                                                                                                              • 1975
                                                                                                              #28
                                                                                                              fwiw, I purchased a portable 16TB SSD drive on Amazon last year for around $80

                                                                                                                Comment

                                                                                                                • plsureking
                                                                                                                  bored
                                                                                                                  • Aug 2003
                                                                                                                  • 4904
                                                                                                                  #29
                                                                                                                  Originally posted by CurrentlySober
                                                                                                                  you forgot to say '& be happy?'


                                                                                                                  Originally posted by Freedom6995
                                                                                                                  fwiw, I purchased a portable 16TB SSD drive on Amazon last year for around $80
                                                                                                                  black friday coming up soon

                                                                                                                  #
                                                                                                                  PornCMS / low cost paysite management with hosting

                                                                                                                    Comment

                                                                                                                    • Freedom6995
                                                                                                                      Friends of Venus founder
                                                                                                                      • Jul 2010
                                                                                                                      • 1975
                                                                                                                      #30
                                                                                                                      Originally posted by plsureking
                                                                                                                      :
                                                                                                                      black friday coming up soon
                                                                                                                      #
                                                                                                                      I had to look, and it was indeed a Black Friday deal for $75. I would post it but the item is longer available.

                                                                                                                        Comment

                                                                                                                        • money biz
                                                                                                                          Confirmed User
                                                                                                                          • Jan 2003
                                                                                                                          • 1016
                                                                                                                          #31
                                                                                                                          Originally posted by wankawonk
                                                                                                                          hard drives at home (even in RAID) are NOT suitable long-term storage devices

                                                                                                                          you'll get minimum 5 years before data loss, and realistically more like 10-20, but you will lose data within a couple decades. almost guaranteed.

                                                                                                                          Interestingly -- if you burn the data to write-only DVDs it will last at least 100 years on the DVDs. I believe this is the most robust long-term at-home data storage format.

                                                                                                                          that said....use amazon cold storage.

                                                                                                                          I have cds from 2002 that have been in storage units in the heat outside that still work on old computer.

                                                                                                                            Comment

                                                                                                                            • jsmih
                                                                                                                              Confirmed User
                                                                                                                              • Oct 2010
                                                                                                                              • 334
                                                                                                                              #32
                                                                                                                              Not big enough for your needs, but there are now 100 gb blu-ray archival disks called M disks).

                                                                                                                                Comment

                                                                                                                                • NoWhErE
                                                                                                                                  Too lazy to set a custom title
                                                                                                                                  • Sep 2005
                                                                                                                                  • 10583
                                                                                                                                  #33
                                                                                                                                  Originally posted by plsureking
                                                                                                                                  $737 USD / month

                                                                                                                                  https://calculator.aws/#/addService/S3Glacier

                                                                                                                                  #
                                                                                                                                  Yep. That’s a hard pass
                                                                                                                                  skype: lordofthecameltoe

                                                                                                                                    Comment

                                                                                                                                    • plsureking
                                                                                                                                      bored
                                                                                                                                      • Aug 2003
                                                                                                                                      • 4904
                                                                                                                                      #34
                                                                                                                                      Originally posted by jsmih
                                                                                                                                      Not big enough for your needs, but there are now 100 gb blu-ray archival disks called M disks).
                                                                                                                                      CDs have 100 year life expectancy but the capacity is too low. their day came and went. i really enjoyed laser disc too.

                                                                                                                                      hard drive capacity is growing every year. i upgrade my cms servers every 2 years just to max out capacity. my media/streaming servers after the upgrade last spring have 8x16TB drives. that's a buttload of porn!

                                                                                                                                      #
                                                                                                                                      PornCMS / low cost paysite management with hosting

                                                                                                                                        Comment

                                                                                                                                        • jamezon
                                                                                                                                          Confirmed User
                                                                                                                                          • Apr 2019
                                                                                                                                          • 136
                                                                                                                                          #35
                                                                                                                                          i have a dozen of cheap barracuda drives from 2005 that work to recover data from time to on the other side i have dozens of Dvds and Cds unable to read or constalty spitting out read errors. i think this thing with ever lasting CD and DVD is a Myth. there are more sensual to external factors like light, sun, dust , heat and prone to failure.

                                                                                                                                            Comment

                                                                                                                                            Previous template Next
                                                                                                                                            Working...