An important update
Backblaze officially added a “server side copy”-feature to their B2 service in August of 2019. Details can be found in this post on their blog. Most tools that support B2 have already added support for the new feature as well, so copying/moving data on B2 has become rather simple. No more need to set up a Packet instance to do it.
In order to simplify some of by backup efforts, I recently had to move just over 1.5TB of data on Backblaze B2. Sounds like a now-brainer, but B2 does not (yet) have a “server side copy”-/”server side move”-feature. Thus, rather than simply telling Backblaze’s servers to move the folder from A to B, you need to move by downloading the data from A and re-uploading it to B. This has a couple of downsides: Most notably, the move might take a “while”, depending on the amount of data and your connection. In my case it would take me several days to down- and upload the data on a 100 Mbit/s down/30 Mbit/s up connection. Furthermore, this will use much if not all of your data volume if you are using a metered connection. Finally, you will have to pay for both the download and some upload-related transfers. For my 1.5TB, the download would cost me about $15 (1500GB * $0.01/GB). The upload itself (as a Class A transaction) is free, but you might use up a few Class B transactions (e.g. b2_get_file_info, cost: $0.004 per 10,000 with 2,500 free per day) and Class C transactions (e.g. b2_list_file_names, cost: $0.004 per 1,000 with 2,500 free per day) using a tool such as rclone. So, I would probably pay a little over $15 total for moving a mere 1.5TB of data — not exactly a steal.
That pretty much describes the “challenge”. Let’s get into finding a cheaper solution.
Introducing: The partnership between Packet and Backblaze
Just before I had to actually move the data, I found a post on reddit when looking for details on the status of Backblaze’s european datacenter. Backblaze’s brianwski mentioned their compute partner “Packet”.
Downloading for B2 currently costs 1 penny per GByte, then re-uploading to the new region is free (as always). If you use one of th Backblaze compute partners (like “Packet”) then the download is free as well, and the data never goes through your personal bandwidth. [Link to source]
A quick look at Packet’s website revealed that their cheapest compute instance currently comes in at only $0.07/hour.
This means that paying $0,07/hour for a t1.small instance would be cheaper than moving the data myself as long as the transfer takes less than ~214hours ($15 divided by $0.07/hour = 214.29 hours). Let’s do some quick napkin-math with those figures: Say the transfer runs at 25MB/s (a very conservative estimate, more on that later). My transfer of 1.5TB/1500GB/1500000MB would take ~16.67 hours which would cost just over $1 (17 hours * $0.07 = $1.19). That would be $1.19 instead of $20 and it would way faster than using my own connection. Sign me up!
Signup for Packet’s services
Signing up for Packet is quite simple. However, my account could not automatically be verified and thus had go through manual verification by Packet’s support staff. They asked for my details on my use case and some sort of identification in form of an online profile such as GitHub, LinkedIn etc. (don’t ask me what they are looking for there). I told them that I was looking to move some data on B2 and provided a link to my (very unspectacular) GitHub profile. After less than an hour total, my account had been approved.
Deploying the t1.small server
I won’t go into details on how to deploy a server on Packet, since they documented that process quite well here. Just be sure to select the “on demand”-option and select “SJC1 (Sunnyvale)” as the server’s location — only transfers between that location and B2 are free (according to Backblaze’s partnership site). Once your t1.small instance is up and running, connect via ssh and use your favorite command line tool to move the data. Finally, delete your server as soon as you are done. Otherwise it will cost you $0.07/hour even if the server is shut down.
Finally, results
The transfer of my 1.5TB took a little over 5 hours at just over 79MB/s using rclone (16 transfers/workers).
So, I payed $0.42 for transferring the data much, much faster and payed about 97% less than I would have otherwise. My Backblaze bill has not been affected by the move whatsoever (even though the transfer used a few 1,000 Class B transactions).
Overall experience: 10/10, would recommend.
