by Stefano Maffulli | Apr 24, 2018 | Stories
Following fruitful conversations between Yannick Guillerm, Director Of Technical Marketing at Scality, and Jim Donovan, VP Product at Wasabi, the Zenko core team added support for Wasabi Hot Cloud as one of its cloud backends.
This was a fairly easy integration since Wasabi uses the standard Amazon S3 API: most of the work went into adding Wasabi to the Zenko Orbit management interface and then testing it. What this integration means is that Zenko Orbit users can now replicate data between Wasabi Hot Cloud Storage buckets and all supported backends: AWS S3, local storage, Microsoft Azure, Google Cloud, Scality’s own RING… and more are coming.
Wasabi is a very competitive storage vendor and we believe adding it to Zenko Orbit is proof of Scality’s commitment to bring freedom and control to people who create value with data. We encourage you to check out the new integration and would love to hear what you think on our forum!
Since Zenko is an open-source project, you can influence where it goes next! Let us know what you’d like to see us support as Zenko’s next backend by opening a thread on the forum! Or you can add a new extension yourself and contribute it to the community with a pull request! Of course, we’ll be there to help every step of the way.
Let’s all welcome Wasabi to the ever-growing family of Zenko-supported data backends!
by Stefano Maffulli | Apr 19, 2018 | Stories
I’ve been following OpenStack for many years now and during the past few months I became more excited noticing how the project has become the home of open infrastructure. The OpenStack Foundation invested a lot of energy in the past year to visibly expand its scope and reach out to other communities and projects. The OpenStack Summits have been hosting lots of content about Kubernetes, Docker and use cases that go beyond public/private clouds using OpenStack. Multi-cloud and multiple technologies were so comfortably at home at the Sydney Summit that when I started reviewing the sponsoring plans for the rest of 2018 for Zenko, the OpenStack Summit felt like a natural fit.
Zenko is the open source multi-cloud data controller and OpenStack is the open source community where all open infrastructure comes together to innovate.
We’re sponsoring the OpenStack Superuser Award to celebrate the innovators in multi-cloud and open infrastructure. The nominees for the awards have been published, great companies and people bringing new meaning to automatic infrastructure. Go read their profiles and vote for your favorite candidate:
The voting process will be open until Tuesday, April 24 at 11:59 p.m. Pacific Time Zone. Based on community voting, the Superuser Editorial Advisory Board will review the nominees and determine the finalists and overall winner.
We’ll see each other next at the Vancouver Summit, on stage where the Zenko team will hand the Superuser award to the winning team.
by Giorgio Regni | Mar 3, 2018 | Data management, News, Stories, Tutorials
Maz from our team at Scality has been working on a simple guide explaining how to use Zenko and Orbit to replicate data between Microsoft Azure Blob storage and Amazon S3.
This is a very important milestone for us as it shows how easy it is to just create an account and login into the Zenko management portal, create a Zenko sandbox and start replicating data between 2 completely different public clouds replication wizard, no command line required. – Giorgio Regni
Why is this news worthy?
It is all about data durability and availability!
Replicating your data across different providers is a great way to increase its protection and guarantee that your data will always be available, even in the case of a catastrophic failure:
- In terms of durability, we now have two independent services each of which has a durability of eleven 9’s. By storing data across both clouds, we can increase our data durability to “22 9’s” that makes a data loss event a statistically negligible probability.
- We can also take advantage of immutability through object versioning in one or more of the cloud services, for even greater protection. We have also gained disaster recovery (D/R) protection, meaning the data is protected in the event of a total site disaster or loss.
- In terms of data availability, what are the chances that two cloud regions in one service (for example, AWS US East and AWS US West) are unavailable at the same time? Stretching this further, what are the chances that two INDEPENDENT cloud services such as AWS S3 and Azure Blob Storage are unavailable at the same time?
Download the ebook or read it here:
by Paul Speciale | Nov 28, 2017 | Stories
In our last Blog, we introduced the new Zenko Orbit portal. Orbit has been designed to radically simplify the management of multi-cloud storage, through easy point-and-click actions. Now, it’s time for us to look at the business potential and impact with a very interesting use case for multi-cloud storage.
So first, what is today’s typical model of using cloud storage services? In most cases, applications are written to use a single cloud such as AWS S3, Microsoft Azure Blob Storage, or Google Cloud Storage. All of these clouds are intrinsically highly-durable, with Service Level Agreements (SLA’s) for up to to an incredible “eleven 9’s” (99.999999999%) data durability. That is an incredibly high number, but it is also very important to understand that it still means data (objects) can be lost. This is a numbers game, and the more objects you store, the greater the chances are that you will lose some data even with this level of durability. The recourse offered by cloud vendors for violation of SLA’s are to provide service credits for future use of their cloud.
Another key consideration is data availability. So durability means “my data is safe”, whereas availability is about “can I get to my data”. Typical availability SLA’s are in the range of 99.0% to 99.95%, considerably lower than data durability. As we have seen over the course of the last few years, some smaller cloud storage services have disappeared entirely, and some of the bigger ones have suffered brief (minutes to several hours) of service outages to one or more of their regions, or even the service entirely. This has lead to some very prominent and publicized outages of well-known applications and services we all know and love. These include popular online video and entertainment services, ride-sharing, travel and communications services – and even enterprise SaaS applications that many business customers depend upon for their own operations.
Is Multi-Cloud Storage a Better Solution?
This problem made us ask the question: what can we do to improve both the durability AND the availability of our data in the cloud? What happens if instead of storing data in just one cloud region or cloud service, we use multi-cloud replication to store two copies of the data?
In terms of durability, we now have two independent services each of which has a durability of eleven 9’s. By storing data across both clouds, we can increase our data durability to “22 9’s” that makes a data loss event a statistically negligible probability. Furthermore, I can take advantage of immutability through object versioning in one or more of the cloud services, for even greater protection. I have also gained disaster recovery (D/R) protection, meaning the data is protected in the event of a total site disaster or loss. So in the end – this is essentially bulletproof data protection against most known events.
In terms of data availability, what are the chances that two cloud regions in one service (for example, AWS US East and AWS US West) are unavailable at the same time? Stretching this further, what are the chances that two INDEPENDENT cloud services such as AWS S3 and Azure Blob Storage are unavailable at the same time? The calculation can reveal the result, but it’s an exceedingly small probability so it essentially ensures that my data will be available.
What is the Cost of this Ultimate Data Protection?
Since we can likely agree that multi-cloud storage has positive benefits on data durability and availability – as with all good things, this has a cost we need to investigate. The cost of storing in multiple clouds has a few different components: storage capacity ($ per GB per month), bandwidth ($ per GB transferred) and transactions (number of PUTs, GETs, DELETEs).
Since most (but not all) cloud storage services make data INGESTED (written) into the cloud free, but data EGRESS (read) out of the cloud to the internet incurs fees, we need to take a look at the cost of not only storing the data but also the cost of EGRESS to replicate it to the second cloud. Note that with Zenko running in the cloud, we get the first copy into the cloud without any bandwidth charges.
One way to look at the bandwidth charges are to create a simplified TCO model. To make the numbers easy to start, we modeled 1 Petabyte (admittedly a lot of data) stored into the cloud for a period of 3-years. We added into that the cost of storage, bandwidth and the transaction fees also charged by most cloud services, and then compared the cost of one-copy stored in one cloud versus two-copies in two clouds. Our first observation was that the cost of bandwidth is a relatively small percentage of the overall TCO, typically 0 to % of the TCO:
View the full infographic
So this of course depends on the cloud vendors we analyzed, since there are variations in vendor charges and most interestingly some cloud vendors do NOT charge for egress charges out of their cloud. The model looks at a baseline in AWS S3 (single copy in US East alone), and compares it to two replicated copies in the the following services:
- AWS US East and AWS US West (using S3 Cross Region Replication, which incurs egress charges across regions)
- AWS US East and Azure Blob Storage Hot Tier (incurs Internet egress charges out of AWS S3)
- Azure Hot Tier and AWS US East (this avoids bandwidth charges, since Azure Hot tier does not have data egress bandwidth charges)
- Azure Hot Tier and Wasabi (Azure Hot tier with no egress charges to the newer Wasabi low cost storage service)
- Azure Hot Tier and Backblaze B2 (Azure Hot tier with no egress charges to the well-known low cost storage service from Backblaze)
Our second observation: it is indeed possible to store two replicated copies of data in two cloud services cost effectively. In fact, it is even possible to intelligently construct a scenario where two replicated copies are lower in TCO than a single copy in better known major cloud services.
Zenko and the Orbit platform provide an enabler to simplify this new world of multi-cloud storage. As we’ve looked at, there are dramatic data protection benefits by leveraging multi-cloud and solutions such as Zenko Orbit will provide ever simpler ways to take advantage of them.
View the full infographic
by Laure Vergeron | Aug 11, 2017 | Meetups & Events, Stories
Laure VergeronI was lucky to attend both London and Chicago AWS Summits for Scality recently. We were promoting Zenko, our multi-cloud data-controller, and the Scality RING, our flagship Software-Defined solution for on premise Storage. As an engineer, I was there to nurture technical conversations whenever possible, and what I found striking was how many of those happened in the course of those two events.
AWS Summit attendees have very diverse backgrounds, and it keeps you on your toes: anyone from a bachelor undergraduate student to the CTO of a global company could be the next person stopping by your booth. I had great conversations with junior software engineers, senior infrastructure engineers, architects… and not all of them were AWS experts! In fact, a lot of attendees stated being new to AWS. But all of them had a massive spark of interest when we talked about an open-source implementation of the S3 protocol.
My three main takeaways from this event are:
- Developing an application that will eventually run against S3 could have huge hidden costs: if you have been playing with AWS S3 for over a year, your free tier trial is over, and you start having to pay for every operation you perform against AWS S3. At this early stage, for any project, such costs can be very hard to bear as there is no revenue generated yet, and load/performance tests usually imply running a ton of operations… each time you try to pass them…
- A lot of big companies are trying to move from on-premise storage to cloud storage, but there is a conflict between the vision of the top executive, who wants to move everything regardless of the costs, and the top architects, who realize there are compliance requirements for some data that are hard to meet in the cloud, along with sensitive data which safety would be better controlled if kept on premise. This discussion needs to happen and, from our experience, we believe that large players in the global companies world will realize the need for “hybrid” solutions (on-premise and Cloud storage working side by side)
- When moving to the Cloud, whether for costs, tiering, or flexibility reasons, most large companies wish to split their assets across different Cloud Storage providers.
These takeaways are very positive for Scality and Zenko, as it means we already have answers to a problem people are just starting to grasp. Indeed, following the same structure as above:
- CloudServer, our Open Source implementation of the S3 API running locally in a Docker container, enables developers and start-ups to take the time they need to test their application against a fully compatible S3 frontend before moving it to AWS S3 as it goes in production.
- One way to connect to the RING is the “S3 Connector”, the Enterprise Edition of CloudServer; it comes with a few more features, especially an extensive support of IAM entities and calls.
Deploying a RING alongside an S3 Connector allows control of your on-premise data and your AWS S3 hosted data via a single API that has become the de facto standard for object storage: the S3 API. As the world moves towards having both kinds of storage available, this kind of ease of use is crucial to have.
- Zenko, which grows the CloudServer and S3 Connector ecosystem, currently enables on-premise high availability deployment architectures. It is scheduled to include an S3-to-Azure translator, and a cross-clouds aggregated metadata search by November 2017. If you are currently using multiple clouds, you know how much easier your life will be with these…
Finally, it comes with a management portal enabling visual monitoring and configuration of the service. The main monitored aspects are disk usage, CPU usage, and various utilization stats (S3 API calls). The main configurable variables are credentials for all organization’s members across cloud providers, and endpoints for all cloud providers.
Of course, all these functionalities will be available in exactly the same terms with your on premise storage provided it runs on a Scality RING. Moving (some of your data) to the Cloud has never been easier.
I strongly believe the multi-cloud approach is really what lies ahead of us, and these AWS Summits have done nothing but reinforce that impression. I encourage you to attend the next one happening close to you: these events are a great opportunity to peek at the future of storage, and to meet future partners! By the way, Scality (and thus Zenko) will attend NYC AWS Summit on August 14th, 2017… just in case 😉
Laure
by Giorgio Regni | Jul 11, 2017 | Stories
It’s hard to go anywhere lately without hearing the term Multi-cloud. But what does multi-cloud really mean for storage? Is it a new fancy word to replace “hybrid cloud”? Stay with me while I try answer these questions and share our definition of Multi-cloud and why we created, Zenko, an open-source Multi-cloud Data Controller.
In our vision, multi-cloud is an acknowledgment that the enterprise world is application centric and that each application has its own infrastructure needs that actually evolve over time. It’s only natural to want the freedom to leverage multiple, different cloud infrastructures at the same time and over time.
When we say Multi-cloud, it actually applies both to private clouds and public clouds. There’s a need to easily and transparently use different clouds based on their strength because in reality, AWS, Azure or Google Cloud each have their own area of expertise.
Multi-cloud is different than “hybrid” because it takes into consideration that an enterprise runs hundreds of different applications. Hybrid is more focused on tiering of old or lower value data to the cloud while Multi-cloud is about optimizing workflows and using the right tool for the right job at the right time. What we heard is that customers still like to manage storage locally in their own data centers, but need to use the cloud to leverage the native services they offer. This requires data mobility between clouds, whether private or public cloud services.
Multi-cloud also includes a notion of freedom: I am in front of customers frequently and one of the recurring topic is about not being locked into a specific cloud platform, whether it’s public or private. True freedom and data mobility can only arise if different cloud platforms uses the same communication protocols and share common abstractions to describe containers, objects, metadata and authentication credentials.
We do not see any initiative from the large public cloud providers or the numerous software defined storage vendors going in that direction so this is why we decided to start working on Zenko last year and are happy to announce that we’re making our source code available today as a set of open community projects on Github under an Apache 2.0 license.
Zenko is a Multi-cloud Data Controller and focuses on 4 pillars:
- AWS S3 API —Single API set and 360° access to any cloud
Gives developers an abstraction layer to enable freedom to use any cloud any time
Single unifying interface using the S3 API, supporting multi-cloud backend data storage, both on-premises (Scality RING and Docker) and public cloud with AWS S3, Microsoft Azure Blob Storage (and Google Cloud Storage to come soon)
- Native Format
Data written through Zenko is stored in the native format of the target storage and can be read directly, without the need to go through Zenko.
Therefore, data written in Azure Blob Store or in Amazon S3 can leverage the respective advanced services of these public clouds.
- Data workflow
Policy-based data management engine used for seamless data replication, data migration services or extended cloud workflow services like cloud analytics and content distribution (available in September)
- Metadata search
Provides the ability to subset data based on key attributes.
Interpret petabyte-scale data and easily manipulate it on any cloud to separate high-value information from data noise
Zenko focuses on ease of use and operation and relies on Docker Swarm for deployment and high availability. It runs as a set of containers either locally or in the cloud, anywhere that Docker can run, be it a laptop, physical servers or any existing cloud provider.
Please head to our community website, zenko.io, to learn more about Zenko and its architecture, look at how to contribute or download and use this new open source Multi-Cloud Data controller today.
