We want to provide all the tools our customers need for data and storage, but sometimes the best solution is one the customer creates on their own. In this tutorial, available in full on the Zenko forums, our Head of Research Vianney Rancurel demonstrates how to set up a CloudServer instance to perform additional functions from a Python script.
The environment for this instance includes a modified version of CloudServer deployed in Kubernetes (Minikube will also work) with Helm, AWS CLI, Kubeless and Kafka. Kubeless is a serverless framework designed to be deployed on a Kubernetes cluster, which allows users to call functions in other languages through Kafka triggers (full documentation). We’re taking advantage of this feature to call a Python script that produces two thumbnails of any image that is uploaded to CloudServer.
The modified version of CloudServer will generate Kafka events in a specific topic for each S3 operation. When a user uploads a photo, CloudServer pushes a message to the Kafka topic and the Kafka trigger runs the Python script to create two thumbnail images based on the image uploaded.
This setup allows users to create scripts in popular languages like Python, Ruby and Node.js to configure the best solutions to automate their workflows. Check out the video below to see Kubeless and Kafka triggers in action.
As the media and entertainment industry modernizes, companies are leveraging private and public cloud technology to meet the ever-increasing demands of consumers. Scality Zenko can be integrated with existing public cloud tools, such as Microsoft Azure’s Video Indexer, to help “cloudify” media assets.
Azure’s Video Indexer utilizes machine learning and artificial intelligence to automate a number of tasks, including face detection, thumbnail extraction and object identification. When paired with the Zenko Orbit multi-cloud browser, metadata can be automatically created by the Indexer and imported as tags into Zenko Orbit.
Check out the demo of Zenko Orbit and Video Indexer to see them in action. A raw video file—with no information on content beyond a filename—is uploaded with Zenko Orbit, automatically indexed through the Azure tool, and the newly created metadata is fed back into Zenko as tags for the video file. Note that Orbit also supports user-created tags, so more information can be added if Indexer misses something important.
Why is this relevant?
Applications don’t need to support multiple APIs to use the best cloud features. Zenko Orbit uses the S3 APIs and seamlessly translates the calls to Azure Blob Storage API.
The metadata catalog is the same, wherever the data is stored. The metadata added by Video Indexer are available even if the files are expired from Azure and replicated to other locations.
Enjoy the demo:
Don’t hesitate to reach out on the Zenko Forums with questions.
CloudServer can now be deployed on a Kubernetes cluster through Helm. The CloudServer Helm chart allows to add S3-compatible storage system to a K8s cluster easily. CloudServer can store data locally or can be used with existing S3 compatible servers by supplying credentials to the values.yaml file in the chart. See the full documentation at the Helm charts Github repository.
To start using CloudServer on an existing Kubernetes cluster, run:
$ helm install stable/cloudserver
In order to connect CloudServer to S3 compatible services, fill in cloud backend credentials on the values.yaml file. An AWS configuration may look like this:
Minikube and Docker for Mac Edge also support single-node Kubernetes for local testing. Docker has a step-by-step guide for such a set up. CloudServer’s full documentation covers other details. Try it out and let us know what you think on Zenko forums.
