Object storage is a strategy to store data in an object format. The objects have a unique identifier and a set of metadata to describe it. Unlike block storage, object storage does not use any structure, making it scalable and flexible storage. Scalability allows you to store unlimited amounts of data hence it is the best solution for data analytics, as it requires unlimited storage and efficiency of retrieval, and object storage is designed for this. Objects can be retrieved efficiently using the metadata. Also, there is no structure to their storage; retrieval is also much more accessible. Object storage is also a perfect solution for cold data storage such as backups or archives, which requires large storage.
Cloud providers offer object storage at different pricing plans which could depend on storage capacity and the frequency of retrieval. For example, if the data is for data analytics or eCommerce sites, the frequency of data retrieval is higher, so here the provider may provide the pricing bases on both storage and access frequency.
S3 Compliant Object Storage
Object storage is the underlying technology for a S3 compliant storage. S3 compliant storage is built on the S3 API allowing data storage as well as manage and retrieve object stores. Applications which comply with the S3 API can communicate easily with any S3 compliant storage.
Building Your 0wn S3 Object Storage
MinIO is an S3 compliant open source object storage server which means it can work with any application that works with Amazon S3. Following are the great features of MinIO:
- Allows plug and play with any S3 compliant application
- Offering a high-performance object storage
- No vendor lock-in as it is built on an open source platform
- Simple installation and configuration
- Designed and developed for the cloud and it is Kubernetes-friendly
Files in MinIO are organised in buckets which can be accessed with an access key, secret key, and the server address on the MinIO instance. The size of each object can be from only a few KB to a maximum of 5TB.
Setting Up MinIO Server on Mac
Step 1: Install Homebrew
To have MinIO setup on Mac, install the MinIO packages using Homebrew first. Homebrew is an open source package to allow easy installation of software on Apple's macOS operating system and Linux.
Paste the following command from the Homebrew page on the macOS terminal to install Homebrew.
$ /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)"
Step 2: Install MinIO Server.
Now that you have successfully installed homebrew, the next step is to install the MinIO server with the following command.
$ brew install minio
If your installation is successful, you should have a successful message at the end.
Step 2: Starting up the MinIO server
$ minio server --address youripaddress:port fu
Once the server startup, you will get the Endpoint, AccessKey, and Secret Key display. You need this information to access the client and browser. It is recommended to change the Access Key and the Secret Key for security purposes.
Setting Up MinIO Client
Step 1: Install the minio client
$ brew install minio-mc
Step 2: Configure the client
$ mc config host add minio http://x.x.x.x:9000 accessCode secretCode
Step 3: Create a bucket
Create a bucket called photos
$ mc mb minio/photos
Step 4: Upload data
Upload some data to the bucket
$ mc cp ~/Downloads/s3.png minio/photos
View the Data in MinIO Browser
Use the address given in the browser access to access MinIO with the given MinIO server information from the
Enter the access key and secret key to access the dashboard. On the dashboard, you will see the bucket called photos and the image uploaded using the MinIO client.
You can also choose to create the bucket via the web interface on the MinIO browser. All you need to do is click on the plus icon at the bottom of the interface. This will give you an option to create a bucket and upload files using the GUI.
Securing Access to Your MinIO Server With a TLS Certificate
Your MinIO server can be secured using a private key and public certificate from a certification authority(CA). A CA like LetsEncrypt will be able to provide a certificate to enable HTTPS on web servers, and Certbot for a free SSL certificate. Follow the steps on Certbot to get your certificate for your webserver.
Backup to S3 Compatible Storage Using Backup Ninja
With the S3 compatible object storage, you have options to work with S3 compatible tools like Backup Ninja for automated backups. Backup Ninja offers safe and secure backup services to local or cloud storage. With Backup Ninja you can easily backup your open source databases like MySQL, MariaDB, MongoDB, PostgreSQL, and TimeScaleDB without any backup scripts to you the MinIO object storage. Follow these steps to configure MinIO as an S3 target on Backup Ninja.
Step1: Select the cloud provider.
Step 2 : Fill in the cloud credentials. This is your server information, the access key, and the secret key which you get when you start your MinIO server.
And you would be able to see the cloud credentials created for your S3 compatible storage.
You can now use this to schedule your back up in a few steps as below.
Step 1: Fill in the backup schedule details
Step 2: Pick the S3 compatible storage you created earlier from the list.
Step 3 : Fill in the settings for your backup or you can leave the settings to default.
Step 4: Set the schedule for your backup duration.
Once you have completed this step, you can see the schedule for backup on the dashboard. You need to click on the play button to start the backup.
MinIO is a great option as a primary or extended storage to your current infrastructure. With the high-performance object storage, you can store large data like database backups and archives at minimal cost without any vendor lock-in. Data recovery during any disaster strike becomes easy as you are in total control of your backup storage. MinIO server is easy to set up and can connect securely from a web interface using Let’s Encrypt SSL/TLS certificate. Lastly, and probably the most distinctive feature of the MinIO, is its S3 compatibility, making it work seamlessly with any S3 compliant tool.