Our colleague Lukas has explained how to use the front-end in previous article Uploading and resizing images with ReactJS: Part 1 - Client Side to user-defined dimensions. This article is going to deal with image uploads and manipulation from the server point of view. We have picked a solution by Amazon.
Amazon is one of the leading cloud service providers along with Microsoft Azure and Google Cloud. You can find an overview of a few alternative providers at clutch.co in an article comparing cloud services. Amazon offers a number of services from video processing, artificial intelligence, through game server operation, VPS, to database tools. AWS runs architectures both involving hundreds of virtual machines and single-machine instances. So, with AWS, the project size does not matter much. Here at Kurzor, we try to utilise AWS on projects planning big-time traffic growth in the future or ones that we want to manage and scale effectively.
Let’s use three basic services to create a standard server on AWS. I use S3 - Simple Storage Service to store files on a server. Next, the API Gateway creates the REST API. Last and most important, Lambda functions written in node.js run the whole process of image manipulation.
Our Architecture on AWS: How does it work?
The following exhibit describes the entire process, including the interfaces between and among all components:
So now let’s take a more detailed look at the individual components.
How to create and set up S3
The first thing we have to create is a new bucket in S3 to upload files. All you need to do is set a policy document that serves to define access to S3. The easiest way to do this is to allow anyone to read files in the bucket but to upload an image, you need to have a link to the required file.
For more detailed information, check out this easy-to-read tutorial from Amazon.
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "PublicReadGetObject",
"Effect": "Allow",
"Principal": "*",
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::kurzor.article.example/*"
}
]
}TIP: It would be more secure but also more time-consuming to create a unique folder for every user in the bucket, giving them rights in the identity access management (AWS IAM). Consequently, every user would need authentication through credentials (access key id, secret access key, session token) before an image request.
ApiGateway
ApiGateway serves to create the REST API that calls the individual Lambda functions. API settings are entered through an easy interface as shown in the exhibit below:
Again, we’d like to refer you to the documentation produced very well by Amazon.
TIP: Be careful with CORS – the ApiGateway makes it possible to set headers in great detail kept by your HTTP response. So, if you know you are going to deal with this problem, do not forget to define the response headers Access-Control-Allow-Origin.
Crop using the Lambda function
An image crop is processed using the Lambda function, which is a computing service from AWS and you only get charged for the time it takes to process a request. You can write the functions in node.js, Python, Java or C#. The example shows the use of node.js.
For a detailed look at the world of Lambda functions, we’d like to refer you to documentation provided by Amazon again.
The library known as gm - GraphicsMagick for node.js is used for the crop. gm contains an effective set of tools and libraries to work with an image in over 88 major formats (GIF, JPEG, JPEG-2000, PNG, PDF, PNM, TIFF, ...).
Lambda Functions
So, now let’s go to the core of our problem: We are going to code, finally! Below, you can see parts of the code from our implementation of the Lambda function. We are not showing the whole thing on purpose because our solution is tailor-made to client specs and the app requirements. We recommend you going down the same path: Understand how it works, and write your own code.
Image download from S3
The following function downloads an image from the AWS S3 storage. The S3 bucket identification and the file path are the input parameters.
function getFileFromS3 (bucket, key, callback ) {
s3.getObject({
Bucket: bucket,
Key: key
}, function (err, data) {
if(err) {
// error processing
callback (false);
} else {
callback (data);
}
});
}Parameters:
- bucket - name in S3 from which an image is downloaded
- key - image identification in S3
- callback - function that continues the flow
Crop
The initial parameters for crop are the coordinates of the upper left corner of the cropped area, the width and height of the cropped image, and the image being cropped. The cropped image is the output. Using the gm library, the crop can also be applied to an animated GIF.
function (context, image, width, height, x, y, callback)
gm(image)
.crop(width, height, x, y);
.toBuffer(format,function (err, buffer) {
if (err){
callback (false);
} else {
callback (buffer);
}
});
}Parameters:
- image - input image
- width, height - of the cropped image
- x, y - the coordinates of the initial point of the crop
- callback - function that continues the flow
Storing an image back to S3
The following function stores the image back to the S3 storage. Access authorisation using ACL is set to public (once the URL of the specific file is known, it is accessible to anyone).
function putObject (bucket, buffer, ext) {
var key = 'test.jpg'; //generate unique name
s3.putObject ({
ACL: 'public-read',
Bucket: bucket,
Key: key,
Body: buffer
}, function(err, data) {
// error handling
});
}Illustration: The entire process
Troubleshooting
The Lambda function and hence also the API Gateway should be able to fully resolve any error status. Such as the following examples:
-
The image download from S3 fails
-
Lambda function runs out of memory because the image is too big
-
The uploaded image is damaged or in an unsupported format
In the API Gateway discussed in the previous chapter, we already created the /crop endpoint called using POST. Data such as the coordinates and the size of the area being cropped along with the identification of the object (image) in the S3 bucket are part of the request.
The response to this call is in an exactly defined format, JSON in our case, and contains the full URL pointing to the output image in the S3 bucket. In this case, HTTP 200 is the mapped response. Other cases, most probably errors, return an HTTP 5xx. If it’s necessary to verify the user identity for the Lambda function (the user must be logged in), you need to modify the settings in the API Gateway, and an error in user verification returns HTTP 401 or 403.
Conclusion
I hope this preview of our solution has helped you understand how to set up a simple web service on Amazon, and you can see how to get from your HTTP request to a specific code.
Of course, you are encouraged to bring in further improvements. For example, you can replace the callback parameter in our functions by Promises. I leave this up to you and your experimenting!
