Serverless Image Handling Approaches in AWS
When a user uploads an image in high quality, we usually transform them to serve in a resolution that fits the device or at least in one standard resolution but not the same uploaded image.
Some use Image CDN like ImageKit or Cloudinary or self-hosted like Thumbor. Some write their own transformation logic like using lambda.
Images can be transformed by multiple approaches using event triggers or on-the-fly and/or at the edge to serve them in multiple resolutions. As time evolves, the approach improves better.
0. Legacy Approach [20xx]
The application code which is responsible for handling image upload also transforms the image in all the necessary sizes and puts it to S3 (or other storage) along with the original image.
1. Eager Transformation [201x]
I still remember the “Yay moment”, once images are uploaded to Amazon S3, they are transformed into multiple sizes to fit the device resolutions better. This eager processing is done by “event trigger” i.e., Lambda with S3 “object create” trigger.
Cons: All the images in all sizes are ready upon upload, but not all are used.
Pros: This is still worth considering when the required number of sizes are very well-known as one or a few standard resolutions and/or a thumbnail.
Tech: Lambda with S3 Trigger (Event Trigger)
2. On-the-Fly Transformation [Jan 2017]
Instead of storing all the sizes of images indefinitely, they can be transformed on the fly upon request.
When requesting (1) a different sized image if found in S3, it is served. If not found in S3, it is redirected (2) to resize API Lambda function (3, 4) which transforms and stores the resized image in S3 (5) and redirects back to the S3 endpoint of the new image (6, 7)
Pros: Cost-effective as only requested images are transformed; Resilience to failure as any image can be re-generated anytime even if images are lost.
Cons: The first request takes more time to redirect, transform, store, and again redirect and retrieve.
Tech: S3 with redirection rule, API Gateway, Lambda
3. Real On-the-Fly Transformation [Jun 2017]
Instead of multiple redirections, let’s implement like an Image CDN by using a caching layer to reduce the need for transformation in subsequent calls.
Any image that is not cached in CloudFront, is forwarded to the API gateway endpoint and triggers the lambda function to retrieve the image from S3 and sends the transformed image.
Pros: The transformed image is cached in CloudFront and not stored in S3.
Cons: Just the first request takes time but still only for transformation.
Tech: CloudFront, API Gateway, Lambda, S3
https://aws.amazon.com/solutions/implementations/serverless-image-handler/
4. Edge Transformation [Feb 2018]
Though the transformation was on the fly, let’s compute at the edge by leveraging the Lambda@Edge in CloudFront
When requesting a different sized image
- using “viewer request” integration, you can replace the requested dimension with the closest pre-defined dimension and also can specify webp for supported clients (1). This happens just at all the edge locations.
- if the specified image (dimension and format) is cached in CloudFront, the response is served (4). If not, request forwarded to origin S3 to get the specified image (2) and comes back at CloudFront (3)
- using “origin response” integration, if the image is retrieved from S3, the response is just forwarded (4). If not, the original image is fetched from S3 (5), transformed and stored back in S3, and also sent back to the client (4)
Pros: Computation like dimension adjustment, opting for webp happens at Edge resulting in a faster response.
Cons: Images that are not cached, still need to go to Origin which consumes the same time.
Tech: CloudFront, Lambda@Edge (Viewer Request, Origin Response), S3
Summary
The trigger for conversion moved away from business logic to event triggers. The “serverless” and “event triggers” are the first game-changer. But as time changes, the approach improves every time. Finally, the computation moved from Data Center to Edge locations
All the approaches have different pros and cons but still, any can be used.
- If you’re sure about the required sizes and it’s usage, [0, 1] are still worth considering. [0] or [1] depends based on the type of application and where it runs
- Some applications like WordPress still handles transformation by themselves [0]
- Approach [2, 3, 4] takes time only for the first request however [2] takes more time that may need to revisit
- If you’re not looking for adjustments of sizes or formats (webp) on the fly, [3] and [4] are almost similar
All we just need to remember is “Technology deprecates with time”. But this doesn’t mean we have to rewrite our applications every time.
