Aggregating results asynchronously with Redis cache, Python
Building a high-performing hotel booking platform while integrating with multiple suppliers.
Background
In collaboration with Ascenda Loyalty, my team and I were tasked to build a high-performing hotel booking platform that aggregates hotel prices based on hotels and destinations from various suppliers. One challenge that we faced was the different response time from different suppliers which could affect user experience negatively. In this article, I will share more about the challenges in detail and the solution that we have designed to overcome those challenges!
Challenges
As a hotel booking platform that displays hotel prices offered by various hotel suppliers, it was important to mitigate potential bottlenecks that may impact the user experience negatively.
1. Displaying search results quickly
As we retrieve prices from our hotel suppliers’, some return a faster response than others. Some may take 1–5seconds while others may take up to 30 seconds. This prevents customers from viewing search results quickly as they may have to wait up to 30 seconds before the full results are displayed.
2. Ensuring consistent display across views
Prices generated by the hotel suppliers are not fixed thus, they may vary between views. This prevents customers from booking a hotel at the same price when switching views.
Thinking out-of-the-box
The gist of the solution is to utilise Redis cache as a mediator between the client and our backend services, and enabling our backend services to retrieve results from suppliers asynchronously. The goal of this is to send any available results back to the client without waiting for the completed results to be available to ensure results are displayed to the customers quickly.
Redis cache as a mediator
Redis cache plays a pivotal role in our solution in aggregating prices from the various hotel suppliers while ensuring fast and consistent results.
We need a mediator between the client and server to aggregate and store results for a period of time to ensure that the results are displayed consistently during the process of aggregation and across views. Redis cache makes an excellent choice for a use case like this as it is able to provide good performance, store data temporarily for a specified duration and ensuring consistent results.
How it works?
To display any available results to our users, our client would need to know if the response returned by our server contains the completed results thus, we will be including a boolean variable `is_completed` to indicate the status of the results. The client will periodically retrieve results from the cache until the results are completed.
A decorator @destination_search_cache, containing the caching logic, is implemented to determine if results exist in cache. If results does not exist in cache, the server will proceed by calling suppliers’ endpoints asynchronously to retrieve hotel prices.
We have utilised threading to call multiple endpoints in parallel and aggregating them into redis cache with the hotel search parameter as the key once the hotel supplier has returned a response.
User Interface
Upon searching for hotels, available hotel results will be displayed to users and the loading bar indicates that the client is still retrieving results from suppliers. Results are sorted according to prices in ascending order. With the help of ReactJS’s state management, we are able to bump up the cheaper prices seamlessly as new results are retrieved.
Feedback
“Cool approach of trying to give users the results as fast as possible, while fetching fresh data and populating the cache.” — Ascenda
Takeaways
Through this project, I’ve gained a better understanding of how performance and availability of a web application can be improved with AWS services. It’s one of the projects that I’m proud of as I had to think out-of-the-box to design and implement a solution and I’m happy that it worked out as I expected!
Conclusion
In essence, the implementation of redis cache has help us provide better user experience for the customers by ensuring good performance and consistent results across views. Hope you’ve managed to gain some insights to how redis cache works, you can also watch it in action here!
