Who hasn’t experienced a system or website that was working perfectly and suddenly starts slowing down and stops responding?
Clients begin to complain, and the most obvious solution is to increase the capacity of the database server — and everything returns to normal.
OK, problem solved — until the hosting bill arrives at the end of the month and you realize you’re paying too much to maintain a relatively simple system that should be low-cost. Worse, once you do the math, you find your profit margin is melting away...
If you’re looking for ways to reduce your database costs, you’ve probably considered solutions like:
- In-memory data caching
- Database optimization (compression, indexing, separating reads from writes)
- Scalable cloud services (AWS, Azure, or GCP)
However, there’s another simple, direct, and low-risk alternative. Unlike the options above, this one doesn’t require changes to your database structure — which means you don’t have to run exhaustive testing cycles to avoid potential side effects.
An extremely efficient alternative is the use of message queues. At 10i9, we are always seeking ways to ensure our applications are scalable, reliable, and available 24/7. Below, we’ll share a case study that brought significant performance improvements.
There are several reliable message queuing systems available on the market to help you implement efficient queue management, including Kafka, RabbitMQ, Amazon SQS, Azure Service Bus, among others.
With proper use of queues, we can drastically reduce the load of simultaneous connections to a relational database, avoiding overloads and ensuring that the application remains operational even during traffic spikes. In our case study, we show how we reduced the number of database connections from 6,000 to just 15, significantly easing the load and generating massive savings for our client.
The best queue solution for your use case depends on data load and complexity. For example, if your application handles high volume and complex data, Kafka is often a better fit — especially for large-scale, real-time processing needs. For more information on Kafka, feel free to contact us.
RabbitMQ, on the other hand, is easier to integrate with existing systems and technologies, allowing you to leverage its benefits without having to overhaul your entire infrastructure.
To further enhance performance, RabbitMQ can be deployed alongside AWS cloud services, such as EC2 instances or Amazon MQ. This allows companies to benefit from the scalability and flexibility of the cloud without compromising performance and reliability.
Industries That Benefit from RabbitMQ:
- E-Commerce: Efficient order processing and fast, accurate delivery of information to users.
- Vehicle Tracking: Manages database load efficiently to ensure connections don’t slow down the system.
- Media & Entertainment: Distributes real-time content across platforms.
- Finance: Ensures accurate and efficient processing of financial transactions.
- Healthcare: Integrates healthcare systems for real-time delivery of medical information.
- IoT: Facilitates communication between interconnected devices.
Case Study
In this project, we used RabbitMQ to reduce the number of simultaneous database connections. The tests were conducted using 330,000 vehicle tracking devices, each sending one request per second, totaling 6,000 requests per second.
The database used was RDS MySQL, and RabbitMQ was deployed on an EC2 instance.
Architecture Diagram: AWS + RabbitMQ + RDS MySQL
To enhance performance, we deployed RabbitMQ alongside AWS, leveraging EC2 and Amazon MQ. This setup enabled the system to scale efficiently while maintaining high reliability.
As shown in the diagram, with only one queue and 12 consumers, the system remained stable even under high load. The number of simultaneous database connections dropped from 6,000 to only 15.
If you’re looking for a scalable, reliable, and easy-to-integrate message queuing solution, RabbitMQ might be the right choice for your business. It ensures your application remains available during peak traffic, significantly reduces database load, and delivers excellent performance.
And by integrating it with AWS, you gain even more in terms of scalability, availability, flexibility, and cost reduction.
The image below shows a graph illustrating the number of messages the system must handle and the message throughput rate.
Message Rate and Load Graph
By properly using message queues, we can reduce the load of simultaneous connections to your database, thus avoiding overload and ensuring your application continues running smoothly even during traffic spikes. Additionally, RabbitMQ is highly scalable and reliable, ensuring your application remains operational even under high volumes of simultaneous data. However, if your application handles a larger and more complex data load, Kafka is a more suitable alternative — it is designed for large-scale applications that need to process vast amounts of data in real time. For more information about Kafka, feel free to contact us.
Performance Comparison with and without Message Queues
In addition, RabbitMQ is easy to integrate with other systems and technologies, as well as with your existing application, allowing you to take full advantage of its benefits without the need to completely overhaul your infrastructure.
It is also a highly recommended solution for companies facing challenges related to message management and queue processing. Some of the industries and use cases that can benefit include:
E-Commerce: RabbitMQ is useful for managing large-scale order processing, ensuring fast and accurate information delivery to users.
Vehicle Tracking: RabbitMQ helps manage database load more efficiently, preventing slow connections and avoiding system performance issues.
Media and Entertainment: RabbitMQ can be used to manage real-time content distribution across different platforms.
Finance: RabbitMQ is ideal for integrating financial systems, ensuring accurate and efficient transaction processing.
Healthcare: RabbitMQ can integrate healthcare systems, ensuring fast and accurate delivery of medical information to healthcare professionals.
IoT: RabbitMQ enables efficient communication between IoT devices by managing data flow and ensuring system stability.
In the case described, we applied RabbitMQ to reduce the number of simultaneous database connections. The tests were based on 330,000 vehicle tracking devices, each making one request per second — totaling 6,000 requests per second. The database used was RDS MySQL, and RabbitMQ was deployed on an EC2 instance.
References:
