Buffering for a better performance
Updated: Sep 20, 2020
Definition
Buffering is used to temporarily store data while it is being moved from one place to another.
How it can help
Cloud migration today has been de facto the ultimate solution for hosting services. I totally agree that moving to the cloud is truly amazing and has a tremendous power with a positive impact on the way we work. Since we care less about performance and much more about business logic. However, either the cloud hardware is limited or it's very costly.
Regardless of which architecture you are working on. A monolithic, micro-service or server-less architecture.
Sooner or later you will face how overwhelming a database, web server or a third party service would end up with your service struggling to process requests.
This is where the Buffer becomes handy.
By implementing the following you can prevent or minimize the impact of overwhelming a service:
Throttling
Request aggregation
Rate limiting
At this post I will discuss the second bullet of request aggregation.
Since I believe I have a powerful solution yet so simple to implement and use. Therefore, I’m offering the following Java library to assist you solving your use case in the best possible way.
The Buffer library provides multiple 'Buffer' implementations which support preventing the above described scenario.
Size Buffer
Buffer that holds requests up to a predefined size. When the size threshold is reached then the action will be triggered.
Time Buffer
Buffer that holds requests up to a predefined time (Millis). When the time threshold is reached then the action will be triggered.
Buffer
The widely used implementation includes time and size thresholds and when the time or size threshold is reached then the action will be triggered.
The Buffer is a simple utility which includes only two public methods:
append(T t) - You guessed it for appending tasks to the buffer.
run() - For running the required action on the buffered task list. As well to flush the buffer when required.
Usage
The following code snippet will generate a Buffer that triggers the buffer action when reaching 100 ms or 20 requests, the earliest.
Upon each action the timer will reset and the action will be scheduled to the next 100 ms.
Buffer b = new Buffer<>(100,
20,
ACTION,
DEFAULT_ON_FAILURE);Buffer demo
IntStream.rangeClosed(1, 101)
.forEach(i -> b.append(i));Output
As you can see below, the last action was triggered after 100 ms from the last appended request. Since the size threshold was not reached yet.
System time: [1600539095335], at this iteration there were [20] entries in the list
System time: [1600539095335], at this iteration there were [20] entries in the list
System time: [1600539095335], at this iteration there were [20] entries in the list
System time: [1600539095335], at this iteration there were [20] entries in the list
System time: [1600539095335], at this iteration there were [20] entries in the list
System time: [1600539095435], at this iteration there were [1] entries in the list
For the full usage and soon the tests please check the Github repository.
Github repository: https://github.com/jhashoul/buffer