In this article, we will show an AWS SQS Polling Example in Java. We will focus on the message receiving part and show you how to receive messages from an SQS standard queue by using short polling and long polling in Java.
1. Introduction
Amazon Simple Queue Service (SQS) is a fully managed message queuing service in the AWS cloud. Using SQS, We can send, store, and receive messages between software components at any volume, without losing messages or requiring other services to be available.
2. Technologies Used
- AWS SQS: distributed message queue
- AWS SDK for Java 2.0: Java APIs for AWS services
- Log4j2: logging APIs
- Gradle 6: build tool
- OpenJDK 13: an open-source implementation of the Java Platform, Standard Edition
Before we start, please follow AWS SDK for Java 2.0 Developer Guide to set up AWS credentials and region for development.
3. AWS SQS Polling Examples
3.1 The Basics
In messaging systems, “polling” is a way that one system continuously checks of other systems to see what state they are in and if they have any message to communicate. Amazon SQS short polling and long polling are two ways to retrieve messages from Amazon SQS queues. Short polling returns immediately, even if the message queue being polled is empty. Long polling doesn’t return a response until a message is available in the message queue or the long poll times out.
To demonstrate how to short poll and long poll messages, we need three components as below:
- A standard queue in SQS: a queue receives messages from producers and distributes messages to consumers.
- A producer: a user application that sends messages.
- A consumer: a user application that receives messages.
3.2 Creating a Standard Queue in SQS
Amazon SQS offers two types of message queues. Standard queues offer maximum throughput, best-effort ordering, and at-least-once delivery. FIFO queues are designed to guarantee that messages are processed exactly once, in the exact order that they are sent. A service client named SqsClient is used to access Amazon SQS. We can create a new instance of SqsClient as below:
SqsClient sqsClient = SqsClient.builder().region(Region.of("ap-southeast-2")).build();Then invoke the createQueue API provided by SqsClient with an instance of CreateQueueRequest to create a new standard queue as below:
CreateQueueRequest request = CreateQueueRequest.builder().queueName("MyQueue").build();
CreateQueueResponse response = sqsClient.createQueue(request);Note that the queue name of the standard queue we specified here is “MyQueue” because there is a contract defined by SqsClient that a FIFO queue name must end with the .fifo suffix.
3.3 Sending a Message
We use a producer to simulate a user application that sends messages to the SQS queue. The SqsClient provides sendMessage API for us to send a message to the SQS queue. Firstly, we create a sample message myEvent and serialize it to JSON string:
// sample message
MyEvent myEvent = new MyEvent();
myEvent.setId(UUID.randomUUID().toString());
myEvent.setSource(Thread.currentThread().getName());
myEvent.setPayload("AWS SQS message attributes example.");
String message = null;
try {
message = objectMapper.writeValueAsString(myEvent);
} catch (JsonProcessingException e) {
logger.error(e);
}Then, we build an instance of SendMessageRequest and call sendMessage API to send the message:
SendMessageRequest.Builder builder = SendMessageRequest.builder()
.queueUrl(queueUrl)
.messageBody(message);
// send the message
logger.info("Sending message to queue {}", this.queueName);
this.sqsClient.sendMessage(builder.build());3.4 Receiving Message by Short Polling
We use a consumer to simulate a user application that consumes messages from the queue. The SqsClient provides straight forward API receiveMessage for us to receive messages from a queue with short polling. It is suitable for applications expecting an immediate response from a receiveMessage call. For example, in a single thread application, the single thread polling messages have to handle other tasks such as responding to user actions as well. So an immediate response can let the single thread keep running without blocking the whole application. A Runnable task is submitted to a single thread pool to short poll messages. More details can be found in the source code attached.
The following code snippet shows an example of using short polling to receive messages from a queue. If there is a message available at the time of polling, the call will return with the message. Otherwise, the call will return an empty list immediately. Note that the key here is to call waitTimeSeconds method to set the waitTimeSeconds to 0 which forces the short polling when building the ReceiveMessageRequest. Please check out the Javadoc of waitTimeSeconds method for more details.
// short polling
ReceiveMessageRequest receiveMessageRequest = ReceiveMessageRequest.builder()
.queueUrl(queueUrl)
.waitTimeSeconds(0) // forces short polling
.build();
List<Message> messages = this.sqsClient.receiveMessage(receiveMessageRequest).messages();3.5 Receiving Message by Long Polling
Similarly, we use the same consumer above to simulate a user application that consumes messages from the queue. But this time we toggle the useLongPolling flag to true to enable long polling. In Amazon SQS, long polling results in higher performance at a reduced cost in the majority of use cases. So it is recommended unless the application expects an immediate response from a receiveMessage call. A Runnable the task is submitted to a single thread pool too long poll messages. More details can be found in the source code attached.
The following code snippet shows an example of using long-polling to receive messages from a queue. If there is a message available in 20 seconds, the call will return with the message. Otherwise, the call will keep waiting until 20 seconds times out and then returns an empty list. Note that the key here is to call waitTimeSeconds method to set the waitTimeSeconds to 20 which forces the long polling when building the ReceiveMessageRequest. Please check out the Javadoc of waitTimeSeconds method for more details.
// long polling and wait for waitTimeSeconds before timed out
ReceiveMessageRequest receiveMessageRequest = ReceiveMessageRequest.builder()
.queueUrl(queueUrl)
.waitTimeSeconds(20) // forces long polling
.build();
List<Message> messages = this.sqsClient.receiveMessage(receiveMessageRequest).messages();3.6 Running the Examples
For ease of dependencies management and build, we use Gradle 6.0 as the build tool. To run the short polling example, you can run the shortPollingExample Gradle task from the command line as below:
gradlew shortPollingExample
The output would be:
[23:54:42,490] [INFO] aws.demo.PollingExample:145 [main] - Create queue MyQueue in region ap-southeast-2 [23:54:43,563] [INFO] aws.demo.PollingExample:149 [main] - Queue URL: https://sqs.ap-southeast-2.amazonaws.com/000000000000/MyQueue [23:54:43,567] [INFO] aws.demo.PollingExample$MyConsumer:325 [pool-3-thread-1] - Short poll messages from MyQueue... [23:54:43,616] [INFO] aws.demo.PollingExample$MyConsumer:337 [pool-3-thread-1] - Polling attempt #1 [23:54:44,211] [INFO] aws.demo.PollingExample$MyConsumer:337 [pool-3-thread-1] - Polling attempt #2 [23:54:44,805] [INFO] aws.demo.PollingExample$MyConsumer:337 [pool-3-thread-1] - Polling attempt #3 [23:54:45,441] [INFO] aws.demo.PollingExample$MyConsumer:337 [pool-3-thread-1] - Polling attempt #4 [23:54:46,027] [INFO] aws.demo.PollingExample$MyConsumer:337 [pool-3-thread-1] - Polling attempt #5 [23:54:46,618] [INFO] aws.demo.PollingExample$MyConsumer:337 [pool-3-thread-1] - Polling attempt #6 [23:54:46,821] [INFO] aws.demo.PollingExample$MyProducer:280 [pool-4-thread-1] - Sending message to queue MyQueue [23:54:47,204] [INFO] aws.demo.PollingExample$MyConsumer:337 [pool-3-thread-1] - Polling attempt #7 [23:54:47,284] [INFO] aws.demo.PollingExample$MyConsumer:341 [pool-3-thread-1] - 1 messages received. [23:54:47,284] [INFO] aws.demo.PollingExample$MyConsumer:369 [pool-3-thread-1] - Processing message 9c2c4895-b337-4b04-93dc-8ba303d27ecf [23:54:47,342] [INFO] aws.demo.PollingExample$MyConsumer:379 [pool-3-thread-1] - Message processed: MyEvent=MyEvent[id='cafb12b2-ae83-4667-9862-dd6f0ed2c37c', timeStamp=2020-08-29T13:54:46.569036800Z, source='pool-4-thread-1', payload='AWS SQS polling example.'] [23:54:47,372] [INFO] aws.demo.PollingExample$MyConsumer:390 [pool-3-thread-1] - Deleting message 9c2c4895-b337-4b04-93dc-8ba303d27ecf from queue: MyQueue [23:54:47,413] [INFO] aws.demo.PollingExample:159 [main] - Delete queue MyQueue [23:54:47,550] [INFO] aws.demo.PollingExample:164 [main] - Queue MyQueue deleted.
We can see from the output above that the consumer was using short polling to retrieve messages from MyQueue. It polled six times (called receiveMessage method six times) until receiving a message at the seventh poll. Each call returned immediately without waiting. Let’s see what the differences are comparing to long polling.
To run the long polling example, you can run the longPollingExample gradle task from the command line as below:
gradlew longPollingExample
The output would be:
[00:04:28,108] [INFO] aws.demo.PollingExample:145 [main] - Create queue MyQueue in region ap-southeast-2 [00:04:28,910] [INFO] aws.demo.PollingExample:149 [main] - Queue URL: https://sqs.ap-southeast-2.amazonaws.com/720596691539/MyQueue [00:04:28,913] [INFO] aws.demo.PollingExample$MyConsumer:325 [pool-3-thread-1] - Long poll messages from MyQueue... [00:04:28,958] [INFO] aws.demo.PollingExample$MyConsumer:337 [pool-3-thread-1] - Polling attempt #1 [00:04:32,126] [INFO] aws.demo.PollingExample$MyProducer:280 [pool-4-thread-1] - Sending message to queue MyQueue [00:04:32,272] [INFO] aws.demo.PollingExample$MyConsumer:341 [pool-3-thread-1] - 1 messages received. [00:04:32,272] [INFO] aws.demo.PollingExample$MyConsumer:369 [pool-3-thread-1] - Processing message 079fa397-6b24-4af4-90b1-9fb06f1e9546 [00:04:32,304] [INFO] aws.demo.PollingExample$MyConsumer:379 [pool-3-thread-1] - Message processed: MyEvent=MyEvent[id='32352ae7-8a76-43a8-a1ea-dd4f04f75685', timeStamp=2020-08-29T14:04:31.914669800Z, source='pool-4-thread-1', payload='AWS SQS polling example.'] [00:04:32,333] [INFO] aws.demo.PollingExample$MyConsumer:390 [pool-3-thread-1] - Deleting message 079fa397-6b24-4af4-90b1-9fb06f1e9546 from queue: MyQueue [00:04:32,376] [INFO] aws.demo.PollingExample:159 [main] - Delete queue MyQueue [00:04:32,494] [INFO] aws.demo.PollingExample:164 [main] - Queue MyQueue deleted.
We can see from the output above that the consumer was using long-polling to retrieve messages from MyQueue. The first polling request (Polling attempt #1) was waiting for about 4 seconds and then returned with 1 message. There was only one receiveMessage call and the call was waiting for available messages instead of returning immediately.
4. Download the Example Source Code
That was a AWS SQS Polling Example in Java.
You can download the full source code of this example here: AWS SQS Polling Example in Java
