In this article, we are going to explain the AWS SQS Delay Queue and Delay Message Example in Java.
1. Introduction
Amazon Simple Queue Service (SQS) is a fully managed message queuing service in the AWS cloud. Using SQS, We can send, store, and receive messages between software components at any volume, without losing messages or requiring other services to be available. The Delivery Delay concept was formally introduced in JMS 2.0 specification in 2013. When sending a message, a delivery delay can be specified and the queue manager does not deliver the message until after the specified delivery delay has elapsed. In this example, we will show you two options supported by Amazon SQS to implement Delivery Delay in Java.
2. Technologies Used
- AWS SQS: distributed message queue
- AWS SDK for Java 2.0: Java APIs for AWS services
- Log4j2: logging APIs
- Gradle 6: build tool
- OpenJDK 13: an open-source implementation of the Java Platform, Standard Edition
Before we start, please follow AWS SDK for Java 2.0 Developer Guide to set up AWS credentials and region for development.
3. AWS SQS Delivery Delay Options
3.1 AWS SQS Delay Queues
Delay queues is one of the two delivery delay options supported by Amazon SQS. With a delay queue, the delivery of messages sent to the queue will be postponed for a number of seconds. The default (minimum) delay for a queue is 0 seconds. The maximum is 15 minutes. Let’s see how to create a delay queue in Java below.
Amazon SQS offers two types of message queues. Standard queues offer maximum throughput, best-effort ordering, and at-least-once delivery. FIFO queues are designed to guarantee that messages are processed exactly once, in the exact order that they are sent. To access Amazon SQS, a service client named SqsClient is used. We can create a new instance of SqsClient as below:
SqsClient sqsClient = SqsClient.builder().region(Region.of("ap-southeast-2")).build();SqsClient provides the createQueue API and we can invoke it with an instance of CreateQueueRequest to create a new delay queue as below:
Map<QueueAttributeName, String> attributes = new HashMap<>();
attributes.put(QueueAttributeName.DELAY_SECONDS, "10");
CreateQueueRequest request = CreateQueueRequest.builder().queueName("MyQueue").attributes(attributes).build();
this.sqsClient.createQueue(request);Note that we set queue attributes QueueAttributeName.DELAY_SECONDS to 10 seconds to create a delay queue. If we send a message to the delay queue created above, the message will remain invisible to consumers for the duration of the delay period. Then it will be consumed by a consumer after 10 seconds.
3.2 Delay Message
Instead of setting the delivery delay on an entire queue, we can set delay seconds on individual messages. Message timers will be used to allow Amazon SQS to use the message timer’s DelaySeconds value instead of the delay queue’s DelaySeconds value. Let’s see how to send a delay message below.
The SqsClient provides sendMessage API for us to send a message to the SQS queue. Firstly, we create a sample message myEvent and serialise it to JSON string:
// sample message
MyEvent myEvent = new MyEvent();
myEvent.setId(UUID.randomUUID().toString());
myEvent.setSource(Thread.currentThread().getName());
myEvent.setPayload("AWS SQS message attributes example.");
String message = null;
try {
message = objectMapper.writeValueAsString(myEvent);
} catch (JsonProcessingException e) {
logger.error(e);
}Then, we build an instance of SendMessageRequest and call sendMessage API to send the message:
SendMessageRequest.Builder builder = SendMessageRequest.builder()
.queueUrl(queueUrl)
.messageBody(message)
.delaySeconds(10); // set delay on individual message
logger.info("Sending message to queue {} with delaySeconds {}", this.queueName, 10);
this.sqsClient.sendMessage(builder.build());We can see that delaySeconds method of SendMessageRequest.Builder is called with value 10 to set the delay to 10 seconds when sending an individual message to the queue.
Note that when sending messages to a FIFO queue, DelaySeconds cannot be set per message. Only queue level delay can be set.
3.3 Verifying the Delay
Whether to use a delay queue or set delay seconds on individual message, we can verify the delay by using a consumer to consume the message. This is because for any option of delivery delay provided by Amazon SQS, any message sent to the queue remains invisible to consumers for the duration of the delay period. As we have already seen in AWS SQS Polling Example in Java, we can use long polling to receive messages from a queue.
The following code snippet shows an example of using long polling to receive messages from a queue. If there is a message available in 20 seconds, the call will returns with the message. Otherwise the call will keep waiting until 20 seconds timed out and then returns an empty list. Note that the key here is to call waitTimeSeconds method to set the waitTimeSeconds to 20 which forces the long polling when building the ReceiveMessageRequest. Please check out the javadoc of waitTimeSeconds method for more details.
// long polling and wait for waitTimeSeconds before timed out
ReceiveMessageRequest receiveMessageRequest = ReceiveMessageRequest.builder()
.queueUrl(queueUrl)
.waitTimeSeconds(20) // forces long polling
.build();
List<Message> messages = this.sqsClient.receiveMessage(receiveMessageRequest).messages();3.4 Running the Examples
For ease of dependencies management and build, we use Gradle 6.0 as the build tool. To run the delay queue example, you can run the delayQueueExample gradle task from command line as below:
gradlew delayQueueExample
The output would be:
[22:57:43,076] [INFO] aws.demo.DeliveryDelayExample:150 [main] - Create queue MyQueue with delaySeconds 10 in region ap-southeast-2 [22:57:44,312] [INFO] aws.demo.DeliveryDelayExample:158 [main] - Queue URL: https://sqs.ap-southeast-2.amazonaws.com/000000000000/MyQueue [22:57:44,327] [INFO] aws.demo.DeliveryDelayExample$MyConsumer:338 [pool-3-thread-1] - Long poll messages from MyQueue... [22:57:44,376] [INFO] aws.demo.DeliveryDelayExample$MyConsumer:350 [pool-3-thread-1] - Polling attempt #1 [22:57:47,520] [INFO] aws.demo.DeliveryDelayExample$MyProducer:296 [pool-4-thread-1] - Sending message to queue MyQueue [22:57:57,629] [INFO] aws.demo.DeliveryDelayExample$MyConsumer:354 [pool-3-thread-1] - 1 messages received. [22:57:57,630] [INFO] aws.demo.DeliveryDelayExample$MyConsumer:382 [pool-3-thread-1] - Processing message 2a494f2a-71d6-4f30-a804-979e36725037 [22:57:57,674] [INFO] aws.demo.DeliveryDelayExample$MyConsumer:392 [pool-3-thread-1] - Message processed: MyEvent=MyEvent[id='ecae4da1-8854-4cb8-b4fa-61713191744c', timeStamp=2020-09-06T12:57:47.319690200Z, source='pool-4-thread-1', payload='AWS SQS polling example.'] [22:57:57,702] [INFO] aws.demo.DeliveryDelayExample$MyConsumer:403 [pool-3-thread-1] - Deleting message 2a494f2a-71d6-4f30-a804-979e36725037 from queue: MyQueue [22:57:57,744] [INFO] aws.demo.DeliveryDelayExample:168 [main] - Delete queue MyQueue [22:57:57,852] [INFO] aws.demo.DeliveryDelayExample:173 [main] - Queue MyQueue deleted.
We can see from the output above that a delay queue MyQueue was created with delay seconds set to 10 seconds. Then the consumer was using long polling to retrieve messages from MyQueue. The producer sent a message to the queue at [22:57:47]. Then the consumer received the message after 10 seconds delay at [22:57:57].
To run the delay message example, you can run the delayMessageExample gradle task from command line as below:
gradlew delayMessageExample
The output would be:
[23:03:00,247] [INFO] aws.demo.DeliveryDelayExample:150 [main] - Create queue MyQueue with delaySeconds 0 in region ap-southeast-2 [23:03:01,010] [INFO] aws.demo.DeliveryDelayExample:158 [main] - Queue URL: https://sqs.ap-southeast-2.amazonaws.com/00000000000/MyQueue [23:03:01,013] [INFO] aws.demo.DeliveryDelayExample$MyConsumer:338 [pool-3-thread-1] - Long poll messages from MyQueue... [23:03:01,067] [INFO] aws.demo.DeliveryDelayExample$MyConsumer:350 [pool-3-thread-1] - Polling attempt #1 [23:03:04,227] [INFO] aws.demo.DeliveryDelayExample$MyProducer:296 [pool-4-thread-1] - Sending message to queue MyQueue [23:03:14,290] [INFO] aws.demo.DeliveryDelayExample$MyConsumer:354 [pool-3-thread-1] - 1 messages received. [23:03:14,291] [INFO] aws.demo.DeliveryDelayExample$MyConsumer:382 [pool-3-thread-1] - Processing message 8c379d6f-088b-4852-ae25-fc4d016d6ef1 [23:03:14,324] [INFO] aws.demo.DeliveryDelayExample$MyConsumer:392 [pool-3-thread-1] - Message processed: MyEvent=MyEvent[id='193c6028-879d-432b-8fec-dcc2da64d314', timeStamp=2020-09-06T13:03:04.015762600Z, source='pool-4-thread-1', payload='AWS SQS polling example.'] [23:03:14,352] [INFO] aws.demo.DeliveryDelayExample$MyConsumer:403 [pool-3-thread-1] - Deleting message 8c379d6f-088b-4852-ae25-fc4d016d6ef1 from queue: MyQueue [23:03:14,393] [INFO] aws.demo.DeliveryDelayExample:168 [main] - Delete queue MyQueue [23:03:14,505] [INFO] aws.demo.DeliveryDelayExample:173 [main] - Queue MyQueue deleted.
We can see from the output above that a normal standard queue MyQueue was created without a delay (the delay seconds was set to 0 second). Then the consumer was using long polling to retrieve messages from MyQueue. The producer sent a message to the queue at [23:03:04]. Then the consumer received the message after 10 seconds delay at [23:03:14].
4. Download the Example Source Code
You can download the full source code of this example here: AWS SQS Delay Queue and Delay Message Example in Java
