Justuno Retry Queue
It happens more than end users realize. Email service providers (ESPs), both free or paid, can go down. This poses a significant threat to businesses of all sizes and jeopardizes the dependability of your marketing communication channels. But even more than that, losing these email addresses can affect your company’s bottom line.
When your livelihood depends on the ability to send and receive emails, you need a dependable solution. A little over eight months ago, the development team here at Justuno set out to make life a little easier for our clients and solve this problem.
We began installing our solution to this problem, the Retry Queue, on several of our larger ESP integrations and have expanded its capabilities. Currently the retry queue is available for our clients who use our eight most utilized ESP integrations: Mailchimp, Klaviyo, Bronto, Responsys, Eloqua, Omnisend, ReSci, and Salesforce Marketing Cloud. We’re planning to integrate it into more ESP partners in the coming months so all Justuno users will have the same email submission protection.
We have had multiple large clients who reaped the unexpected benefits of the Retry Queue when their ESP’s went down without warning. The queue immediately flooded with thousands of emails that would have otherwise been lost. By the next day the ESP was up and running again, and the Retry Queue sat empty, with all the missing data submitted, as if there was never an issue at all.
How it works
When an email fails to pass through an ESP, it lands in the Retry Queue database. At increasing increments, the database attempts to resend the email. The retry is initiated at 15 minutes, an hour, four hours, one day, and then two days later. If the retry is successful at any point, that email is automatically removed from the queue.
If an email makes it through the entire cycle and the retry is not successful, it is removed at that point.
How it's triggered
The Retry Queue is triggered any time there is a failure or a bad response from the ESP when Justuno submits an email on your behalf. This happens for a variety of reasons, some of which are easily remedied by submitting again later, through the retry queue.
Some of the responses that can trigger the queue include:
- The ESP is down
- You’ve hit a rate limit
- Your ESP account credentials changed
- Other miscellaneous technical issues
There are also miscellaneous user errors that can cause an email to be placed in the retry queue, including not filling in the required fields, or submitting an invalid email address. This often comes in handy, too.
Your organization likely counts on the reliability of email communication. For the most part it is an extremely dependable means to communicate digitally. But like anything else online, it is susceptible to going down from time to time.
The Retry Queue is an excellent way to ensure that your business is collecting all the emails possible and not losing access to potential customers. Our continual retries for up to two days give the ESP time to resolve issues on their end or clients time to update their credentials.
The Retry Queue is built on a cockroach database table and can hold billions of records. For our intended use, there is no limit to the number of emails and records it can hold for our clients. In other words, this can benefit a client no matter how many emails may be moving through the system during an outage.
Guard yourself from costly user mistakes
The Retry Queue has an additional benefit besides that of picking up when a client’s ESP goes down. It can also help with issues such as credentials. Occasionally clients might regenerate a token or API key by accident, or reset account or sub-account permissions. There are any number of unintentional ways for a user to invalidate their ESP credentials, when submitting an email or data.
The Retry Queue saves these emails as well and gives the client time to sort out their credential issues and put new, or valid credentials in place. Once this has occurred, the previous emails that were entered can be submitted. Even when it’s not related to an ESP going down, the Retry Queue can help.
The Retry Queue is seamless for the end user
The end user may be aware of the Retry Queue’s existence, but there is no accessible interface for them to access. They do not have to worry about selecting the right settings or otherwise making sure that it is operational. Every important action takes place behind the scenes, with no technical knowledge necessary on the clients’ behalf.
In fact, unless they receive a prompt from the ESP notifying them of an outage, they’ll never know when the Retry Queue is being utilized. In other words, it’s a seamless benefit that requires no user action.
The queue continues to use the newest credentials provided to an account and allows the user to update any credentials they wish, in order pass the entries in the queue. It is a script written in node that queries the cockroach database retry queue table for records and then sends them through the backend API to submit the emails again. This results in a submission that is indistinguishable from a live site submission.
Providing a high level of value to our clients
While ESP outages are infrequent, the impact can be costly. Even when the ESP only goes down for a few moments, it can be potentially damaging, depending on a client’s needs, and the timing of an outage. This provides a way for us to help them mitigate any potential problems without a loss of customer submission data.
The Retry Queue has provided a high level of value to some of Justuno’s busiest clients over the last eight months, and it is something that we are looking to add in to all of our integrations in the future.