Ensuring Perfect Email Testing: Best Practices for Deliverability and Cross-Client Compatibility

Emails are the one of best ways to bring users back to the application. It’s a way of communication between products and users. Businesses like e-commerce and search sites want their marketing emails to land perfectly in the inboxes of users. The emails should be displayed perfectly across different devices and email clients. So for this purpose, thorough testing is required. 

Nowadays, the majority of the products engage their users by sending them emails frequently. They want their users to get back to using their products. Also, users create alerts e.g. ‘job alerts’ on job search platforms and ‘items alerts’ on e-commerce apps for their convenience.

LinkedIn allows users to create job alerts which help users to get the relevant job. AliExpress sends emails to users if they have price drops for the items that are added to the user's cart. So, they think about the items again and they may buy them.

Generally speaking, emails are the best way to engage users of a product in the easiest way. But it doesn't mean that they are easy to develop and test.

In this blog, we’ll talk about the process of email testing. It will include their creation, triggering, UI testing, data testing, and compatibility tests across different devices, and email clients.

How to Generate Emails for Testing

We have multiple techniques to generate emails for testing purposes

Manual Testing:

• For manual testing, create test emails or test users on different platforms e.g. Yahoo, and Gmail.
• Use Mock tools like Faker and Mockaroo to add some data to email templates 
• Some companies have their internal consoles. Using them, they generate emails with Mock data for testing purposes
• Some companies use endpoints/URLs for email creation. They receive email in their inboxes once the URL is hit in a browser.
• Some companies use API testing tools e.g. Insomnia to generate emails for testing purposes.

Automation Testing:

• For automation testing, companies are using Postmark, Mailosaur, or SendGrid depending upon the project need.
• Mailosaur is mostly used and designed primarily by QAs and developers because it provides both testing and validation services.
• Postmark is used for transactional emails.
• SendGrid offers marketing email services.
• These tools should use CI/CD pipelines to trigger testing emails with the deployment of every build.

How to Trigger Emails for Testing

In the production environment, emails e.g. subscription emails and marketing campaigns are triggered when their respective scheduled jobs run. In testing environments, they don’t trigger as expected, so in that case, we trigger them forcefully.

• Consoles are GUI tools, used to trigger emails forcefully. It can be Mock data or can be taken from the previous data populated in the database. 
• Normally the QAs don’t have access to test environment databases. In that case, they can ask developers to run the commands for testing emails. 
• Webhooks can be used to simulate real-world scenarios and events like form submission, payment confirmation, account registration, or password resets. You can manually trigger a webhook using API or an internal tool. They can be triggered automatically from your system based on an event. 
• We can also trigger testing emails, using third-party services e.g. Mailosaur, to simulate events and trigger webhooks.

How to Test Emails UI

Responsive email design really matters for clients and customers. The clients want their email to fit completely on different devices and email clients. Proper testing is needed to ensure they look perfect:

• Cross-client testing can be done using tools like Email on Acid. Email on Acid provides previews of the test email across devices and clients. It provides web, desktop, and mobile previews of a test email. It creates templates on different devices and email clients. It also creates emails in dark mode.
• Due to limited budget, some teams may not go for cross-client tools in that case they can test emails on physical machines (Mac, Windows) and mobile phones of QAs/Devs, to ensure the emails are adapting to different resolutions.
• It’s very important to test emails in landscape mode and dark mode because many users prefer to view emails in landscape mode and dark mode. QAs must ensure that UI is not broken in landscape mode and dark mode. Buttons, texts, and images should be clearly visible in landscape mode and dark mode. 
• For visual comparisons, tools like Percy can be used to detect unintended visual changes. Percy is designed for web applications but it can be used to check the UI of emails. 

Follow the below steps to test the UI of an email.

To test email UI, 

• Generate the previews of an email using Email on Acid. 
• Use Cypress, WDIO, or any tool, to capture the previews as screenshots.
• Integrate Percy in Cypress or WDIO to compare these snapshots and identify any unintended visual changes.

How to Test Email Data

In order to make sure, the email is relevant, accurate, and secure. It’s very crucial to test the email data. Besides email subject and placeholder, there are some other checks that can help QAs to test email data.

Steps to Test Email Data:

1. Generate an email for a test user
2. Force triggered it to test the user
3. Open your inbox and check the email subject and preheader
4. Open the email and verify values like user ID, msg-id, etc, this can be done by copying the URL of an element of the email. Sometimes the URLs are encoded, you can decode them using free online tools like base64decode  
5. Check for other details in the email like order number, booking dates, etc
6. Check the redirection of the call-to-actions (CTAs).
7. Check for nulls in email,  special characters, or long names
8. Check ads support in email.
9. Check for sensitive data. 

The email should have accurate information, it should support accessibility. We should check the usability of emails as well.

How to Test Emails on Different Clients

The UI of an email can vary on different devices and clients. QAs should check the UI of an email on various clients and platforms. 

• A QA can manually check an email on different devices and platforms like Yahoo, Gmail, Apple Mail, and Outlook. But it’s very time-consuming because you will have to send the emails to all test users logged into different devices and platforms. Also, some companies have limited devices.
• A QA can use Email on Acid to generate all possible previews of an email. Email on Acid supports multiple devices and clients. You can either trigger direct email to Email on Acid. Sometimes, you will forward an email but forwarding breaks email.  

How to Generate Previews on Email on Acid

Below are steps to generate previews on Email on Acid

• Open email client e.g. Gmail
• Click on the three vertical dots
• Click show original
• Open any encode/decode email site e.g. https://www.webatic.com/quoted-printable-convertor
• Copy everything in the "original" when it opens in a new tab
• Paste into the encoded (bottom) box at the encode/decode email site
• Click "decode"
• Copy decoded message
• Go to Email on Acid
• Go to Email Testing
• Click "yes" you have HTML
• Paste decoded text
• Remove everything above <!DOCTYPE html>
• Check "All Previews" are selected in Email Clients to test
• Run email test

You will get several previews covering web, desktop, and mobile platforms. 

Beyond Testing: Optimizing Emails

Once you are done with thorough testing of email, you will ensure the emails are well optimized for performance:

• Analyze open rate and click-through rate (CTR) using analytics tools
• Regularly update email templates. Monitor the control vs XP user groups. Analyze their interactions with email e.g. CTA clicks.   

Email Testing: Staging vs Production testing

As discussed above, when we are testing emails on staging environments, we don’t wait to receive them. We trigger them forcefully via internal console or endpoints/URLs or using API testing tools. But the data in emails are Mock data added in an email template via an internal console or Mock sender. But to verify that data is accurate, we don’t do a testing iteration on the production environment as well. In the production environment, we don’t want forced sends but we wait to receive the real sends of email. 

Those real sends are triggered when a backend job runs. The data in the production emails are real data because they come up from live searches. So, the data testing of an email is very important in a production environment and should not be ignored.

Email Testing: Email Delivery

Email delivery is something that could give testers positive feedback from clients and customers. Testers should ensure that emails are received in customers' inboxes on time and there should not be any chance of delay. An email should be landed in the inbox instead of the spam folder.

We can use the tool Mailosaur to check if an email lands in the spam folder or inbox.

Testers should also monitor the factors that influence email delivery, such as authentication, quality of the content, sender domain reputation, and some email metrics. Testers should address them to developers.

1. Proper authentication is required to prevent emails from being flagged as suspicious.
2. Avoid using words that reduce the quality of content in emails. The email content should be clean and must have a professional layout.
3. Invest in famous and trustable sender domains. 
4. We should proactively track email metrics like delivery rates, spam complaint rates, and bounce rates to identify, investigate, and resolve issues.

Conclusion

Thorough email testing requires creativity, precision, and collaboration. You proactively do email creation, and triggering. Your testing ensures email is received in user inboxes without any delay. You do UI and data testing to ensure the emails are not broken and have the correct information. You do cross-client and platform testing for compatibility to ensure every email fits on user devices and email clients.  

Your testing adds value and strengthens your product. Remember, you should do email testing proactively, with creativity, precision, and collaboration. These will help you to get trust and confidence from stakeholders and customers.