WO2018167755A2

WO2018167755A2 - Method and system for creating and maintaining quality in email address list

Info

Publication number: WO2018167755A2
Application number: PCT/IB2018/054605
Authority: WO
Inventors: Dominique VERVIN; Gerana ESPINOZA; Joshua BENAVIDES
Original assignee: Quantic Vision
Priority date: 2018-06-22
Filing date: 2018-06-22
Publication date: 2018-09-20
Also published as: WO2018167755A3

Abstract

According to embodiments of the present invention, methods, systems, computer program and therefore integration, provides monitoring of email flow through the system, real time control of the quality of recipient's email addresses, classification of known recipient's email addresses in a database, handling black-list, white-list and "grey-list", checking of unknown recipient's email addresses through suitable mechanism and method before blocking the email or being handled out to a third party sending system, receiving and collecting bounce-back messages in order to feed-back and update the system, in real time and continuously.

Description

METHOD AND SYSTEM FOR CREATING AND MAINTAINING QUALITY IN EMAIL

ADDRESS LIST

OBJECT OF THE INVENTION

The present invention relates generally to data processing systems and communication, more particularly, and not by way of any limitation, the present disclosure relates to electronic mail, and even more particularly to the control and maintenance of quality in an electronic mail addresses list or database.

BACKGROUND OF THE INVENTION

Spam is unsolicited electronic mail communications (email). Spam is considered annoying and a time waster by many, in addition, spam may be considered offensive to some, spam may be a danger to the gullible with offers of get-rich quick schemes or quasi-legal offers.

Spam costs consumers in terms of lost productivity and fraud. Spam costs Internet Service Providers (ISPs) money at least because ISPs have had to add extra capacity to cope with the deluge of spam, and have had to divert employees' time to dealing with the spam.

Spam was considered such a serious problem that as early as 2003 the United States Congress considered and passed anti-spam legislation. See CAN-SPAM Act, Controlling the Assault of Non-Solicited Pornography and Marketing Act of 2003, 15 U.S.C. Chapter 103, §§7701 -7713. The CAN-SPAM Act became law on Jan. 1 , 2004. Violation of the law may result in a fine of $1 1 ,000 for each offense, which means $1 1 ,000 for each unsolicited email sent to a recipient. ISPs around the country have already successfully sued spammers for a lot of millions of dollars under this law.

Experts have designed ways to filter out spam. A typical spam filter may look at a long list of criteria to judge whether an email received from a sender is spam. The spam filter may assign a point value for each criterion met by the mail. If the email's "spam score" exceeds a threshold, generally, the spam filter determines the email is spam. Based on that determination, action may be taken depending on whether the recipient is a person or an ISP. A person's email program typically contains a spam filter or "junk" evaluator. Emails the person receives that are determined to be spam may be sent to a junk.

Some spam emails do not reach the intended recipient, but instead are blocked by the spam filter(s) used by the recipient's ISP, as in Chadwick et al. US 2008/0133682 A1 , and in Wood, US 2010/0153394 A1 , incorporated herein by reference.

An ISP that repeatedly receives spam from a sender of email may refuse to transmit additional emails from that sender or either "mark" the email as "SPAM" in its "X- headers". X-Headers are tags handled within the "non-payload" part of the email altogether with other tags like the sender address (From:), destination address (To:) and other tags. An ISP may also tag the Internet Protocol (IP) address or the FQDN (Full Qualified Domain Name) used by the sender in its own System and thereafter block any mail sent from this IP address or FQDN or either mark as SPAM all email sent from this specific IP address.

One of the most common criteria used to elevate the "spam score" associated with IP address or FQDN is the "bounce rate" of email sent from this IP address or FQDN. Bounce or bounce-back occurs when an email is undeliverable for whatever reason, as for example, bad or incorrect address of the destination domain email server (SMTP Server), destination user not existing, destination mail box full, etc. It exists a lot of reason for email bounces (in the tens), some are soft because are typically temporary (example because network congestion) others are hard because definitive and permanent (example non-existent FQDN).

Nevertheless, some company or organization may have legitimate, honest and fair reasons to send big quantity of email, whether it be in batch campaign (a lot of emails sent in one time) or in an "on line" form (in the form of a continuous flow). Among other we can list for example big organizations as banks (sending large amount of statement of account or transactions notifications), insurance companies, credit card processing platforms (who may even use this way of notification as a mean of fraud prevention), we can also cite email (marketing) service providers (ESP) companies, which are offering email marketing campaign or bulk mail services to its customers (also referred to as users) to send for example legitimate information like newsletters.

Nevertheless, email destination addresses of these legitimate recipients of the previously quoted legitimate senders (as banks for example) are not always accurate. In the real life, the rate of mistakes in legitimate recipients' email addresses of organization such as big banks or insurance companies (among other organizations) could be relatively high for a lot some reasons. People are using workplace email address and they change of work and not always update and maintain their new email address accurately. Sometimes (very frequently) these email addresses are acquired through a human intermediate process implying filling a handwritten form or even through a phone conversation, or both, and then a third person staff of the organization may oversee the data entry of the email address. These processes are prone to misspelling and any kind of mistake.

As a result, the recipient list or database can be stored with a high rate of erroneous email addresses. In that case an even very legitimate organization, processing a fair sending of a big quantity of emails can generate a high rate of email bounce-back and his IP address or FQDN can be scored and classified as a spam originator. DESCRIPTION OF THE INVENTION

The object of invention is carried out implementing the system and method of the claims. DESCRIPTION OF THE DRAWINGS

To complement the description being made and in order to aid towards a better understanding of the characteristics of the invention, in accordance with a preferred example of practical embodiment thereof, a set of drawings is attached as an integral part of said description wherein, with illustrative and non-limiting character, the following has been represented: Figure 1 .- Shows a block diagram of systems according to an embodiment of the present invention, in a real "Internet and Wan ecosystem" including the topology of connections with third party sending systems, recipients SMTP servers and reception of bounce messages.

Figure 2.- Shows a block diagram representing the flow and processing of emails inside an embodiment of the invention, including the look-up of black-list, white-list, grey-list, the suitable processing test and check on unknown recipient email addresses.

Figure 3.- Shows a block diagram representing the flow and processing of bounce messages inside an embodiment of the invention, including the automatic maintenance of black-list, and domain checker.

Figure 4.- Shows an example table of bounce-back table with an example of soft / hard bounce classification. Figure 5.- Shows an example of table Time to Live parameters maintained in a Black- List and/or White-List and/or Domains database.

PREFERRED EMBODIMENT OF THE INVENTION As per figure 1 , where a block diagram of systems according to an embodiment of the present invention is depicted, in a real "Internet and Wan ecosystem" including the topology of connections with third party sending systems, recipients SMTP servers and reception of bounce messages; It Illustrates the concept of the system according to a preferred embodiment of the present invention: As illustrated, an embodiment of the invention will be fed by a source (1 ) of emails, which can be a continuous flow of email which is sometimes called "online input" or a batch file sending of a big or small quantity of email to be processed and sent by the embodiment of the invention. In both case continuous flow or batch file input, this source (1 ) can be embodied by a API connector as for example, but not only, a Webservice (SOAP, REST, HTTP GET or POST, or other method), or a SMTP connector, or another suitable connector, interface, method or way. This source (1 ) can also be embodied by a web interface to allow user, human or not, to load a file in a structured format or not, like a CSV file or any other kind of file in a suitable form, the source (1 ) can also be embodied in an automatic or not automatic mechanism of importation of files in a suitable form.

In a preferred embodiment of the invention a single server (2) is deployed, nonetheless this server (2) can also be cluster of servers, either physical or virtually deployed, distributed or not. A preferred embodiment of the invention embraces a database (3) or a formatted list of email addresses and associated time stamped events and comprising: black-list, white-list and grey-list of email addresses.

The server (2) is further configured for checking of the validity of email addresses or part of email addresses like a NS-Lookup (4) mechanism or sub-system and a SMTP- Hello (5) mechanism or sub-system.

The server (2) is connected to and / or integrated with Communications Networks (8) which support diverse protocols of exchange of email like, but not only, SMTP or TCP/IP protocols, based on physical or logical supports like Internet or Ethernet networks or other means of transport of information.

As per Figure 1 , the Communications Networks (8) are labelled as "Internet", only as an example and because it is the most common communication network used today. The server is also connected to a Third Party Sending System (6) where the flow of validated or not validated emails are transmitted to. The connection between the server (2) and the Third Party Sending System (6) can be based, but not only, on protocols like SMTP, HTTP, among others.

The flow of email (7) processed is then transmitted and routed through the Third Party Sending System (6) and the Communication Networks (8) to destination or intermediate SMTP Servers (9) through appropriate routing and appropriate packet switching methods and systems supported by, or embedded in, the Communication Networks (8). This appropriate routing and packet switching methods includes, but not only, Domain Name Systems (DNS) lookup and handling, to determine Internet Protocol address from Fully Qualified Domain Name (FQDN) and through MX Records in DNS System the IP address of Destination SMTP Servers (9).

A Bounce (10) or Bounce-back message (10) is generated upon either delivery error or delay in the Communications Networks (8), including, but not only, at DNS Level, or at switching and routing level, or at SMTP Server (9) level, or at email recipient device level through the SMTP Server (9). Bounce Messages (10) are routed to the Server (2) through the Communication Networks (8). Thanks to a header record or field in the email which generated the error, the "return- path" header field, which one was correctly and appropriately setup in the header of the email by the embodiment of the invention, the Server (2), the bounce messages (10) are correctly and appropriately routed and delivered to the server (2). Figure 2 is a block diagram representing the flow and processing of emails inside an embodiment of the invention, the Server (2). Including the look-up of black-list (12), white-list (13), grey-list (14), and the suitable processing test and check on unknown recipient email addresses; It Illustrates the flow of emails within an embodiment of the invention, the Server (2) and details the processing of such flow to guaranty the quality of email addresses of email sent at the end of the process to the Third-Party Sender System (6).

As a first step, the email address of a processed email within the flow is checked again an internal database of blacklist (12). This database includes the results of previous bounce or bounce-messages received related to mails previously processed and sent to this recipient email address through the preferred embodiment of the invention, the server (2). If the result of such query or internal lookup in the blacklist database is positive, then it means that it still exists a good reason within the communication networks (8) or either the destination SMTP server (9) or either in the recipient device through the destination SMTP Server (9) to generate an error, hence a further new bounce message (7). As a result, the email message is not further processed (13), the flow of this specific email is stopped, and this first time-stamped event is logged in the Server (2) databases (3) for further reporting or any kind future use of this information. If the result of such query or internal lookup in the blacklist database is negative, then the process of the email within the flow go to the next step: the white-list lookup (13).

In a second step, the email address of a processed email within the flow is checked again an internal database of white-list (13). This database includes the results of previously processed emails successfully sent to this recipient email address through the preferred embodiment of the invention, the server (2) without having received any bounce message (7). If the result of such query or internal lookup in the whitelist database is positive, this means that there is no good reason to anticipate an error occurring within the communication networks (8) or either the destination SMTP server (9) or either in the recipient device through the destination SMTP Server (9) to generate an error, hence a further new bounce message (7). As a result, the email message is processed and transmitted to the DKIM Processing System (1 1 ), to follow the flow of this specific email and further be transmitted to the Third Party Sending System (6), and this second time-stamped event is logged and stored in the Server (2) databases (3) for further reporting or any kind future use of this information. If the result of such query or internal lookup in the whitelist database is negative, then the recipient email address is considered as new and unknown and recorded in the greylist database (14) and the process of this specific email continue within the flow going to the next step: the domain lookup (15).

In a third step, the email address of a processed email within the flow is checked again an internal database of domain (Fully Qualified Domain Name or FQDN) (15). This database includes the results of previously processed emails sent to this recipient FQDN through the preferred embodiment of the invention, the server (2). This database includes the result of previously queried NS Lookup for FQDN of the email address. If the result of such query or internal lookup in the FQDN or Domain database (15) give a "Domain-OK" result then, as a result, the email message is processed and transmitted to the DKIM Processing System (1 1 ), to follow the flow of this specific email and this third time-stamped event is logged in the Server (2) database (3) for further reporting or any kind future use of this information. If the result of such query or internal lookup in the FQDN or Domain database give a negative result, then the recipient email address domain or FQDN is considered as new and unknown and the process of this specific email continue within the flow going to the next step: The Name Server (NS) lookup (16).

Since the FQDN of the email address is new and, the Server (2) performs a DNS or NS Lookup on the FQDN with the goal of looking for a MX record, which is returning the IP address or addresses of the SMTP server or servers (9) of the recipient of the email. If this lookup fail, then it means that either the domain (FQDN) does not exist in the international DNS system or either that his SMTP server (9) is not correctly defined (MX record), hence not accessible through appropriate routing and packet switching within the communication networks (8), hence would generate an error at Communication Networks (8) level or either at SMTP Server (9) level and hence a bounce-back message (10), as a result, the process of this specific email is stopped (18), the Server (2) databases (3), (12), (13), (14) and (15) are updated with a fourth time-stamped event.

As an additional step, in case of success of NS Lookup a SMTP Hello (17) is performed. A SMTP Hello (17) consists of opening and initiating a SMTP session with the recipient destination SMTP server (9), based on the IP address delivered by the NS Lookup (16) (MX record). This initiated SMTP session certify that the recipient destination SMTP server is up and running, hence that sending the email will not generate an error of access at SMTP server (9) level, hence will not generate a bounce message. If the SMTP Hello (17) end up with success then, as a result, the email message is processed and transmitted to the DKIM Processing System (1 1 ), to follow the flow of this specific email and this time-stamped event is logged in the Server (2) databases (3) for further reporting or any kind future use of this information. If the SMTP Hello (17) end up with a failure, then, as a result, the process of this specific email is stopped (18), the Server (2) databases (3), (12), (13), (14), (15) and (18) are updated with a time-stamped event for further reporting or any kind future use of this information.

The DKIM Processing System (1 1 ) is an optional final step of the internal processing flow of email, and an embodiment of the invention may or may not include a DKIM processing System (1 1 ). Domain Key Identified Mail is a mechanism of cryptographic authentication defined by IETF (Internet Engineering Task Force) RFC (Request for Comment) number 6376, which precise how to include within email header an encryption private key that should match a public encryption key accessible as a public DNS record of the FQDN. After this optional ultimate step of the preferred embodiment of the invention, the email is transmitted to a third party sending system (6). The Third Party Sending System (6) may include a DKIM Processing System by his own hence the optional nature of the presence of a DKIM Processing System (1 1 ) within the preferred embodiment of the present invention.

Figure 3 depicts a block diagram representing the flow and processing of bounce messages (10) inside an embodiment of the invention, including the automatic maintenance of black-list databases (12,19), and domain checker (15). It Illustrates how bounce messages (10) are received and collected by the preferred embodiment of the invention, the Server (2) and processed within. Upon reception of a bounce message (10), internal databases (3,12,13, 14, 15, 18,19) are updated with time- stamped event through and thanks to appropriated and suitable algorithms.

As per figure 4, where an exemplary table of bounce-back table with an example of soft / hard bounce classification is shown, it can be appreciated the handling and storing the time stamped events associated with bounce and bounce back messages altogether with recipient's email addresses and recipient's domains or FQDN. The column type of bounce is shown as an example only and may or not may be different, may or not may be used in an embodiment of the invention. The columns Bounce Code and Description shown as an example only and may or not may be different, may or not may be used in an embodiment of the invention, may or not may be used in the algorithm of the computer programs of the invention.

Figure 5 depicts an exemplary table Time to Live parameters maintained in a Black-List and / or White-List and / or Domain database. It illustrates how internally the preferred embodiment of the invention is handling and storing the time stamped events associated with bounce and bounce back messages altogether with recipient's email addresses and recipient's domains or FQDN. The columns are shown as an example only and may or not may be different, may or not may be used in an embodiment of the invention, may or not may be used in the algorithm of the computer programs of the invention, as well as the values which are shown in as an example only and may or not may be different, may or not may be used in an embodiment of the invention, may or not may be used in the algorithm of the computer programs of the invention.

Claims

A system for creating and maintaining quality in email address list, the system comprising:

a. a source of emails,

b. at least one server accessible by a Communications Networks which support exchange of email and connected to the source of emails, c. a series of databases,

wherein the databases comprise a black list, a white list and a graylist of email addresses.

The system of claim 1 wherein the server is configured for checking of the validity of email addresses or part of email addresses by means of a NS- Lookup mechanism or sub-system and a SMTP-Hello mechanism or subsystem.

3. The system of claim 1 wherein the server is configured to read a header record or field in the email.

A method for creating and maintaining quality in email address list, the method comprising:

a. checking an email address of a processed email against an internal database of blacklist which includes the results of previous bounce or bounce-messages received related to mails previously processed and sent to this recipient email address,

b. non-processing the email when the email address is the blacklist database, and

c. generating a first time-stamped event, and

d. saving the time-stamped event in the server databases.

The method of claim 4 wherein the email address is not in the blacklist database, the method further comprising:

a. checking the email is checked again an internal database of white-list which includes the results of previously processed emails successfully sent to this recipient email address without having received any bounce message,

b. If the result of such query or internal lookup in the whitelist database is positive the email message is processed and transmitted to an DKIM Processing System,

c. transmitting the email to the Third Party Sending System, and d. generating a second time-stamped event, and

e. storing the second time-stamped event in the server. 6. The method of claim 5 further comprising:

a. checking the email address against an internal database of domain (Fully Qualified Domain Name or FQDN) which includes the results of previously processed emails sent to this recipient,

b. transmitting the email to an DKIM Processing System when the of the previous step is "Domain-OK" result then,

c. generating a third time-stamped event, and

d. storing the third time-stamped event in the server.

The method of claim 5 further comprising a Name Server (NS) lookup looking for a MX record, which is returning the IP address or addresses of the SMTP server or servers of the recipient of the email.

8. The method of claim 7 further comprising a SMTP Hello consisting of opening and initiating a SMTP session with the recipient destination SMTP server, based on the IP address delivered by the NS Lookup (16) (MX record).