Saturday, April 10, 2021

Handling Bounced Email

Using PHPMailer-BMH and AuthSMTP

Probably the best way to handle bounced emails when sending mass mailings is to use a Variable Envelope Return Path (VERP) system. Variable Envelope Return Path (VERP) is a technique used by some electronic mailing list software to enable automatic detection and removal of undeliverable e-mail addresses. It works by using a different return path (also called "envelope sender") for each recipient of a message. The basic idea is to include information about the recipient in the return path of the email going out, so that if it does bounce, the original recipient information is readily available without having to analyze the returned mail to get this information.  For example, if the recipient email address was taken from a database and the index of the address was 1234, the return path for the email could be something like user-1234-bounce@mydomain.com. All you would have to do is have a method to analyze the return path to know that the email address at index 1234 of the database was bounced. While this system works great, there are some instances in the real world where it may not be applicable.

This method assumes:

  • you can filter the returned mail to an address like bounce@mydomain.com
  • you can alter the return path of the original email you are sending
  • you have, or can set up, a catch-all email address

If you are running your own server, these requirements are not a problem. If using paid services, there might by some issues as I have found. When working on a recent project using AuthSMTP.com to send out mass mailings, I elected to use the PHPMailer-BMH toolkit to manage bounced emails. As presented on the PHPMailer website:

Bounce Mail Handler (BMH) is a PHP toolkit (class and rules) to help webmasters handle bounce-back mails in standard DSN (Delivery Status Notification, RFC-1894). When the DSN is mal-formed or missing, BMH audits the message body for RFC codes.

Bounce Mail Handler is a complete mailbox management tool that can also delete messages globally across all mailboxes based on user supplied date. Global deleting excludes any 'sent' mailboxes. Global delete is optional and performed before the bounce-back processing.
 

Though the class is quite complete, there were some problems out-of-the-box. These are discussed below.

First off, by default, you cannot change the return-path of emails sent out by AuthSMTP.com. If you set a different return path from the path in your From: address, AuthSMTP will simply change it to your From: address. This doesn't work very well if trying to set up VERP system. if fails right out of the gate. With some testing I found that you can set up a different return-path if you set up AuthSMTP to accept all email addresses from your domain instead of the default single address. Easy enough to do. But doesn't matter. I couldn't filter returned messages and could not set up a catch-all email address with the hosting company being used.

The easiest way to deal with the AuthSMTP limitations was to use a return-path of: no-reply@mydomain.com. Easy to set up in PHPMailer for both the From and Return-Path addresses. Easy to set up in AuthSMTP. And just let PHPMailer-BMH handle all the filtering.

There were a few issues with the PHPMailer-BMH toolkit. Lots of PHP errors when first run due to some coding errors in the toolkit. I have fixed those. The toolkit had also been stripped down from the original authors version, I believe. The original email header, which is normally returned with bounced emails, was stripped from the code. I altered the code slightly to allow access to this header information. The biggest change I made to the toolkit was to add the ability to identify a custom header and return that information to the handling procedure. I could now create a custom header in PHPMailer and retreive it in PHPMailer-BMH. The header I needed to create would simply identify the newsletter ID as X-Mailing-ID:.

The lines I added to the toolkit are:

<?php
...
/*
* Identifies whether a custom header tag has been added to original message header
*/
public $isCustomHeader = false;

/*
* if $isCustomHeader=true then look for this header tag in returned headers
*/
public $customHeader = '';
...
?>

These are easily accessed in your PHP code.

<?php
include>(_PATH_BMH . 'class.phpmailer-bmh.php');

...
// testing examples
$bmh = new BounceMailHandler();
...
$bmh->isCustomHeader = true; // false is default, no need to specify
$bmh->customHeader = 'X-Mailing-ID'; // blank is default, no need to specify
...
?>

 And just set all your other parameters are needed. The example callback functions have also been changed to allow access to the full original headers and the custom header.

To make all this work, I also needed to turn on php.imap in the php.ini file as this toolkit does not use PHPMailer to handle the mail post office functions. It uses the PHP built-in IMAP functions, which may have to be turned on in your version.

So handling bounced emails is now quite simple and foolproof. Follow these steps:

  • set up on email address of no-reply@yourdomain.com with your hosting company
  • set up an email address of no-reply@yourdomain.com with AuthSMTP.com (assumes you have an account with them)
  • install PHPMailer-BMH and use custom header to retrieve bounced email address information

Hope you find this useful 


 

delicious digg facebook stumble twitter myspace linkedin technorati reddit google springpad blogger | addthis Share More...

we are listening

kidmoses said:

Sunday, August 28, 2021

Not really sure what you want to parse or how you want to parse the messages, but it should be possible. You may have to rewrite some of the code. It's been awhile since I looked at this code, so you are on your own with working out a solution.

Thomas said:

Thursday, August 25, 2021

Is it possible to use PHPMailer-BMH to parse abuse messages?

pre-sales.ch said:

Sunday, August 14, 2021

The "isParameter" function did not catch the key word "delivery-status" in the bounced message I received from a particular server although it is in it. So I added: OR (stripos($structure_dump, 'delivery-status')) in the code below if ($structure->type == 1 && $structure->ifsubtype && strtolower($structure->subtype) == 'report' && ($structure->ifparameters && $this->isParameter($structure->parameters,'report-type','delivery-status') OR (stripos($structure_dump, 'delivery-status')) )){ with: ob_start(); var_dump($structure); $structure_dump = ob_get_clean(); which drops the structure in the variable $structure_dump and make it easier to look for the key-word "delivery-status"...

Arif said:

Sunday, March 27, 2021

All I see the toolkit reads a mailbox. Is it possible if I filter the mail and pipe it to the script so that the bounce can be detected almost in real time ? Is this a good way or I should try parsing the mailbox instead ? Thanks a lot.

Cosmos said:

Tuesday, March 15, 2021

Hi Kidmoses, i test your script and looks prety nice, but i have a question, all messages have a blank custom header, so i don´t know what´s wrong with mi code. Can you send me an example code of header setup to use when i send mail from my php script. Now i have this headers: $headers = "Content-Type: text/html; charset=iso-8859-1 \r\n"; $headers .= "MIME-Version: 1.0 \r\n"; $headers .= "Content-Transfer-Encoding: 7bit \r\n"; $headers .= "From: $nombre <$email> \r\n"; $headers .= "Reply-To: $email\r\n"; thanks for your help.

Boopathy said:

Sunday, December 19, 2020

Hi

I have a problem in recieving the bounced emails with the original message i sent. It says only the message  failed Undeliverable email address. I need the original message body in the same  message? Can any help me to get bounced emails with original message i sent?

anonymous said:

Tuesday, October 26, 2020

kobe bryan [url=http://www.suprashoesbox.com/]supra shoes[/url] ters for about eight minutes said.[url=http://www.kobebryantzone.com/]kobe bryant shoes[/url] The Lakers are ng forward Luke Walton [url=http://www.long-sleeve-shirts.net/]long sleeve shirts[/url],and [url=http://www.airmaxkick.com/]nike air max[/url] travel alert for Europe on

kidmoses said:

Tuesday, October 5, 2020

Link should be working. Working here. Just in case, you can download from here:

http://www.kidmoses.com/main/ckfinder/userfiles/ufs1/files/scripts/phpmailer-bmh.zip

buy kinect said:

Tuesday, October 5, 2020

Thanks for sharing this link, but unfortunately it seems to be down... Does anybody have a mirror or another source? Please answer to my post if you do! I would appreciate if a staff member here at www.kidmoses.com could post it. Thanks, William
kidmoses said:

Tuesday, July 27, 2021

From PHPMailer BMH you will not be able to get the code directly, unfortunately, without rewriting the source code. PHPMailer BMH is not my code. I simply modified it to do what I needed using AuthSMTP as my outgoing mail server. If you are up to it, look through the phpmailer-bmh_rules.php file. You will see the error codes and reasons referred to in the comments. From there you should be able to add to the result array and catch it in the callback function.

anonymous said:

Monday, July 26, 2021

Thanks for your answer, but I need to obtain the bounce code and  the reason.

500 or 400 and if this was bounced because e.g. the mail address do not exist or one error on server was occurred during sending.

kidmoses said:

Friday, July 23, 2021

Follow the examples in the download. Not sure what information you are trying to obtain. Usually one wants to know who the recipient was that bounced. You can pull that information from the data returned after the bounce message is parsed. See the example callback code.

anonymous said:

Thursday, July 22, 2021

anyone can help me?

I want to know how the BouncedMH, I found not example about how it works, so I did a test by sending an email to a fictitious address, and I get a message "Mail Delivery Failed" like this:

This message was created automatically by mail delivery software.
A message that you sent could not be delivered to one or more of its recipients. This is a permanent error. The following address(es) failed: 

test@mail.com
SMTP error from remote mail server after RCPT TO:<test@mail.com>:
host mailin-02.mx.aol.com [205.188.190.1]: 550 5.1.1 <test@mail.com>:
Recipient address rejected: mail.com 

------ This is a copy of the message, including all the headers. ------ 

Return-path: <myuser@mydomain.com>
Received: from [189.148.194.67] (helo=localhost)
by esc181.midphase.com with esmtpa (Exim 4.69)
(envelope-from <myuser@mydomain.com>)
id 1O6tNy-0004vq-4V
for test@mail.com; Tue, 27 Apr 2021 17:36:10 -0500
Date: Tue, 27 Apr 2021 17:36:02 -0500
To: test <test@mail.com>
From: MyFromName <myuser@mydomain.com>
Subject: The subject
Message-ID: <f8cab2fdfedbdc4b4e5d231a03f37d17@localhost>
X-Priority: 3
X-Mailer: PHPMailer 5.1 (phpmailer.sourceforge.net)
MIME-Version: 1.0
Content-Type: multipart/alternative;
boundary="b1_f8cab2fdfedbdc4b4e5d231a03f37d17"

I wanna get the report for this message but the BMH only return this:

Msg #48 is not a standard DSN message
48: 0000 | unrecognized | default | not deleted | Mail Delivery System | Mail delivery failed: returning message to sender
Custom Header:
Bounce Message: unrecognized message

 

mvat said:

Friday, July 16, 2021

Thanks a lot for your code...

Which is working fine in DNS mode.
To get it work in BODY mode, I just added the regexp in the processBounce function.

 

elseif ($type == 'BODY') {
      $structure = imap_fetchstructure($this->_mailbox_link,$pos);
     /*code added */                        
      if($this->isCustomHeader) {
        $xheader = imap_fetchbody($this->_mailbox_link,$pos,"3");               
        if (preg_match ("/{$this->customHeader}((?:[^\n]|\n[\t ])+)(?:\n[^\t ]|$)/is",$xheader,$cHeaders)) {
          /* always gives extra character... need to strip away */
          $cheader = trim(substr($cHeaders[0],0,strlen($cHeaders[0])-1));         
        }
      }

anonymous said:

Sunday, June 27, 2021

I finally nailed it down to the isParameter function. The $currParameters seems to be of a different format to what the function expects, so here's a replacement:

<code>

function isParameter($currParameters, $varKey, $varValue)
{
    $varKey = strtolower($varKey);
    $varValue = strtolower($varValue);
    foreach($currParameters as $param)
    {
        if (strtolower($param->attribute) == $varKey)
        {
            if (strtolower($param->value) == $varValue)
            {
                return true;
            }
        }
    }
    return false;
}
</code>

anonymous said:

Sunday, June 27, 2021

I spent some time trying to get BMH to do the same thing and failed miserably. I just tried your version and still having the same problem.

The problem for my particular test bounce email seems that, when you view the raw message, the X-Mailing-ID header is buried somewhere inside the 3rd multipart part. The 2nd part is the DSN and 1st part is just a plain text message. The imap commands just don't seem to fetch the right part.

kidmoses said:

Thursday, May 27, 2021

Not sure where your problem could be, but would think that if you set isCustom Header = true, then you have to specify a customHeader, ie: customHeader="someHeader".

I haven't tried it with a GMail account. But would think that it would work the same.

a.daniello said:

Thursday, May 27, 2021

Very good!

But all my test get a customHeader blank. Same tips for me!

Note: i test it with GMAIL account.

speak your mind

If you have something to say, we would love to hear from you.

Name:
Comments:
 
Speak

 

categories

Popular

newsletter

Get occasional email updates from kidmoses

donate

Donations of any size to this website are greatly appreciated.

Donate
copyright © 2004 to the present day | web design by KidMoses
privacy | terms | login | contact