Welcome to my new blog, this is my first post in which I’ll show you how to setup a database transport for your CakePHP Emails.
CakePHP makes it easy to send emails from within your models and controllers, for example maybe you will send a order confirmation email after a customer purchases an item, or perhaps a user activation email when they first sign up for your website. The problem with this direct approach is that your user will have to wait for your application to send the email before the page will load and in some instances this can make the experience very slow at times.
I’ll show you a method in which all emails will automatically be queued up in the database and sent from a shell using a cron job, whilst it’s not the perfect solution for applications sending thousands of emails each hour, the chances are this will speed up your application for your users.
Step 1 – Setting up the Database Transport
To begin the code I am showing you is freely available here as a full plugin, I will show you how to recreate this exactly rather than building it directly into your application. First we need to define how CakePHP should handle the email request and save it into the database.
File: app/Plugin/DatabaseTransport/Network/Email/DatabaseTransport.php
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 |
<?php /** * DatabaseTransport * * CakePHP(tm) : Rapid Development Framework (http://cakephp.org) * Copyright (c) Cake Software Foundation, Inc. (http://cakefoundation.org) * * @copyright Copyright (c) Stephen Speakman * @link http://www.stephenspeakman.co.uk * @license The MIT License (MIT) */ App::uses('AbstractTransport', 'Network/Email'); /** * Any emails being sent by DatabaseTransport will be saved to the database instead * * @package app.Network.Email */ class DatabaseTransport extends AbstractTransport { /** * CakeEmail Object * * @var object */ protected $_cakeEmail; /** * Content of email to return * * @var string */ protected $_content; /** * Build the model data from the CakeEmail object * * @return array The model record data */ protected function _prepareData() { $headers = $this->_cakeEmail->getHeaders(array('to', 'cc', 'from', 'subject')); return array( 'EmailQueue' => array( 'to' => (! empty($headers['To']) ? $headers['To'] : null), 'cc' => (! empty($headers['Cc']) ? $headers['Cc'] : null), 'from' => (! empty($headers['From']) ? $headers['From'] : null), 'subject' => (! empty($headers['Subject']) ? $headers['Subject'] : null), 'serialized' => serialize($this->_cakeEmail) ) ); } /** * Set the configuration * * @param array $config The configuration to send email from * @return void */ public function config($config = array()) { $this->_config = $config; } /** * Saves an email to the model queue * * @param object $email The CakeEmail object * @return bool True if successful * @throws InternalErrorException If the email could not be saved */ public function send(CakeEmail $email) { $this->_cakeEmail = $email; $model = ClassRegistry::init('DatabaseTransport.EmailQueue'); $model->create(); if(! $model->save($this->_prepareData())) { throw new InternalErrorException('Unable to save email to queue!'); } return true; } } |
The focus of the above code is the send method, all this is doing is taking the information from CakeEmail and serializing the email content and extracting the headers such as ‘to’ and ‘from’. This is then saved as a database record in the email_queue table which will be picked up by the shell later on.
Step 2 – Setting up the database table
This is a nice and straightforward step, we just need to make sure we setup the database table to hold these emails.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
CREATE TABLE `email_queue` ( `id` mediumint(10) unsigned NOT NULL AUTO_INCREMENT, `to` varchar(255) DEFAULT NULL, `cc` tinytext, `from` varchar(255) DEFAULT NULL, `subject` varchar(255) DEFAULT NULL, `serialized` mediumtext NOT NULL, `status` tinyint(3) unsigned NOT NULL DEFAULT '0', `consumer` varchar(128) DEFAULT NULL, `processed` datetime DEFAULT NULL, `created` datetime NOT NULL, `modified` datetime NOT NULL, PRIMARY KEY (`id`) ) ENGINE=MyISAM AUTO_INCREMENT=42 DEFAULT CHARSET=utf8; |
Step 3 – Creating the Shell & Model
This is the really fun part, we’re going to create a shell (and associated model) which will query the database table for new emails at set intervals, when it detects new emails it will assign a unique id to the new batch and start sending them one by one. The reason we use a unique id is because let’s say you check every minute, but your shell is handling 1000 emails which takes more than a minute to send, we don’t want the next shell to queue up emails again and send duplicates.
Part 1 – Create the Shell Skeleton
File: app/Plugin/DatabaseTransport/Console/Command/EmailQueueShell.php
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 |
<?php /** * EmailQueueShell * * CakePHP(tm) : Rapid Development Framework (http://cakephp.org) * Copyright (c) Cake Software Foundation, Inc. (http://cakefoundation.org) * * @copyright Copyright (c) Stephen Speakman * @link http://www.stephenspeakman.co.uk * @license The MIT License (MIT) */ App::uses('Shell', 'Console'); App::uses('CakeEmail', 'Network/Email'); /** * Checks for new emails to be sent out and sends them accordingly. * * @package app.Console.Command */ class EmailQueueShell extends AppShell { /** * Tell the shell which model it should use * * @var array */ public $uses = array('DatabaseTransport.EmailQueue'); /** * This is a convience method to print the same header on each method * * @return void */ protected function _renderHeader() { $this->out(' Database Transport -> Email Queue Shell', 2); } /** * Default (no argument) method, inform the console of available commands * * @return void */ public function main() { $this->_renderHeader(); $this->out(' Usage:', 1); $this->out(" * <info>run</info> - Processes the email queue immediately.", 1); $this->out(" * <info>status</info> - Output the status of the email queue from within the last 24 hours.", 2); } /** * Find a potential queue and if found, send each email one by one * * @return void */ public function run() { } /** * Get the queue status including sent and failed in the last 24 hours * * @return void */ public function status() { } } |
In the above snippet all we’re doing is laying out the outline of the shell, which is as follows:
- Line 13: Include the CakeEmail object for when we send each email
- Line 27: Tell the Shell which models it should use (for queries and such)
- Line 34: Write a convenient header method which we can call on all methods (consistency is king)
- Line 43: Write the main method which tells our console of available commands
Part 2 – Create the Model Skeleton
Before we can start our main email handling method of the shell, we’re going to need to create a model to handle some of the data first, set up the following model skeleton.
File: app/Plugin/DatabaseTransport/Model/EmailQueue.php
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 |
<?php /** * EmailQueue. * * CakePHP(tm) : Rapid Development Framework (http://cakephp.org) * Copyright (c) Cake Software Foundation, Inc. (http://cakefoundation.org) * * @copyright Copyright (c) Stephen Speakman * @link http://www.stephenspeakman.co.uk * @license The MIT License (MIT) */ App::uses('AppModel', 'Model'); /** * Methods for the shell/transport for emails which are added to the queue * * @package App.model */ class EmailQueue extends AppModel { /** * useTable * * @var string */ public $useTable = 'email_queue'; } |
The above is a simple model layout, we will build it up with four crucial methods next, it’s important to have the $useTable parameter set up so CakePHP knows which table to use.
Model Method: updatePending
This method will find all pending emails and assign a consumer id to them, so they may be picked up by the shell and processed.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 |
/** * Updates all pending emails with a shell consumer id (prevents double processing) * * @param int $consumerId The consumer id (Shell process id) * @return bool True if successful, else false */ public function updatePending($consumerId = null) { $setAsPending = $this->updateAll( array( 'EmailQueue.consumer' => "'{$consumerId}'" ), array( 'EmailQueue.status' => 0, 'EmailQueue.processed' => null, 'EmailQueue.consumer' => null ) ); return $setAsPending; } |
Model Method: pendingItems
This method will find all email items by consumer id which have not yet been processed by the shell.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 |
/** * Finds all records against assigned consumer id which are pending * * @param int $consumerId The consumer id (Shell process id) * @return array The pending items found */ public function pendingItems($consumerId = null) { $pendingItems = $this->find('all', array( 'conditions' => array( 'EmailQueue.status' => 0, 'EmailQueue.processed' => null, 'EmailQueue.consumer' => $consumerId ) )); return $pendingItems; } |
Model Method: saveItemStatus
This method will set a status (0, 1 or 2) and processed date to an email record by primary key, used to log whether that particular email was successful or not.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 |
/** * Saves the item status of an email queue * * @param int $id The id of the EmailQueue record * @param int $status The status to save * @return bool True if successful, else false */ public function saveItemStatus($id = null, $status = 0) { $this->create(); $saveItem = $this->save(array( 'EmailQueue' => array( 'id' => $id, 'status' => $status, 'processed' => date('Y-m-d H:i:s') ) )); return $saveItem; } |
Model Method: countByStatus
This method is used to calculate statistics for the status shell command, giving statistics for the last 24 hours.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 |
/** * Counts email queue items by status and optionally date * * @param int $status The status to check (0, 1 or 2) * @param string $date Optional date string to check against * @return int The number of records found */ public function countByStatus($status = 0, $date = null) { $conditions = array('EmailQueue.status' => $status); if(! is_null($date)) { $conditions['EmailQueue.processed >='] = $date; } $count = $this->find('count', array( 'conditions' => $conditions )); return $count; } |
The completed model
Just for your reference, here is the completed model file in its entirety.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 |
<?php /** * EmailQueue. * * CakePHP(tm) : Rapid Development Framework (http://cakephp.org) * Copyright (c) Cake Software Foundation, Inc. (http://cakefoundation.org) * * @copyright Copyright (c) Stephen Speakman * @link http://www.stephenspeakman.co.uk * @license The MIT License (MIT) */ App::uses('AppModel', 'Model'); /** * Methods for the shell/transport for emails which are added to the queue * * @package App.model */ class EmailQueue extends AppModel { /** * useTable * * @var string */ public $useTable = 'email_queue'; /** * Counts email queue items by status and optionally date * * @param int $status The status to check (0, 1 or 2) * @param string $date Optional date string to check against * @return int The number of records found */ public function countByStatus($status = 0, $date = null) { $conditions = array('EmailQueue.status' => $status); if(! is_null($date)) { $conditions['EmailQueue.processed >='] = $date; } $count = $this->find('count', array( 'conditions' => $conditions )); return $count; } /** * Finds all records against assigned consumer id which are pending * * @param int $consumerId The consumer id (Shell process id) * @return array The pending items found */ public function pendingItems($consumerId = null) { $pendingItems = $this->find('all', array( 'conditions' => array( 'EmailQueue.status' => 0, 'EmailQueue.processed' => null, 'EmailQueue.consumer' => $consumerId ) )); return $pendingItems; } /** * Saves the item status of an email queue * * @param int $id The id of the EmailQueue record * @param int $status The status to save * @return bool True if successful, else false */ public function saveItemStatus($id = null, $status = 0) { $this->create(); $saveItem = $this->save(array( 'EmailQueue' => array( 'id' => $id, 'status' => $status, 'processed' => date('Y-m-d H:i:s') ) )); return $saveItem; } /** * Updates all pending emails with a shell consumer id (prevents double processing) * * @param int $consumerId The consumer id (Shell process id) * @return bool True if successful, else false */ public function updatePending($consumerId = null) { $setAsPending = $this->updateAll( array( 'EmailQueue.consumer' => "'{$consumerId}'" ), array( 'EmailQueue.status' => 0, 'EmailQueue.processed' => null, 'EmailQueue.consumer' => null ) ); return $setAsPending; } } |
Part 3 – Completing the Shell
Finally we may now complete the shell’s run and status methods and tie up some loose ends to give it a whirl. I’ll begin with the run method, I’ve commented the method more than I normally would but I’ll also attempt to explain a few things under the snippet too.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 |
/** * Find a potential queue and if found, send each email one by one * * @return void */ public function run() { // The consumer ID will be placed against all pending emails $consumerId = uniqid(); // Inform the user of the consumer id before proceeding $this->_renderHeader(); $this->out(" Consumer ID: {$consumerId}", 2); // Search for pending emails and apply the consumer id to them if(! $this->EmailQueue->updatePending($consumerId)) { return $this->out(' Found 0 items in queue, exiting ...', 2); } // Find all the emails which were assigned with the consumer id $pendingItems = $this->EmailQueue->pendingItems($consumerId); $this->out(' Found ' . count($pendingItems) . ' items in queue, dispatching:', 2); // Loop through each email one by one foreach($pendingItems as $item) { // Inform the user which email is being processed currency $this->out(' ' . str_pad("# {$item['EmailQueue']['id']}", 7, ' ') . ' ' . str_pad($item['EmailQueue']['to'] . ' ', 40, '.'), false); // Use your preferred email config here $email = new CakeEmail('mailgun'); // Attempt to send the serialized email using the above config, log the status if($email->transportClass()->send(unserialize($item['EmailQueue']['serialized']))) { $status = 1; $this->out(' [<success> OK </success>]', 2); } else { $status = 2; $this->out(' [<warning> !! </warning>]', 2); } // Attempt to update the email queue record with the new status if(! $this->EmailQueue->saveItemStatus($item['EmailQueue']['id'], $status)) { throw new Exception("Saving queue item failed, orphan created for ID #{$item['EmailQueue']['id']}!"); } } } |
There are a few important things happening in the above method which I will outline below
- Line 15: We’re calling the updatePending method from earlier to assign a batch
- Line 20: We’re calling the pendingItems method from earlier to query the batch so we can loop and send further down.
- Line 29: We’re defining our CakeEmail config to actually send the email (change this to whatever you normally use)
- Line 32: We’re attempting to send the email by chaining through transportClass() directly.
- Line 41: We’re logging the status of the email to the item record (success, failed).
Next up is the status method, it’s very straight forward as it just uses our countByStatus method from earlier to output information to the console window.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
/** * Get the queue status including sent and failed in the last 24 hours * * @return void */ public function status() { // Gather the stats needed for the output $pending = $this->EmailQueue->countByStatus(0); $sent24h = $this->EmailQueue->countByStatus(1, date('Y-m-d 00:00')); $failed24h = $this->EmailQueue->countByStatus(2, date('Y-m-d 00:00')); // Output the status to the console $this->_renderHeader(); $this->out(' Status:', 1); $this->out(" * Queued: {$pending}", 1); $this->out(" * Sent (last 24h): {$sent24h}", 1); $this->out(" * Failed (last 24h): {$failed24h}", 2); } |
The complete shell
Below you can find the complete shell snippet for you to copy and paste if you wish.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 |
<?php /** * EmailQueueShell * * CakePHP(tm) : Rapid Development Framework (http://cakephp.org) * Copyright (c) Cake Software Foundation, Inc. (http://cakefoundation.org) * * @copyright Copyright (c) Stephen Speakman * @link http://www.stephenspeakman.co.uk * @license The MIT License (MIT) */ App::uses('Shell', 'Console'); App::uses('CakeEmail', 'Network/Email'); /** * Checks for new emails to be sent out and sends them accordingly. * * @package app.Console.Command */ class EmailQueueShell extends AppShell { /** * Tell the shell which model it should use * * @var array */ public $uses = array('DatabaseTransport.EmailQueue'); /** * This is a convience method to print the same header on each method * * @return void */ protected function _renderHeader() { $this->out(' Database Transport -> Email Queue Shell', 2); } /** * Default (no argument) method, inform the console of available commands * * @return void */ public function main() { $this->_renderHeader(); $this->out(' Usage:', 1); $this->out(" * <info>run</info> - Processes the email queue immediately.", 1); $this->out(" * <info>status</info> - Output the status of the email queue from within the last 24 hours.", 2); } /** * Find a potential queue and if found, send each email one by one * * @return void */ public function run() { // The consumer ID will be placed against all pending emails $consumerId = uniqid(); // Inform the user of the consumer id before proceeding $this->_renderHeader(); $this->out(" Consumer ID: {$consumerId}", 2); // Search for pending emails and apply the consumer id to them if(! $this->EmailQueue->updatePending($consumerId)) { return $this->out(' Found 0 items in queue, exiting ...', 2); } // Find all the emails which were assigned with the consumer id $pendingItems = $this->EmailQueue->pendingItems($consumerId); $this->out(' Found ' . count($pendingItems) . ' items in queue, dispatching:', 2); // Loop through each email one by one foreach($pendingItems as $item) { // Inform the user which email is being processed currency $this->out(' ' . str_pad("# {$item['EmailQueue']['id']}", 7, ' ') . ' ' . str_pad($item['EmailQueue']['to'] . ' ', 40, '.'), false); // Use your preferred email config here $email = new CakeEmail('mailgun'); // Attempt to send the serialized email using the above config, log the status if($email->transportClass()->send(unserialize($item['EmailQueue']['serialized']))) { $status = 1; $this->out(' [<success> OK </success>]', 2); } else { $status = 2; $this->out(' [<warning> !! </warning>]', 2); } // Attempt to update the email queue record with the new status if(! $this->EmailQueue->saveItemStatus($item['EmailQueue']['id'], $status)) { throw new Exception("Saving queue item failed, orphan created for ID #{$item['EmailQueue']['id']}!"); } } } /** * Get the queue status including sent and failed in the last 24 hours * * @return void */ public function status() { // Gather the stats needed for the output $pending = $this->EmailQueue->countByStatus(0); $sent24h = $this->EmailQueue->countByStatus(1, date('Y-m-d 00:00')); $failed24h = $this->EmailQueue->countByStatus(2, date('Y-m-d 00:00')); // Output the status to the console $this->_renderHeader(); $this->out(' Status:', 1); $this->out(" * Queued: {$pending}", 1); $this->out(" * Sent (last 24h): {$sent24h}", 1); $this->out(" * Failed (last 24h): {$failed24h}", 2); } } |
Step 4 – Tying it all together
We’re almost done now, we just have to load the plugin in our app/config/bootstrap.php file using the following snippet CakePlugin::load('DatabaseTransport'); and last but not least we should setup our email configuration to use the DatabaseTransport. In app/config/email.php do the following:
1 2 3 4 5 6 7 |
public $database = array( 'transport' => 'DatabaseTransport.Database', 'from' => array('no-reply@yourhost.com' => 'Your Sender'), 'emailFormat' => 'html', 'charset' => 'utf-8', 'headerCharset' => 'utf-8', ); |
Now any time you send an email you can do the following $email = new CakeEmail('database'); and set up the rest of the email as normal.
Don’t forget to set up your cron job to run the email queue shell using the following command app/Console/cake DatabaseTransport.EmailQueue run
And that’s that! I hope you liked this blog post, please let me know if you have any questions or suggestions on this topic, I’d love to hear them.
Hi there Stephen – *exactly* was I was looking for – thank you so much… except – it’s just not firing for me…
— I’ve loaded the files/folders into app/Plugins; database table is in; updated bootstrap; and added the $database option in config/email.php.
— I’m now firing off email via CakeEmail(‘database’), but no response from Cake – it keeps on using the default CakeEmail.
— I’ve tried debug = 2; manually clearing all tmp cache etc – it just purrs along, working, as if the plugin wasn’t fired.
I’d really value your thoughts. Shoot me an email direct (I’ll post any resolution to my stoopidness here afterwards!).
Very happy to offer you a lifetime Pro membership on my site in return thanks! AB
OK – solved, and an update/clarification, I think…
$email = new CakeEmail(‘database’); transport(‘database’); transport(‘database’);
+ the rest of your email settings…
Worked perfectly! (Still give my site a lookin, and I’ll promote you to a Pro)
Many thanks, Stephen.
AB
[Edited to reformat/properly display code symbols]
OK – solved, and an update/clarification, I think…
$email = new CakeEmail(‘database’);
– By itself, did not work for me: it kept using the default setup in config/email.php
$Email->transport(‘database’);
– But declaring the transport authoritatively within the CakeEmail worked a charm : )
So, for clarity:
$Email = new CakeEmail();
$Email->transport(‘database’);
+ the rest of your email settings…
Worked perfectly! (Still give my site a go, and I’ll promote you to a Pro)
Many thanks, Stephen.
AB