Don't mail me my site errors

on 19/4/07

I wrote recently about how to have your site(s) email you whenever a problem occurs. Trouble is "problem" turns out to be a rather broad scope. If you have a missing file or fundamental problem of some kind, you would want to know so you can fix it; if there was a dead link on your site, you would also want to fix it; if someone just mistypes the url (or follows a link from an outdated source) you may not want an email each time this happens. At some point in between reading the comment from (the esteemed) Dr Tarique Sani and deleting the 4000+ emails I received from my site in the meantime, I came to realize that there has to be a better way to record what errors are occurring on a site. So that's what I'll be talking about today ;).

Limiting Scope

When I reviewed the emails that my site has sent me in the past week, there were many duplicates. The cause for most of the generated errors were either links to old pages which don't exist anymore or have been moved, or broken links on the site. When I switched host I also changed from (case insensitive) php4 to (case sensitive) php5, this caused many links to stop working. I was glad to know of how and why these errors were occurring, but once would have been enough ;). Of the types of errors that can occur, there is one case in which I want an email every time it happens - if there is a missing connection (which should be rare to never). So changes have been made to check for and only email in this circumstance, for the rest of the time...

Save errors to the database

As far as I'm concerned, there are 2 parts to the data of an error - the generic details i.e. the class/url/controller/action that is affected, and the specific (and potentially private) details of the circumstances in which the error occurred. For example, if a helper file is missing, I want to know about it, but if that helper file is only included under certain circumstances it would be wise to be aware of how the error occurred such that it is possible to reproduce, prevent and add a test case. Below is the new and improved error class:

<?php
/* SVN FILE: $Id: app_error.php 28 2007-02-08 20:51:38Z Andy $ */
/**
* Short description for file.php.
*
* Long description for file
*
* PHP versions 4 and 5
*
* Copyright (c), Andy Dawson
*
* Licensed under The MIT License
* Redistributions of files must retain the above copyright notice.
*
* @filesource
* @copyright Copyright (c) 2007, Andy Dawson
* @package noswad
* @subpackage noswad.generic
* @since Noswad site version 3
* @todo TODO Edit this block
* @version $Revision: 28 $
* @created 26/01/2007
* @modifiedby $LastChangedBy$
* @lastmodified $Date: 2007-02-08 21:51:38 +0100 (jue, 08 feb 2007) $
* @license http://www.opensource.org/licenses/mit-license.php The MIT License
*/
uses('error');
function request_headers() {
if (function_exists("apache_request_headers")) // If apache_request_headers() exists...
{
if ($headers= apache_request_headers()) // And works...
{
return $headers; // Use it
}
}
$headers= array ();
foreach ($_SERVER as $skey => $sVal) {
if (up(substr($skey, 0, 4)) == "HTTP") {
$headers[$skey]= $sVal;
}
}
return $headers;
}
/**
* AppError class. Used to customize the way errors are handled.
*
* This class allows the application to modify the behavior in the case of an error.
*/
class AppError extends ErrorHandler {
/**
* Constructor method. Overriden to email the site admin if a 404 is generated, with the real reason for the error
* message.
*
* @param type $name description
* @return type description
*/
function __construct($method, $messages) {
//if (!DEBUG) {
$this->recordError($method, $messages);
//}
parent :: __construct($method, $messages);
}
function recordError($method, $messages) {
if ($_SESSION) {
ob_start();
echo '<pre>';
print_r($_SESSION);
echo '</pre>';
$sessionData= ob_get_clean();
} else {
$sessionData= null;
}
if ($_POST) {
ob_start();
echo '<pre>';
print_r($_POST);
echo '</pre>';
$postData= ob_get_clean();
} else {
$postData= null;
}
ob_start();
echo '<pre>';
print_r(request_headers());
echo '</pre>';
$headerData= ob_get_clean();
if ($method == 'missingConnection') {
$logItAswell= true;
$message= 'Site ' . env('SERVER_NAME') . ' has generated an error message' . "\n\n";
$message .= "Error Type: $method\n";
foreach ($messages[0] as $key => $value) {
$message .= str_pad($key, 10) . ": $value\n";
}
$message .= 'Referer : ' . env('HTTP_REFERER') . "\n";
$message .= 'Remote add: ' . env('REMOTE_ADDR') . "\n";
@ mail(env('SERVER_ADMIN'), env('SERVER_NAME') . ' Error: ' . $method, $message);
if ($logItAswell) {
$message .= 'Session : ' . $sessionData . "\n";
$message .= 'Post data : ' . $postData . "\n";
$message .= 'Header : ' . $headerData . "\n";
$this->log($message);
}
}
elseif (!($method == 'missingTable' && in_array($messages[0]['className'], array (
'Bug',
'BugDetail',
'BugSection',
'BugStatus'
)))) {
loadPluginModels('mi_bugs');
$Bug= new Bug();
if (isset ($messages[0]['base'])) {
$sectionName= $messages[0]['base'] ? r('/', '', $messages[0]['base']) : APP_DIR;
} else {
$sectionName= 'MainApp';
}
$section= $Bug->BugSection->findbyName($sectionName, null, null, -1);
if (!$section) {
$Bug->BugSection->saveField('name', $sectionName);
$section= $Bug->BugSection->read();
}
switch ($method) {
case 'error404' :
$controller= $action= '';
$title= $method;
break;
case 'missingAction' :
$controller= $messages[0]['className'];
$action= $messages[0]['action'];
$title= $method;
break;
case 'missingComponentClass' :
$controller= $action= '';
$title= $method . ' : ' . $messages[0]['className'];
break;
case 'missingComponentFile' :
$controller= $action= '';
$title= $method . ' : ' . $messages[0]['className'];
break;
case 'missingController' :
$controller= $messages[0]['className'];
$action= '';
$title= $method;
break;
case 'misingHelperClass' :
$controller= $action= '';
$title= $method . ' : ' . $messages[0]['helper'];
break;
case 'misingHelperFile' :
$controller= $action= '';
$title= $method . ' : ' . $messages[0]['helper'];
break;
case 'missingLayout' :
$controller= $action= '';
$title= $method . ' : ' . $messages[0]['layout'];
break;
case 'missingModel' :
$controller= $action= '';
$title= $method . ' : ' . $messages[0]['className'];
break;
case 'missingTable' :
$controller= $action= '';
$title= $method . ' : ' . $messages[0]['className'] . ' ' . $messages[0]['table'];
break;
case 'missingView' :
$controller= $messages[0]['className'];
$action= $messages[0]['action'];
$title= $method . ' : ' . $messages[0]['file'];
break;
case 'privateAction' :
$controller= $messages[0]['className'];
$action= $messages[0]['action'];
$title= $method . ' : ' . $messages[0]['className'];
break;
default :
$controller= $action= '';
$title= $method;
if (isset ($messages[0]['controller'])) {
$controller= $messages[0]['controller'];
$title .= ' : ' . $messages[0]['controller'];
}
if (isset ($messages[0]['action'])) {
$action= $messages[0]['action'];
}
break;
}
$constraint['Bug.site']= env('SERVER_NAME');
$constraint['Bug.controller']= $controller;
$constraint['Bug.action']= $action;
$constraint['Bug.title']= $title;
if (isset ($messages[0]['url'])) {
$url= $messages[0]['url'];
} else {
$url= 'N/A';
}
$constraint['Bug.url']= $url;
$bug= $Bug->find($constraint, null, null, 0);
$messagesX= $messages[0];
if (!$bug) {
$bugData['site']= env('SERVER_NAME');
$bugData['controller']= $controller;
$bugData['action']= $action;
$bugData['url']= $url;
$bugData['title']= $title;
$bugData['description']= 'See bug incident detail(s) for more info.' . "\n";
$bugData['section_id']= $section['BugSection']['id'];
$openId= $Bug->BugStatus->findByName('open', array (
'id'
), null, -1);
$bugData['status_id']= $openId['BugStatus']['id'];
$bugData['reported_by']= 'The friendly error bot';
$Bug->save($bugData);
$bug= $Bug->read();
}
elseif (!in_array($bug['BugStatus']['name'], array (
'open',
'won\'t fix',
'not fixable',
'duplicate'
))) {
$reopenedId= $Bug->BugStatus->findByName('reopened', array (
'id'
), null, -1);
$Bug->saveField('status_id', $reopenedId['BugStatus']['id']);
}
// Add more specific (and potentially private) details to the incident.
$bugData['body']= '';
unset ($messagesX['url']);
unset ($messagesX['base']);
foreach ($messagesX as $key => $value) {
$bugData['body'] .= str_pad($key, 10) . ": $value\n";
}
$bugData['bug_id']= $bug['Bug']['id'];
$bugData['site']= env('SERVER_NAME');
$bugData['error_type']= $method;
$bugData['client_ip']= env('REMOTE_ADDR');
$bugData['referer']= env('HTTP_REFERER');
$bugData['session_data']= $sessionData;
$bugData['post_data']= $postData;
$bugData['header_data']= $headerData;
$bugData['reported_by']= 'The friendly error bot';
$Bug->BugDetail->save($bugData);
}
}
}
?>

It is a little more complicated than the previous version, and I won't include the necessary table definitions in this blog post (although they can be easily deduced). I will however make a version of the administrative plugin available as an online demo/download. To explain in brief the extra logic, the class now does the following:

If there is a problem connecting to the database, email me
Otherwise, so long as the error doesn't relate to the bug tracking tables themselves:
1. Get the session data
2. Get any form data
3. Get the sent headers
4. Load the models for the plugin "mi_bugs"
5. Find or create the (unique) bug for the error or url in combination with the plugin/controller/action which caused it.
6. Create a detail report linked to the bug with the session, form and header data

In the past few hours, there have been approximately 50 incidents logged, and 10 unique 'bugs', it is now a relatively simple (and actually interesting at times) task to browse what has been logged and prioritize which, if any, require action. It can be extended also; I've hooked my anti-Spam system into it such that form submissions which are identified as spam are recorded for later analysis.

Wrapping Up

The code presented here allows you to know, in a useful and manageable way, what errors are generated by your site(s). A demo/download will be available shortly for (amongst other functionality) browsing/administering the info this class will collect.

Bake on!