"Linux Gazette...making Linux just a little more fun!"

Content Management with Procmail

By Pradeep Padala and Prakash Bulusu

"Small is beautiful" -- Unix Philosophy

Introduction

Ever wondered what it takes to create a web site that can be completely managed by e-mail? Are you one of those e-mail buffs who wants to manage everything with mail? Are you the guy who wants something different? If your answer is yes to any of these questions, read on....

Have you ever thought how a special correspondent for CNN reporting from the remotest place on earth can cause a web page to change its content based on the latest he is reporting ? Or how the Newspaper dot coms manage hundreds of HTML pages every day ? The achieve this through a concept called a Content Management System.

Content Management is one of the prominent issues every website has to deal with, from lightly-loaded sites to very content-intensive sites). A very basic Content Management System (CMS) at the very least should provide a user friendly interface to modify the Web Content. A sophisticated CMS does much more than that, not only providing management of creation, modification and deletion of content but also services like revision control, roles hierarchy, multichannel content management and delivery etc. In this article we talk about a relatively new channel for Content Management very well known to readers: e-mail! Though this particular channel has already been put to use by commercial CMS's like Vignette, they are rather very expensive and sold on a per-feature basis.

If the above paragraph looked like Greek to you then you are an Eligible Candidate to read this article. Carry on and you will learn the simplest ever implementation of e-mail based Content Management System with Procmail.

Here we will show how a web page can be updated just by sending an e-mail. We will use this test page (Javascript should be enabled to properly view this page). All it contains is a streamer. We update the text of the streamer when we receive mail with proper subject!!!

procmail

Now what's the connection to Procmail? Procmail follows the UNIX philosophy that a program accomplishes one task, efficiently . Procmail is an extremely powerful mail filtering utility. All one has to do is write recipes which get executed when certain type of mail arrives. It is generally used to filter incoming mail into seperate directories. In the Procmail developers' words,

Procmail can be used to create mail-servers, mailing lists, sort your incoming mail into separate folders/files (real convenient when subscribing to one or more mailing lists or for prioritising your mail), preprocess your mail, start any programs upon mail arrival (e.g. to generate different chimes on your workstation for different types of mail) or selectively forward certain incoming mail automatically to someone.

If you don't understand any of those terms above, dont panic. Just sit back and relax. We will show some basic examples on how it works and explain a cute method to update the streamer on your web page.

The intial blues

Procmail is a mail filtering utility. It can be run on each mail sent to you and process it accordingly. Say, you wanted to save all the mail with subject discuss to a folder named discuss. Then you have to write the following lines in the files specified.

In your home directory create a file named .forward and put the following lines

	"|IFS=' ';exec /usr/local/bin/procmail USER=<username>"

Replace the username with your username and make sure that the path to the Procmail binary is set correctly. Don't worry about the details. We will dig deeper into them in the next section.

Create a file named .procmailrc in your home directory and add the following lines

	:0
	* ^Subject:.*discuss
	discuss

Create a directory named discuss in your home directory for putting the mails with subject discuss.

Now try sending a mail to yourself which contains at least the word "discuss" in the subject. This mail will automatically be moved into a directory discuss.

Let's try to understand what's happening here. When a mail is sent to an smtp server, it will be delivered to the corresponding user by a MDA(Mail Delivery Agent) like sendmail. This program looks for a file named .forward; if it exists it tries to execute the rules specified in the .forward file. Usually the .forward contains a mail address to which one wishes to forward all mail. We may also write commands within this file to execute a program like Procmail. This is exactly what the lines above in the .forward do. For a detailed explanation of things to be put in .forward see here Once Procmail is executed, it looks for the file .procmailrc which contains mail processing directives - what actions have to be taken for different kinds of mail.

.procmailrc file

The file .procmailrc contains rules for how to filter the mail. In above example, the rule says that for all the mails with subject discuss, forward them to a directory named discuss. General syntax for writing rules (called recipes in Procmail jargon) is

	:0 [flags] [ : [locallockfile] ]
	<zero or more conditions (one per line)>
	<exactly one action line>

You can ignore the first line for now. From the second line, you can start writing conditions. A condition starting with '*' specifies an extended regular expression to be matched. If it matches then the action line is executed. If the action line starts with a '|' then it executes the program whose name follows the '|'. You can use the '!' character to forward to mail to another address. If the action line doesn't start with either of these two characters or a '{', then it is assumed to be a directory or file to which mail has to be delivered, as in the above example.

Another example will clear the mist.

	
	:0 c
	* ^Subject:.*discuss
	discuss

	:0
	* ^Subject:.*discuss
	! yourname@somewhere.com

This recipe forwards the mail to the address yourname@somewhere.com and keeps a copy in the directory discuss. The flag 'c' in the first stanza tells Procmail to continue reading recipes even if this recipe matches. Normally Procmail stops after the first match. The procmailrc manpage says 'c' generates a carbon copy of the message, but it's easier to think of 'c' as meaning "continue". Either way amounts to the same thing. Another useful but ill-named flag is 'D', which makes the match expressions case sensitive.

The following example shows the usage of '|'

	:0
	* ^Subject:.*discuss
	| formail -I "" >> index.html

If a mail with subject discuss comes, formail is executed. formail is a small utility which can be used to format mail. The above action line extracts the body from the mail and appends it to the file index.html.

Ready for action?

You have seen some basic examples. There is a lot of information on Procmail recipes on web. Refer to the section on resources for links. In the next few sections, we will show how Procmail can be used to update a streamer on a web page.

We put the following lines in .procmailrc. My procmailc at it stands now is here

	
	SHELL=/bin/sh
	PATH=/bin:/usr/bin:/usr/local/bin
	
	:0
	* ^Subject:.*announce
	| formail -s parse.pl announce;

	:0
	* ^Subject:.*notice
	| formail -s parse.pl notice;

The first lines set some variables so that Procmail works properly. See procmailrc man page for details.

Here, mail containing the subject "announce" or "notice" is forwarded to formail. formail parses the mail and each individual mail is given to a perl script named parse.pl. The perl script updates the streamer with the string in the body of message. The text version of the script is here

#!/bin/perl

$option = $ARGV[0];                                 # Get the option
$my_html_dir = "/cise/homes/ppadala/public_html";   # My web directory
$tmp_file = "/tmp/tmp.out";
$start = 0;     #start would be false(0) until we get to a message beginning 
$line = "";     #The streamer line

if($option eq "announce") {
    $line = "ANNOUNCEMENTS:";
}
elsif($option eq "notice") {
    $line = "NOTICE:";
}
else {
    exit(1);
}

#Read the input. In this case the mails
#Parse the body and attach it to line

while()
{	chomp;

	if(/From.*/) {
		$start = 0;
	}	
	if($start == 1) {
		chomp;
		$line = $line . $_;
	}
	if($_ eq "") {
		$start = 1;
	}
}

#Open the web page containing streamer 
#and update it

$my_homepage_file = $my_html_dir . "/procmail.html";
open(MY_FILE, "<$my_homepage_file") || die("Cannot open input file");
open(TMP, ">$tmp_file");

$replace = 0;

#Replacement is done just after the line
# //Replace strStreamer .....

while()
{	if($replace == 1) {
		print TMP "var strStreamer = '${line}';\n";
		$replace = 0;
		next;
	}
		
	if(/\/\/Replace str.*/) {
		$replace = 1;
		print TMP $_;
	}
	else  {
		print TMP $_;
	}
}
	
close(TMP);
close(MY_FILE);
system("mv $tmp_file $my_homepage_file");
system("chmod go+r $my_homepage_file");

All the perl script does is update the variable strStreamer in the web page. The web page contains a streamer written in javascript. You can test it by sending mail You can see the streamer at http://www.cise.ufl.edu/~ppadala/procmail.html. It is magically updated when you send mail to my address with a subject "announce" or "notice". The text in the body goes to streamer.

Conclusion

This is a small example of Content Management. Content Mangement is a huge topic with a lot of ramifications. It requires well planned procedures for updating web pages, keeping their style sheets intact etc.. We have shown a small example of how easy it is to create a basic content management system with Procmail. The permuatations are endless. As Descartes had said "It is not enough to have a good mind. The main thing is to use it well".

Resources

Procmail.org website
Procmail Documentation Project : Lot of resources
Procmail Tips page : A must-see page

Pradeep Padala

I am a master's student at University of Florida. I love hacking and adore Linux. My interests include solving puzzles and playing board games. I can be reached through p_padala@yahoo.com or my web site.

Prakash Bulusu

I am a master's student at the University of Florida.