SLAC form-mail Utility Documentation

SLAC, Created January, 1996; Last Updated: September 7, 2004

Page Contents


Introduction

Form-mail is a CGI utility script in Perl that provides form authors with a simple mechanism to create and send both simple Email items and more complex Email. By complex in this case we are referring to an Email item with lots of variable names and values that are meant to be parsed and acted upon by a mail filter program. In both cases the script expects the form to use the "POST" method to provide the input to form-mail.pl, for example:
<FORM METHOD="POST" ACTION="/cgi-bin/form-mail.pl">

Using form-mail for simple Email Forms

In this case the following 8 variables can be supplied by the form (note that the variable names are case sensitive). Those marked "Must" must be supplied, those marked "optional" are optional.
Variable NameTypePurposeExample
toMustThe Email address of the recipient(s) jochen@slac.stanford.edu or
sarah@slac.stanford.edu,jxh@slac.stanford.edu
usernameMustThe Email address of the sendercottrell@slac.stanford.edu
realnameMustThe real name of the senderLes Cottrell
subjectMustThe Subject line for the EmailYour Page on Email
commentsMustThe body of the EmailI liked your page and found it informative.
form_contactMustEmail address of person to contact about the form (e.g. the author/owner of the form). Usually this will be given in a type ="hidden" variable.cottrell@slac.stanford.edu
ccOptionalCarbon copy listcottrell@slac.stanford.edu or
cottrell@slac.stanford.edu,pbloom@slac.stanford.edu
bccOptionalBlind carbon copy listcottrell@slac.stanford.edu or
cottrell@slac.stanford.edu,pbloom@slac.stanford.edu
You may wish to review the SLAC suggestion form source for a an example of a simple Email Form using form-mail.pl.

Using form-mail for complex Email Forms

In this case it is expected (by form-mail) that the output from the form is supposed to be sent by form-mail to an Email parsing filter. Thus the variables are of the form nnname where nn are two decimal digits. The 2 decimal digits (leading zeroes are needed) are used to guarantee the order in which the variable name and value will be included in the body of the Email. The variable names must be composed of alphanumerics (i.e 0-9 a-Z _ A-Z), other characters may cause problems. The appearance of any variable names with 2 leading digits will cause form-mail to provide the complex Email format (see below). Note that only variables whose names start with 2 decimal digits will appear in the complex Email sent.

In order to properly address the Email the four variables with names to, username, realname and subject are required to be provided to form-mail by the form. Note that these variable names may be prefixed by the 2 decimal digits if they are to appear in the sorted list of variable names and values sent in the body of the Email.

The variables with names starting with 2 decimal digits are listed in the body of the Email following a line with composed of multiple dashes (-----------) and before a second line with multiple dashes. Note that in this listing the 2 decimal digits are stripped from the front of the variable name. The format of each variable is:
variable-name .............. variable-value

Thus a complete complex Email item as received might appear as:

Date: Fri, 02 Feb 1996 18:33:51 -0800 (PST)
X-UIDL: 823314928.001
Subject: Pager Survey
To: cottrell@SLAC.Stanford.EDU
X-Envelope-to: cottrell@MAILBOX.SLAC.Stanford.EDU

cottrell@slac.stanford.edu (Les Cottrell) sent the following:

------------------------------------------------------------
subject         ........   Pager Survey   
realname        ........   Les Cottrell   
username        ........   cottrell@slac.stanford.edu
Phone           ........   2523           
Account         ........                  
Type            ........   900 MHz        
CAP             ........                  
Serial          ........                  
Alternate       ........                  
Group           ........                  
Voice           ........                  
SoCal           ........                  
Comments        ........                  
Remember        ........                  
AdminStuff      ........   ------------------------------------
TO              ........   cottrell@slac.stanford.edu
cc              ........                  

------------------------------------------------------------
Server protocol: HTTP/1.0
Remote host: unixhub.SLAC.Stanford.EDU
Remote IP address: 134.79.16.15
Mail program: /usr/lib/sendmail
Referer_URL: http://www.slac.stanford.edu/~cottrell/form-mail-form.html
You can review an example of a form using the complex form-mail for more insight.

Customization of Response to User

In order to allow the form writer to customize the response that form-mail provides to the form user, form-mail accepts a variable with name response_msg. The value of this variable will be used to provide the response to the user (apart from the response page's <title> and first <h1> header). The format of the value is HTML text which may include other form variables embedded between at signs (@). For example the response_msg variable could be set in the form as follows:
<input type="hidden" name="response_msg" value="
Thank you, @realname@, for taking the time to fill out
       the form.">
Note that these other form variables are referenced without any leading 2 digits (e.g. @realname@ NOT @01realname@). Note also that any HTML markup delimiters (< and >) that appear between the quotation marks (") in the value field must be entered as &lt; and &gt; respectively. Finally note that quotation marks (") may not appear in the body of the response_msg value field, since the first occurence will terminate the value field. If you wish to include an HREF in the response_meg value then do not use quotation marks to delimit the stuff following the HREF=.

For a more complete demonstration of the use of response_msg review the source of this example that provides a user response in a hidden field of the form.

In some cases you may wish to provide the customized response_msg from a file. There is therefore a variable with the name response_msg_file. The value assigned to this variable, if it occurs in your form, is the partial URL of the file containing the response_msg. For example let's say the file is saved at /u/is/rye/public_html/ann_msg.txt then the variable will be written in the form as name="response_msg_file" value="/~rye/ann_msg.txt". Currently the file must be accessible directly to the server. Tests are made on the URL to ensure it contains valid characters etc., that the file exists, that it is readable, that it is a text file, to check for possible security/privacy exposures and that the filename is valid.

If you supply both a response_msg and a response_msg_file then response_msg takes precedence and response_msg_file is ignored. Variable names of the form @name@ that appear in the file are replaced with the values of the relevant variables.

HTML markup that appears in the file is simplified compared to response_msg in that special characters such as < are inserted directly without converting to &lt; form. Also the file must be a complete HTML file, e.g. it must have the <html><head><title>...</title> </head><body> etc. since nothing is inserted by form-mail. A spin-off benefit of this is that it enables the form-writer to avoid the "Form Response" line that is inserted at the top of the response if one uses response_msg.

See the source of Example of response_msg_file Form for an example of how to use response-msg-file.

Customizing Format of Body of Email Sent

To customize the format of the body of the Email sent by form-mail you can use the variable email_msg in a similar fashion to response_msg. Since the form writer may delimit the output with tabs in this fashion, this may be used to simplify input into a database or spreadsheet which accepts tab delimited input. Since the Email will not be displayed by a WWW browser there is usually little point in using HTML formatting in email_msg. Also, since email_msg determines the order of the variables in the Email, there is no need to order the variables by including 2 digit numbers in front of the names. Note that the same restriction on the use of the mark-up characters (< > ") apply for email_msg as for response_msg (see the previous section).

See the source of the form-mail Example Using Email Template for an example of how to use email_msg. Typical Email generated by such a form appears as:

Date: Fri, 09 Feb 1996 14:17:14 -0800 (PST)
Subject: Pager Survey
To: cottrell@SLAC.Stanford.EDU
X-Envelope-to: cottrell@MAILBOX.SLAC.Stanford.EDU



To:             cottrell@slac.stanford.edu
realname:       (Les Cottrell)  username:       cottrell@slac.stanford.edu      
subject:        Pager Survey
Date:    	Mon Feb 12 10:37:30 PST 1996
comments:       {
Nice talking to you yesterday.
Best wishes 
Les
}        
Phone:          (415)926-2523

Best of luck!
In some cases you may wish to provide the customized email_msg from a file. There is therefore a variable with the name email_msg_file. The value assigned to this variable, if it occurs in your form, is the partial URL of the file containing the email_msg. For example let's say the file is saved at /u/is/rye/public_html/ann_msg.txt then the variable will be written in the form as name="email_msg_file" value="/~rye/ann_msg.txt". Currently the file must be accessible directly to the server. Tests are made on the URL to ensure it contains valid characres etc., that the file exists, that it is readable, that it is a text file, to check for possible security/privacy exposures and that the filename is valid.

If you supply both an email_msg and an email_msg_file then email_msg takes precedence and email_msg_file is ignored. Variable names of the form @name@ that appear in the file are replaced with the values of the relevant variables.

See the source of form-mail Example of using Email Template file for an example of how to use email-msg-file.

Built-In Variables

To facilitate customization you can also include the following built-in variables in your response_msg or email_msg.
Variable NamePurposeExample
DATECurrent date and timeMon Feb 12 9:55:16 PST 1996
MAIL_AGENTMail Agent, i.e. form-mailform-mail.pl v1.1
REFERERThe URL of the Form from which form-mail was calledhttp://www.slac.stanford.edu/suggestion/cottrell
REMOTE_HOSTThe name of the host on which the browser was runningatlas.slac.stanford.edu
SERVER_HOSTThe name of the host the server is running onwww.slac.stanford.edu
HTTP_USER_AGENTThe name of the browserMozilla/3.0b4 (X11; I; AIX 2)

Specifying the Form in Form-mail's Path

It is also possible, and in fact recommended, that the form itself be invoked by including the path to get to it in the URL. In this case the URL embedded in the page that wants access to your form appears as: http://www.slac.stanford.edu/cgi-bin/form-mail.pl/path/to/your-form. For example, /path/to/your-form might appear as /~cottrell/suggest.html (which is translated to /u/sf/cottrell/public_html/suggest.html or as /slac/www/stats/roll-your-own-err.html which would be translated to /afs/slac/www/slac/www/stats/roll-your-own-err.html.

In such a case where the path is provided to form-mail, form-mail will display your form and then your form will again call form-mail when the user pushes "Submit". In other words form-mail is self-referencing. The Email address of the person to whom the form is to be sent may also be specified following the path, by preceding the Email address with a question mark (?). In this case form-mail will replace the name="to" variable value with value="QUERY_STRING" (where QUERY_STRING is the value of the environment variable QUERY_STRING which is whatever follows the question mark (?) in your URL) in the form template. Thus if the Email address following the question mark is say jxh@slac.stanford.edu then form-mail will look through the template for something like name="to" value="any old text" and change it to: name="to" value="jxh@slac.stanford.edu"

A complete example of such a URL is html://.../cgi-bin/form-mail.pl/~cottrell/suggest.html?jxh@slac.stanford.edu where the form template is at /u/sf/cottrell/public_html/suggest.html and the mail generated from the form is sent to userid jxh@slac.stanford.edu (John Halperin).

Security

To prevent the use of form-mail as a mail relay (for example for spammers), by default form-mail can only send mail to addresses in the slac.stanford.edu domain. To enable sending of emails to non slac.stanford.edu addresses form-mail can be called through the SLAC CGI wrapper (cgi-wrap). In this case the script is called via the URL http://www.slac.stanford.edu/cgi-wrap/form-mail.pl. All scripts called through cgi-wrap are registered and each has a rule associated with the script. The rule associated with form-mail is that it can only be called from clients (usually browsers) in the SLAC Internet address domain (134.79.*.*). As a special case if form-mail is called via cgi-wrap then it is enabled to send email to non slac.stanford.edu addresses.

Frequently Asked Questions

Can I view the input form-mail receives from my form?
There is a special variable debug that may be set to provide debugging output from form-mail. It is turned on from your form by using:
<input type="hidden" name="debug" value="1">. The debugging is turned off by setting <input type="hidden" name="debug" value="0">.
Can I enter a to address like distribution list <user@node.site.edu>?
Yes. To avoid the browser interpreting the stuff between < and > as markup in your form, the < has to be entered in your form as &lt; and the > as &gt;. For example: <input type="hidden" name="to" VALUE="distribution bbr_speak &lt;bfmail@slac.stanford.edu&gt;>
View the source of the BaBar Speakers Bureau Mailing Form for an example of this.
Can I have more than one Email address in the to or cc or bcc values?
Yes. Just insert a comma (,) with no spaces between the Email addresses. See the example in the table above.
How do I insert a Cc: field in the header
As of 8/30/2004 the Cc: and Bcc: header fields are supported. You use the variables cc and bcc respectively, e.g.
<input type="hidden" name="bcc" value="cottrell@slac.stanford.edu">
Can I enter multiple values into a single name?
Yes. If they need to be outputted (e.g. in an @name@ substitution) then the values after the first will be preceded by a newline.
See the Views from the Summit form for an example of this.
Can I use separate input boxes for the first and surnames.
Yes, the variable name would be realname in both input cases. For example:
<INPUT TYPE="text" SIZE="35" VALUE="" NAME="03realname"> <b>Your First Name</b><br>
<INPUT TYPE="text" SIZE="35" VALUE="" NAME="04realname"> <b>Your Last Name</b><br>

Form-mail will concatenate the two values entered for the realname variable separating them by a space.
I can't get form-mail to substitute the value for @request-form@ in my email_msg.
The name request-form is invalid, names must be composed of alphanumeric characters only (a-z A-Z 0-9 _ ), the minus sign (-) is invalid.
Is there a public domain equivalent of form-mail that I can get for my site?
Lincoln Stein has a similar package called mailmerge that is publicly available.
My form variable @01realname@ does not appear to have its value substituted in the response_msg, how come?
The form variable must be referenced as @realname@, i.e. the leading 2 digits must be stripped off.
When using complex Email, do I have to put the 2 leading digits at the start of the names of variables that I want to appear in the complex Email sent?
Yes.
Why does the body of the mail sent by form-mail seem to have no line ends in it sometimes.
Some browsers, in particular Microsoft Internet Explorer 3.01 on a Macintosh, strip all the line-ending information from the text before sending it to the server. The best fix is to update to a newer browser, and curse Microsoft :-). Another possibility is to insert &#13; at the end of lines in the form. This will result in doubled line ends for most browsers.
Can a variable such as "comments" in the Simple Email Form be used more than once.
Yes. In the case of comments, the values from the various comments will be concatenated and appear between the ----- bars in the sent email.
Can I get rid of the first line that says "Form Response" in the page that is displayed after submitting the form when using response_msg.
Use response_msg_file instead, see above.
Can I get a copy of the email to be sent also to the user of the form?
Yes, you can use the 'username' variable name in the 'to' variable, e.g.
<input type="hidden" name="to" value="cottrell@SLAC.Stanford.Edu,@username@">
I am trying to use form-mail to send email to a non slac.stanford.edu address, but it rejects it, how come?
To avoid the relaying of email by spammers, form-mail does not allow the use of non slac.stanford.edu addresses. However if you use the http://www.slac.stanford.edu/cgi-wrap/form-mail.pl form of the URL to reference form-mail in your form (see the section above on Security, then it is possible to get form-mail to send email to non slac.stanford.edu addresses.
The form that calls form-mail is in the SLAC only domain, why can't it use http://www.slac.stanford.edu/cgi-bin/form-mail.pl to send email to non slac.stanford.edu addresses.
Even if the form is in the SLAC only part of the web, the CGI script that it uses (form-mail) runs on the www.slac.stanford.edu server that is open to the outside world. Thus the CGI script can only allow certain features else a cracker could compose a URL to be sent to form-mail to get it to send mail to addresses in the Cc:, Bcc: or To: lists. Thus these lists only allow slac.stanford.edu addresses.
Can we set up an instance of form-mail on a SLAC only domain web server?
That would be easy to do if we had a SLAConly web server running Unix (by the way, in effect this is what using http://www.slac.stanford.edu/cgi-wrap/form-mail.pl does). For a Windows web server (e.g. such as www-internal.slac.stanford.edu), the server would need to have perl installed (this is not the default for Windows hosts at SLAC), and form-mail would need to be ported to Windows. The main problem in porting would be the use of Unix utilities such as sendmail. On the other hand there maybe native Windows/FrontPage mail utilities (e.g. .asp scripts) that could meet your needs and are already installed on a SLAConly Windows server such as www-internal, that may be suitable.
I am using the http://www.slac.stanford.edu/cgi-wrap/form-mail version of form-mail, but when travelling or at home, it will not allow me to send email to non slac.stanford.edu addresses, how come?
The CGI wrapper (cgi-wrap) will only allow form-mail to be called from a client (usually a browser) that is inside the SLAC Internet address range (134.79.*.*). You can get around this by tunneling into SLAC by logging onto SLAC via a Virtual Private Network (VPN) or using Citrix.
What happnes if I enter an invalid address to send the email to?
Usually the mail gateway will send an email back to the email address of the user of the form (usually you) indicating that the email could not be delivered.

[ Feedback ]
Les Cottrell