Home

| Home | Mailinglist | Download | License |

| Documentation | db/suite | db/common | db/base | db/relay | db/user | db/greylist |

qpsmtpd - Plugin db/greylist Documentation


NAME db/greylist

db/greylist - database-driven greylisting plugin

go top


VERSION 2007.1

go top


DESCRIPTION

This plugin implements the 'greylisting' algorithm proposed by Evan Harris http://projects.puremagic.com/greylisting/whitepaper.html

Greylisting is a form of DENYSOFT filter, where unrecognised new connections are temporarily denied for some initial period, to foil spammers using fire-and-forget spamware, http_proxies, etc.

See also http://en.wikipedia.org/wiki/Greylisting

go top


REQUIREMENTS

db/greylist requires the plugins db/common to be installed and db/base to be loaded.

It requires the Perl modules

 Qpsmtpd::DSN

It's tested with Qpsmtpd 0.32 and MySQL 5.0.32 on Debian Etch.

grey_table

A table is needed to store the triplet [ grey_remote_ip, grey_mail_from, grey_rcpt_to ] - e.g. (MySQL):

 USE `maildb`;

 CREATE TABLE `greylist` (
   `remote_ip` varchar(15) NOT NULL default '',
   `mail_from` varchar(255) NOT NULL default '',
   `rcpt_to` varchar(255) NOT NULL default '',

   `block_expires` datetime NOT NULL,
   `record_expires` datetime NOT NULL,
   `blocked_count` bigint(20) NOT NULL default '0',
   `passed_count` bigint(20) NOT NULL default '0',
   `aborted_count` bigint(20) NOT NULL default '0',
   `origin_type` enum('manual','auto') NOT NULL,
   `create_time` datetime NOT NULL,
   `last_update` timestamp NOT NULL default CURRENT_TIMESTAMP on update CURRENT_TIMESTAMP,

   PRIMARY KEY  (`remote_ip`,`mail_from`,`rcpt_to`)
 ) ENGINE=MyISAM DEFAULT CHARSET=latin1;

For the other fields see CONFIGURATION.

go top


CONSTANTS

Texts

These constants are only used in this package and aren't exported.

go top


CONFIGURATION

/etc/qpsmtpd/plugins

All other database plugins must follow the basic plugin db/base - e.g.:

 db/base
 db/relay
 db/user
 db/greylist

/etc/qpsmtpd/db_greylist

ParameterExampleOptional/Default
grey_tablegrey_table=my_tablegreylist
grey_remote_ipgrey_remote_ip=ip_fieldremote_ip
grey_mail_fromgrey_mail_from=from_fieldmail_from
grey_rcpt_togrey_rcpt_to=to_fieldrcpt_to
grey_block_expiresgrey_block_expires=exp_timeblock_expires
grey_record_expiresgrey_record_expires=rec_timerecord_expires
grey_blocked_countgrey_blocked_count=bloc_cntblocked_count
grey_passed_countgrey_passed_count=pass_cntpassed_count
grey_aborted_countgrey_aborted_count=abo_cntaborted_count
grey_origin_typegrey_origin_type=org_typeorigin_type
grey_origin_type_valuegrey_origin_type_value=serverauto
grey_create_timegrey_create_time=crea_timecreate_time
grey_last_updategrey_last_update=upd_timelast_update
grey_delaygrey_delay = 1060 * 5 (5 minutes)
grey_removegrey_remove = 60 * 60 * 260 * 60 * 24 * 36 (36 days)
grey_message_linkgrey_message_link=http://domain.tld/greylist.htmlNO default!
grey_regexgrey_regex=noyes

How the fields are used

The delays, the link and the regex

The delay entries tell db/greylist how long blocking and whitelisting is valid. The time values have to be in seconds. Expressions like

 seconds [ * minutes [ * hours [ * days ]]]

are accepted and will be evalued. All values must be > 0 !

/etc/qpsmtpd/db_greylist_regex

This is based on the idea of Chris Garrigues for reducing complex or single-used senders of mailinglists. Not using this would disable the whitelisting effect - e.g.:

The first three triplets are all blocked and later whitelisted, but will never be used again. So it's necessesary to fold them using the regex given below to the entry shown in the last line:

remote_ipmail_fromrcpt_to
63.251.223.186qpsmtpd-return-7369-user=domain.tld@perl.orguser@domain.tld
63.251.223.186qpsmtpd-return-7368-user=domain.tld@perl.orguser@domain.tld
63.251.223.186qpsmtpd-return-7367-user=domain.tld@perl.orguser@domain.tld
63.251.223.186qpsmtpd-return-*@perl.orguser@domain.tld

The configfile contains patterns to fold together an address into the same bucket when seen in the mail from field.

Each line contains two columns separated by whitespace; the first column is a regular expression to match in the email address and the second is what to replace that regular expression with. For example,

 -return-.*\@   -return-*\@

will effectively remove anything between ``-return-'' and the ``@'' in an emailaddress when building the triplet.

See db_valid_config and greylist_process.

go top


CONSTRUCTOR METHODS

init

Call: $self->init ( $qp )

Called from qpsmtpd on startup.

Calls isa_plugin('db/common') and db/common::init ( $qp ).

go top

db_init_config

Call: $self->db_init_config ( $config_fields, $config_fields_empty, $config_fields_default )

Called from db/common::init.

Sets the local config hashes. See /etc/qpsmtpd/db_relay.

go top

db_valid_config

Call: $self->db_valid_config ()

Called from db/common::init.

The config entries grey_delay and grey_remove are checked and calulated, if expressions are found.

Calls db/common::db_die on errors.

On errors in db_valid_config qpsmtpd won't start.

go top


HOOKS

hook_rcpt

Calls db/common::db_deferred ( 1 ), if sender is <> (null sender).

Calls greylist_process ( $qp->connection->remote_ip, $transaction->sender, $transaction->address ) otherwise.

Returns:

db/common::db_declinedcontinue, if sender is <> (null sender)
Result from greylist_processotherwise.
db/common::db_denysoft_erroron error.

go top

hook_data_post

Calls greylist_process ( $qp->connection->remote_ip, $transaction->sender, $transaction->recipients->address ) for each recipient, if db/common::db_deferred() = 1.

Returns:

Qpsmtpd::DSN->addr_bad_from_systemreject the message, log entry/message:
Empty sender not allowedif sender is <> (null sender)
db/common::db_declinedcontinue, if not deferred.
db/common::db_declinedcontinue, if not deferred.
Result from greylist_processotherwise.
db/common::db_denysoft_erroron error.

go top

hook_deny

If the previous plugin (which caused the DENY) is not this plugin, hook_deny increments the table field grey_aborted_count for each address in $transaction->recipients, if an entry is found.

Calls greylist_get_record and greylist_put_record.

Returns:

db/common::db_declinedcontinue.
db/common::db_denysoft_erroron error.

go top


HELPER METHODS

greylist_process

Call: $self->greylist_process ( $remote_ip, $mail_from, $rcpt_to )

Called from hook_rcpt or hook_data_post.

The specific methodology for a fairly basic greylisting implementation as proposed on http://projects.puremagic.com/greylisting/whitepaper.htm reads:

1.Check if the sending relay (or network) is whitelisted, and if so, pass the mail.
2.Check if the envelope recipient (or domain) is whitelisted, and if so, pass the mail.
3.Check if we have seen this email triplet before.
3.1.If we have not seen it, create a record describing it and return a tempfail to the sending MTA.
3.2.If we have seen it, and the block is not expired, return a tempfail to the sending MTA.
3.3.If we have seen it, and the block has expired, then pass the email.
4.If the delivery attempt should be passed and the delivery is successful:
4.1.Increment the passed count on the matching row.
4.2.Reset the expiration time of the record to be the standard lifetime past the current time.
5.If the delivery attempt has been temporarily failed:
5.1.Increment the failed count on the matching row.
5.2.If the sender is the special case of the null sender, do not return a failure after RCPT, instead wait until after the DATA phase.
6.Note: For all checks, we ignore records whose lifetime has expired

This algorithm is implemented in following order:

5.2.Wait until after the DATA phaseif sender is <> (null sender), see hook_rcpt.
1.Allow messageif sender is relay client, see db/relay::hook_rcpt.
2.Whitelisted recipientnot implemented
3.0Fold senderif grey_regex is used
3.1.Deny messageif triplet [$remote_ip,$mail_from,$rcpt_to] is unknown
3.2.Deny messageif block is not expired (grey_delay)
5.1.grey_blocked_count++if message is denied
3.3.Allow messageif record is not expired (grey_remove)
4.1.grey_passed_count++if message is allowed
4.2.grey_record_expires = NOW + grey_remove if message is allowed
6.Deny messageif record is expired (grey_remove)

The expiring of old triplets is controlled here by grey_remove. The cleanup cronjob is left as an excercise to the reader ;-)

On DENY, a formatted string using template GREY_MSG is returned - e.g.:

 Greylisted for 300 seconds (see http://domain.tld/greylist.html)

based on grey_delay and grey_message_link.

Calls greylist_get_record and greylist_put_record.

Returns:

db/common::db_declinedif whitelisted.
db/common::db_denysoftif greylisted.
db/common::db_denysoft_erroron errors.

go top

greylist_get_record

Call: $self->greylist_get_record ( $remote_ip, $mail_from, $rcpt_to )

Calls db/common::db_open.

Reads grey_table.

Returns: (three-state)

undefon errors.
record as reference to a hashif triplet was found.
reference to an empty hashif triplet was not found.

greylist_put_record

Call: $self->greylist_put_record ( $record )

Calls db/common::db_open.

Writes grey_table.

Returns: (three-state)

undefon errors.
0on failure - SQL statement result touches no record.
1on success - SQL statement result touches one record.

go top


CREDITS

Thanks to Ask Bjoern Hansen for qpsmtpd.

http://smtpd.develooper.com/

And Evan Harris for the 'greylisting' algorithm.

http://projects.puremagic.com/greylisting/whitepaper.htm

And Chris Garrigues for the regex idea.

http://www.trinsics.com/blog/?p=59

go top


COPYRIGHT

(c) Ernesto 2007, ernesto@dienstleistung-kultur.de

http://dienstleistung-kultur.de/qpsmtpd/

go top


LICENCE

As per the qpsmtpd license.

go top


This is a service of dienstleistung-kultur.de            Mailsystem QPSMTPD            Comments to Ernesto at ernesto@dienstleistung-kultur.de            Impressum