|NOTE: The documentation is structured around the format of the MPP GUI menu tree|
MPP Configuration File
The MPP configuration file, mppd.conf.xml, has become unwieldy so we are focusing heavily on our GUI. Configuration file will not be loaded, however mppd will stay running with the old working configuration. This accounts for a large percentage of support cases that we deal with. MPP will do it’s best to stay running in spite of a bad configuration. On the other hand, if there is a bad configuration and mppd is stopped, not reloaded, mppd will not load the configuration file and mppd will not start.
While hand editing the configuration file is possible please be careful. Read the DTD, which shows all XML options, to see the available options and use the GUI generated configurations as a model.
MPP Documentation Set
The complete MPP documentation set includes this manual, the MPP Complete Installation Guide, the Archive and Spam Quarantine Management Guide, release notes in your installation package and the knowledgebase found on http://messagepartners.com/esupport.
The status page provides basic operational statistics, message count, licensing information and shortcuts to configure the primary MPP services.
- Server status should be green if MPP is operational. If server status is Red it is advisable to check the logs of MPP to ascertain the reason that MPP is not running. Logs can be accessed from the shortcut on the status page or in Status > Monitor and Report > MPP Logs.
- This section shows information about the MPP license key and allows you to install a new key. Please note that MPP trial keys will mark each message with a scanned by MPP link. If you would like to trial MPP without the message marks please contact Message Partners or your local partner to obtain a trial key that will remove this mark.
Send Critical Alerts
- Enabling critical alert warnings at the bottom of the page will cause MPP Manager to send you an email if mppd is down or there are any critical errors. Since it is possible that your MTA will not be able to send emails in some failures it is highly recommended to send these alerts via some other SMTP host. This feature is very helpful for administrators and it is highly recommended to use this in most environments. Large installations should take care with this however to ensure that the analysis does not create too much load. If load is an issue you should run the script more frequently. This may seem counter intuitive but the more frequently the script is run the less work it does during each run. The script is run from cron.
- This section contains links to configure the primary services of MPP. Services show as enabled if they are configured in the default MPP policy group. Please see the Advanced > Policy Engine section for more details on policy configurations.
Shortcuts and Plug-in Updates
- This section provides shortcuts to common MPP tasks and an interface to update certain MPP Plug-in Modules
Content Filter Updates
- MPP provides an interface to update filter modules that are integrated in the MPP installation package.
- Please see the MPP Install Guide for more details on content module updates.
Monitor and Report
- This section contains critical statistics to monitor MPP’s performance. There is one important monitor option
- Statistics are gathered from MPP logs using a parser script that is automatically run a few times a day. The frequency can be changed in crontab for user root. To change the date range of statistics, force a rebuild or change the location where the parser script should look for statistics click the change date range link.
- Most tabular stats are self-explanatory, here are the less obvious explained:
|WBL Matched||Messages that matched a spam white or black list|
|RBL Matched||Messages that hit a Real Time Black hole list|
|Unauthorized||Messages that matched a content filter|
|Spamatraps||Messages from an IP that matched a spam trap entry|
|Client-Host BL||Messages that matched a client-host (SMTP) black list or auto-blacklist.|
Per Engine Statistics
- These are generated from the log parser script and give general guidelines as to engine hits but are not completely accurate due to configuration options. There are more accurate statistics available from Advanced > Message Tracking if you have enabled this feature.
- These statistics show how MPP is using the scanning threads configured in Advanced > Daemon. If you see many threads with zero counts or very low counts then you have configured too many scanning threads. Too many or too few scanning threads will impact performance so it is a good idea to monitor these statistics to look for an even distribution of scanning amongst the threads.
Top 5 Viruses
- Self-Explanatory, derived from scanning reports.
- These statistics are only available if Greylisting has been configured. Greylisting is only available when the Postfix Policy Server is enabled.
- This section provides graphical representation of the statistics. Graphs are only available if lib-gd and the GD perl module are installed. http://www.libgd.org/Main_Page.
- Configure this section to have MPP statistics sent to you via email on a regular basis.
- Enabling critical alert warnings will cause MPP Manager to send you an email if mppd is down or there are any critical errors. Since it is possible that your MTA will not be able to send emails in some failures it is highly recommended to send these alerts via some other SMTP host. This feature is very helpful for administrators and it is highly recommended to use this in most environments. Large installations should take care with this however to ensure that the analysis does not create too much load. If load is an issue you should run the script more frequently. This may seem counter intuitive but the more frequently the script is run the less work it does during each run. The script is run from cron.
- This section allows you view MPP logs in real time. You can filter for a specific string in real-time, such as a domain name, to only see log lines related to an area of interest. To view logs in real time select view, press stop realtime viewing to stop viewing the logs. If the stop process fails select clear log to clear the window contents and view again.
- This option provides a real-time view of the system mail logs.
- Use this section to search existing MPP logs. There is an option for sub-searches of query results.
- This section is used to delete, compress or archive log files on the MPP server.
The Services section provides an interface to configure the essential services of MPP. Since MPP is extremely flexible and configurable a concerted effort has been made to move less frequently configured items to ‘advanced’ sections to reduce configuration clutter.
MPP supports multiple configuration policies to apply configuration options to groups of users. More information is available in the Advanced > Policy Engine section. The dropdown menu on the top right hand of the screen shows which policy you are configuring. Most installations have one default policy.
| NOTE: For many configuration options there are options to store lists in the configuration file, in text files, in a MySQL database or a LDAP directory. Storing long lists is not practical in the configuration file, hence the option to store lists in text files. The downside of storing information in the configuration file or text files is that the lists are only loaded if the mppd daemon is restarted.
Store configurations in databases or directories to have changes take effect immediately or in configurable reload intervals or to share lists between multiple instances of MPP.
This section allows you to configure options pertaining to the SMTP connection. MPP has the most control over SMTP connections with the Postfix MTA, especially when the Postfix Policy server is enabled.
Block Connection/Never Scan Connections
- Add IP addresses or host names of SMTP servers that should be exempt from all MPP scanning or always blocked. This feature is known as ‘client-host white-black lists (CH-WBL’s). In most cases simply enter the IP’s or hostnames in the text boxes provided and save and restart to have your configuration options applied.
- If you have long lists of hosts to block or allow use a text file and enter the full path to the file such as /usr/local/MPP/mpp_wbl. If the file does not exist MPP Manager will create it. The format of entries in the file are host, value with one per line, for example:
- raeinternet.com, whitelist
- 184.108.40.206, blacklist
- 220.127.116.11, whitelist
- Use a database to store long lists that are shared among multiple instances of MPP or if changes need to be real-time without requiring a reload of MPP.
|NOTE: MPP has multiple levels of white and black lists. Client-host WBL’s exempt or block SMTP hosts from all scanning, per-user spam WBL’s exempt users from spam checks, optionally content filters. Additionally, SMTP level features such as thresholds, greylists and others have feature specific Whitelists to exempt from processing.|
- Use this section to enter Real Time Black Hole lists that MPP should check. Only use reject if you use the Postfix MTA, for all other MTA’s use discard or quarantine options to avoid being a source of backscatter email.
- Greylisting is a process where all SMTP servers that connect to MPP must first pass a ‘tempfail’ test before they can send. Many spam bots do not respond to temporary failure messages, thus they will never be able to send to your network. Greylisting can dramatically reduce spam but can also introduce delays in email processing that will annoy some. For example if you are waiting for an instant confirmation on an auction bid or reservation and this email is delayed by thirty minutes it can be quite frustrating. Luckily MPP has many controls to apply greylisting to groups of users or to exempt IP’s and domains from grey list processing, however, greylisting should be applied with this caution in mind.
- Greylisting has the following requirements:
- Postfix must be the MTA and the Postfix Policy Server must be enabled.
- A MySQL database must be available
- To configure greylisting enable the grey list feature and enter the database credentials for the MySQL server. Other options can generally be left alone and are self-explanatory.
- The Greylist Whitelist feature allows you list hostnames or IP’s that are exempt form Greylist processing and the URI option should be used for very long lists of exemptions.
- Spamtraps allow you configure email addresses or patterns as ‘traps’. Any IPs that sends to these traps is automatically placed in a temporary automatic client-host blacklist. MPP Manager has a GUI interface to configure simple regular expressions but the real power of spamtraps is unlocked with a full understanding of the construction of regular expressions.
- Spamtraps can be stored in text files for complex regular expressions, but this is not required in most cases. The simplest method to add a spam trap address is to select add new and then add a ‘wildcard expression. MPP will automatically create the correct regular expression syntax for a wildcard match of the value that you provide. For example raeinternet.com will match any host with raeinternet.com somewhere in the name.
- There is an option with spam traps, Spam traps for this group have global effect, that will add IP’s that match a group level spam trap to a global client host black list.
- After defining the spam traps define the time to block hosts that are matched and the storage method. Use RAM storage for small installations and MySQL is you wish to share the spam traps IP’s amongst multiple instances of MPP.
- Enter IP’s or hostnames to exempt from spam trap processing the spam traps white list section. If you have a long list of IP’s or hostnames to exempt then put them in a text file and define a path to the text file in the URI field, /usr/local/MPP/spamtrap. MPP will make the text file if it does not exist.
Sender Policy Framework
- Sender Policy Framework (http://www.openspf.org/) is a standards based method of verifying that host is allowed to send email for a domain. MPP supports SPF on a per-policy basis. When SPF is enabled MPP will query the DNS server for the sending domain to check for a valid SPF record and will take the configured action based on response.
- It is only recommended to take an action on spffail, using a condition on spfpas for example can cause sapm with legitimate spf records to not be scanned.
- If you want to take actions with spf states other than fail it is highly recommended to use SPF in conjunction with MPP Spam Scoring.
- To enable SPF set to enable and in most cases this should be enough. There are numerous options to customize for advanced users, must are self-explanatory, but a brief background discussion will help to understand the options.
SPF Advanced Options
- The following options are available for SPF results:
|quarantine||all further checks (spam, virus, etc.) will be skipped, message will go to quarantine and then ON_QUARANTINE_... will be performed.|
|reject||all further checks will be skipped, message will not be relayed further, reject reply will be returned to MTA|
|discard||all further checks will be skipped, message will not be relayed further, accept reply will be returned to MTA.|
|pass||all further checks will be skipped, message will be relayed further, accept reply will be returned to MTA.|
|markheader||all further checks will be skipped, header specified with SPF_HEADER option will be added to message, message will be relayed, accept reply will be returned to MTA.|
|scan||“do nothing” message will be scanned further as usual.|
|defer||all further checks will be skipped, message will not be relayed further, and temporary failure reply will be returned to MTA.|
|add_mpp_spam_score||value specified with corresponding MPP_SPAM_SCORE_SPF_... will be added to total MPP spam score, message will be scanned further as usual.|
- MPP users a binary, spfquery, from /usr/local/MPP/spf. This is invoked for each SPF query, however, optimizations have been made to reduce system overhead for this process. All options in the advanced section are provided to fine tune how this command is called. Defaults are recommended for most cases, however, slow network response times or low system resources may make it necessary to increase timeout values for read/write operations.
SPF And MPP Custom Spam Scoring
- SPF is integrated in MPP custom spam scoring to create aggregate scores form SPF results, spam scanner scores, RBL sites and content filter expressions.
This section controls how MPP reacts when a virus is detected by a plug-in module. Available options are discard, reject, and quarantine, mark header, disinfect and delete. Delete will attempt to delete a virus-infected attachment. Disinfect will attempt to clean the virus from a legitimate payload. Since many viruses can’t be disinfected or deleted as the emails to which they are attached contain no legitimate payload, as in the case of worms, these actions should be used with caution.
The antispam section contains many configurations related specifically to antispam protection. However, since antispam protection is a central application for MPP this is not a comprehensive place for all antispam configurations. For example spam traps and thresholds are effective antispam tools, however, they are configured in other sections that fall into the organizational structure of the configuration. For example reputation lists are an antispam measure but since they related to SMTP connections we put them into the connections area.
- Each MPP policy has three spam thresholds – high, medium and low – each with its own action. Possible actions are marksubject, markheader, forward, quarantine, reject, discard, and pass.
- Each MPP plug-in module submits a numerical score to MPP telling us the likelihood that a message is spam.
- In this section plug-in scores are correlated to MPP thresholds and actions are assigned to each threshold. These settings are applied to an entire MPP policy but spam actions can be applied to individual users in the subsequent configuration section.
- It is generally not a good idea to move the threshold scores from their defaults unless you have a very good reason to do it. If you find that an engine is not reliable at the low threshold it is better to set that action to pass or markheader rather than moving the threshold higher.
- In the case of the Co touch plug-in do not adjust the thresholds.
Understanding Spam Actions
|Pass||No action is taken on the message based on spam score and the message continues on processing path.|
|Discard||The message is silently discarded. Considered bad etiquette, but it sure can be convenient.|
|Marksubject||Renames the subject of the email with the subject of your choice.|
|Markheader||Adds a header of your choice to an email|
|Quarantine||Mail is stored in quarantine and discarded|
|Reject||Not recommended as the entire message is rejected and usually sent to a bogus sender|
|Forward||Mail is forwarded to an address of your choice|
|Note on Quarantine and Forward: By default email is discarded after it is quarantined or forwarded, however, this is configurable.|
|Note on MPP Headers: There are a few options for header additions. The markheader option here applies to a header applied to each message identified as spam. To control scanned by MPP headers check in Services -> Utility.|
Macros Available for Header and Subject Maker
|%RCPT_NAME%||the user id from the receiver's e-mail address before the ‘@’ symbol.|
|%SUBJECT%||the original subject of the message.|
|%RCPT_DOMAIN%||the domain of the receiver's e-mail address after the ‘@’ symbol.|
|'%RCPT%||the full e-mail address of the receiver.|
|%SENDER%, %SENDER_NAME%, %SENDER_DOMAIN%||are also available|
|%GROUP%||is the name of the group that the user belongs to.|
|%SPAMSCORE%||SpamAssassin will calculate a value based on numeric weightings of matches. Partial matches will have a value less than one.|
|%SCORESYMBOLS%||will be replaced with a number of symbols (default is *) equal with the difference between SpamAssassin current score and SpamAssassin max score between brackets, e.g. (****). The * symbol can be replaced using the parameter spam_spamassassin_subject_symbol. Only applicable in subjects, not in other message headers.|
|The %TYPE%||macro that expands to one of 'virus', 'spam', 'harass', ‘other’ replaces the SPAM_QUARANTINE_DIRECTORY and HARASS_QUARANTINE_DIRECTORY options from version 2.0.|
Per-user Spam Settings
- Within each policy group MPP offers per-user antispam settings to override the group actions. The available per-user actions are white and black lists and spam actions. Per-user settings can be stored in the MPP configuration file, in a text file (convenient for very long lists), or in a MySQL database.
- MySQL is the ideal choice for per-user storage if you have long lists of user preference\s and you require instantaneous changes. MySQL is also ideal if you have multiple MPP instances that need to share the same data.
- If you store per-user settings in a text file or in the default location MPP must be restarted for changes to apply. Restarting MPP is non-disruptive. If you don’t make many changes to per-user settings it is recommended to leave all settings in the default and restart MPP after you make changes.
- While it is possible to configure the per-user settings here, it is preferable to do this within the MPP quarantine manager for a few reasons. In the quarantine manager individual users can make their own changes or a domain administrator can make changes for there entire domain.
Per-user Spam Actions
- Define the domain or email address that the action applies to and then define the spam action. Again, it is recommended to make these changes within the MPP quarantine manager.
- Regardless of storage method a few conventions apply. Storage methods are configuration file, text files or database. Use database or text files for long lists, leave as default for all other applications.
- Mppd will always read WBL’s from the XML configuration file even if text files or MySQL storage is implemented. So if you have WBL entries in the XML configuration file and have also defined them in MySQL the WBL entries in XML will always be used and will take precedence over other methods.
- Possible actions for blacklist are mark header, reject, discard or quarantine. Possible actions for white list are pass or mark header. The default header for black list match is X-Blacklisted: TRUE, there is no default header for a Whitelist match.
- User – The email address for which the WBL entry applies.
- Contact – The contact affected by the WBL, think of an address book, wildcard (*) accepted
- Direction – Refers to user, if they are sender or recipient
- For example, my email address is firstname.lastname@example.org and I want to whitelist all email from Citibank.com.
- User = email@example.com, contact = citibank.com, direction recipient
- I am the domain administrator for company.com and I want to whitelist all outgoing email from spam checks.
- User = company.com, contact = *, direction = sender
- I am firstname.lastname@example.org and I want to blacklist all email from myex.com
- User = email@example.com, contact = myex.com, direction = recipient
Customer Spam Scores
Custom spam scoring is a major MPP feature module that allows you to create aggregate scores from MPP tests. Currently custom spam scoring supports plug-in module spam scores, RBL matches and SPF results as scoring metrics. Here are some common reasons to use MPP custom spam scoring.
- Mark a message header if there is an RBL match rather than make a reject decision.
- Aggregate the scores of multiple spam engines to make a quarantine decision
- Aggregate SPF, RBL and content scanner results to create one score.
- Create custom spam scores based on content rules.
- Create content based white or black lists
The basic idea behind custom spam scoring is that you create an aggregate score for each threshold, say 10 for high, 8 for medium and 6 for low. Then you assign a point value to a test. For example, a Cloudmark high score is 7 and a Spamhaus RBL hit is 3. Now an email is required to have both a Cloudmark high score and an RBL hit before MPP will make a spam decision.
Integrating Content Engine and Spam Scoring
- There is incredible flexibility and power when the content engine is combined with custom spam scoring. The idea is that any content rule expression can have a positive or negative spam score. This functionality allows content based white or black lists or to create your own custom spam rules. This feature is especially useful in situations such as an outbreak of spam that filters have not caught yet or you never want email with an attachment named customer_invoice.pdf to be marked as spam.
- Spam scores for content expressions are configured in the content inspection section of MPP Manager.
Spam Score Headers
Use this control to add the spam score , from an anti-spam plug-in or from MPP custom spam scoring, to each spam email. You may customize the name of the header or use our default X-Spam-Score.
Add SpamAssassin Rule Hits
- This control allows you to insert the complete SpamAssassin score report in the header of an email. We do not recommend enabling this feature because performance degrades considerably, but it is an available option.
Alternate spam engine subject symbol
- By default, if you are using the %SCORESYMBOLS% macro as a subject mark then MPP will put the * character for each number of the spam score. For example, if the spam score is 20 then 20 *’s would be inserted in the message subject. Use this option to use a character different than * as the symbol.
The Archival section controls where email is stored and fine-tunes which email stored. There are many archive controls in the MPP Archive Review application, QReview, such as import controls, user access, searching and more. Please review the Archive and Spam Quarantine Management documentation to fully understand the MPP Archive solution.
Location of Email Archive
- MPP email archives can be stored as files, in a database or in a hybrid metadata approach.
|NOTE: For single systems it is recommended to use the metadata storage model for email archives. In this model file locations are stored in a database while the actual mail ‘mailstore’ storage and configure metadata storage on the remote instance of MPP. See appendix for more details.|
Archive to Directory
- When this option is enabled all email is stored in the file system as MIME encoded files. The file path is constructed with macros as shown below. File paths can be scaled with %RAND16% and %RAND64% macros. These create randomly named directories that can be nested to avoid file system limits on the number of files within a directory.
|NOTE: It is mandatory to use the %RCPT_DOMAIN% macro for the archive review application to work correctly. It is not recommended to use file f|
|'%RAND16%||generates random number between 00 and 0F for use in directory names|
|%RAND256%||generates random number between 00 and FF|
|%RCPT_NAME%||the user id from the receiver's e-mail address before the ‘@’ symbol.|
|%SUBJECT%||the original subject of the message.|
|'%RCPT_DOMAIN%||the domain of the receiver's e-mail address after the ‘@’ symbol.|
|'%RCPT%||the full e-mail address of the receiver|
|%SENDER%, %SENDER_NAME%, %SENDER_DOMAIN%||are also available|
|%GROUP%||is the name of the group that the user belongs to.|
|%TYPE%||macro which expands to one of 'virus', 'spam', 'harass', ‘other’|
|%UID% UserID, same as %RCPT_NAME%||except that user should be a Unix system user. If the user doesn't exist it will be replaced with "root".|
|%GID% GroupID, same as %RCPT_NAME%||except that user should be a Unix system user. If the user doesn't exist it will be replaced with "root".|
- Here is an example directory location:
- Email stored on the file system cannot be indexed for full text searching as of this writing.
Archive to MySQL Database
- Define the MySQL database permissions for your message store and define if the entire message is stored in the MPP message database or if only ‘metadata’ is stored in the database while the entire message is stored on the file system.
|NOTE: Storing the complete message in MySQL is the recommended archive storage message method. MySQL replication is strongly recommended for scalable storage.|
- If you opt to use metadata then information about the message is stored in a database while the actual message is stored on a file system, hence, you must define a path to store the email on the file system.
- If you are storing the complete message in MySQL set the max storage size equal to the largest field size supported in MySQL. Messages that exceed this amount will be stored on the file system. If you have many large files file system or metadata storage may be more appropriate than complete MySQL storage.
Archive to Maildir
- Maildir storage is identical to file system storage, however, email is stored in the Maildir format. This is useful if you want standard MUA’s such as Squirrelmail to access MPP email archives. Besides directory name there are two additional required fields – owner and group for the messages, %UID% and %GID%
- Sample URI’s
Archive to Message Store
- MPP has a hierarchical message storage model that can be deployed in larger sites. Even for smaller sites it makes because in this model MPP on your MTA will never have a direct connection to your database. This is important because large emails or database failures can impede mail flow if MPP must wait for the database to complete a write.
- In the hierarchical model a remote instance of MPP will use ESMTP to transport email messages to a remote MPP that maintains a direct connection to the database. This way any failure or slowness in the database will not affect email flow.
- The ‘remote’ instance of MPP can be on the same server or on a different server. If the email store is on a remote server use this type of socket identifier:
- If the message store is on a separate instance of MPP on a local server use this type of socket path:
- If you are considering this model of storage it is recommended to work with MPP support. This is a very scalable model for MPP archive storage.
- Setting up the ‘remote’ instance of MPP, or message store, is handled in the advanced section.
Archive Message Settings
- Archive MS Exchange Journaled Email is an analog for MPP option of archive with with/without SMTP envelope. MPP supports Message Journaling as found in Exchange 2003 and other journaling schemes such as email sent via Postfix always_bcc.
- When this option is set to yes MPP will archive messages with information from the SMTP envelope. When this option is set to no, MPP will extract the receipt/sender information from the actual message rather than from the SMTP envelope.
|NOTE: When archiving journaled email set ‘Action on Archive Success’ to discard to avoid the redelivery of messages after archival.|
Archiving MS Exchange 2007 Email
- MS Exchange 2007 introduced new Journal formats and they are supported by MPP. When this is enabled MPP will archive the journal reports with correct sender/receiver information, however, the actual message will be stored in TNEF format, just as Exchange encapsulates the message.
Archive Other Message Types
- These commands are self-explanatory - archive infected, spam, malformed, etc.
- Set on archive success to discard when you are archiving journaled email, otherwise leave as the default.
Secure Archive with Digital Signatures
- MPP supports the ability to securely archive messages with a hash signed by a digital signature that ensures that a message in archive has not been modified. qReview can verify the hash value and digital signature of a message as it is viewed. This feature requires that the openssl library.
- In order to take advantage of MPP’s secure archive feature MySQL must be utilized to store the signatures and messages. Secure archive is compatible with both complete database message storage or metadata mode. As always, metadata is the preferred and recommended storage model.
Secure Archive Implementation in mppd.
- Currently these steps must be performed manually.
- Generate DSA parameters param_4096.pem file for 4096-bit key:openssl dsaparam -out param_4096.pem 4096
- Generate private key file prv_archive_4096.pem from DSA parameter:openssl gendsa -out prv_archive_4096.pem param_4096.pem
- Generate public key file pub_archive_4096.pem from private key file:openssl dsa -in prv_archive_4096.pem -pubout -out pub_archive_4096.pem
- Define DSA Engine in mppd.conf.xml:Administrator defines an instance of DSA Engine in mppd.conf.xml file under <mppd><engines> node with node name <dsa>. ID if an instance of the engine is defined with id attribute of the node. DSA Node Structure is described further in “Structural View” section.
- Use DSA Engine:Engine can be used by specifying its ID in any context that supports this. Engine that is not used is not actually loaded.
Example code in mppd.conf.xml
<mppd> <engines> <dsa id=”0”> <key_id>0</key_id> <private_key_file> <path>/usr/local/MPP/prv_key.pem</path> <format>pem</format> <password_id>prv_key_password_id</password> </private_key_file> </dsa> </engines> </mppd>
- In order to activate secure archive within a policy add the following command to the policy:
- Group-level XML option that specifies signer engine ID to use for digitally signing archived messages. If is not specified or empty then signing will not be performed.
Description of node values and attributes
|<key_id>||Specifies||ID of that identifies and instance of private-public key pair. It will be available for other parts of MPP that uses DSA engine and will be stored along with signature wherever appropriative.|
|<private_key_file>||Specifies||Path to private key file and file properties|
|Value||breaks further on sub-nodes|
|<path>||Specifies||Path to private key file on local file system.|
|Default||not applicable (value must be specified)|
|<format>||Specifies||Format of private key file.|
|<password_id>||Specifies||Encryption pass-phrase if key file is encrypted. Refer by id to a password from a passwords file defined by PASSWORDS global option. If this option is not specified or empty the it is assumed that key file is not encrypted.|
|Value||string without spaces|
Importing Email into MPP Archives
- MPP can import email from MIME Files, IMAP Message Stores or MBOX files. IMAP import is managed from MPP Manger in the Archive>Import section. File import is managed from an available script which can be obtained from support.
- Content filtering is a major feature module of MPP and is supported by our own content filter engine. The content filter module has been completely redesigned in MPPv4 to be more efficient and far more flexible.
- Some uses of MPP content filtering include…
- Surveillance – Forward all email that matches some criteria to a supervisor.
- Quarantine – Quarantine all email that matches a filter and deliver the message or stop it for review.
- Blocking – Block email that matches some criteria from leaving our entering your organization
- Routing – Route email based on content to some department or mailbox for processing
- The content filter module is one of the most powerful and flexible modules of MPP and there are endless possibilities to use content-based controls with MPP.
- The key to MPP’s powerful content filtering module is the breadth of expressions and matching algorithms that we support combined with content filtering actions and MPP policies. Since MPP content filter policies can be applied to groups of users they can be applied to small groups within your organization.
Content Filter Actions
- Messages that match a static content filter can be forwarded, delivered, rejected, quarantined or discarded.
- Header filters are applied to all message headers, including the subject.
Message Content Filtering
- Message content filtering is applied to text within the body of a message.
- MPP supports filtering by attachment name, but since this is typically used for extension blocking this capability is found in the Utility > Extension Blocking section. MPP can filter within text based attachments, please contact support for more information regarding this capability.
- All MPP filters used regular expressions, however, the GUI obviates the need to understand regular expressions for most cases. As outlined previously there are tremendous capabilities to construct complex expressions with MPP, but the GUI will suffice for most cases.
- Long regular expressions can be stored in text files defined in the ADVANCED section of the content inspection page.
- To add a regular expression using the editor clicks SHOW next to ADD EXPRESSION. There are two options available – to use a wildcard expression or create your own expression. The wildcard option requires no knowledge of regular expressions and there is a drop down of common message headers to filter on.
- For example, if you wanted to block all email with ‘sex’ in the subject you would select the wildcard filter, select the Subject: header and type sex in the text box. If you want to match words like Viagra, but you know that the spelling is usually off, you can try via*. The help pop-up provides some useful examples.
- You can add expressions that don’t use wild cards, such as email address in the Regexp filter section. MPP will add the regexp symbols corresponding to case sensitivity, single or multiple line matches.
- In most cases these are sufficient for filters, however, if you are comfortable with expressions you can select SHOW REGEX LIST FILE and modify the regexp file manually. Again, see the Appendix for more details on complex expression construction.
Access Control Lists
Access Lists or ACL’s are a powerful module of MPP that allow you to limit the senders or receivers of email on your email system. This is useful for a number of applications:
- Check valid recipient lists
- Limit employee communication
- Quarantine email sent to or from certain locations
- Reject email from unauthorized sender/receiver pairs
- Create ‘Chinese Firewall’ types of environments
ACL’s can be stored in the MPP configuration file, text files, MySQL databases or an LDAP directory. Use text files for long lists and databases or directories for long lists that are very dynamic. The default storage is fine for most applications.
There are two types of matches for ACL’s. Senders must be on the list, or they must be off the list, in other words, either exclusive or non-exclusive to the list.
Actions for ACL violations include quarantine, discard or reject.
MPP can limit by sender, receiver or sender or receiver. Sender or receiver is seldom used.
The format for ACL text files is address,direction, one entry per line. It is recommended to only use recipient ACL’s and specify direction in the ACL list – either in or out.
Example ACL file:
The utility section controls features that relate to MPP as a general-purpose email processing utility.
- Signatures are text that is appended to each email. Signatures can be stored in text files or in the MPP configuration. The MPP configuration is fine for most cases. MPP supports both any character encoding of signatures, however, the character set of the email must match the character set of the signature. If an email is ASCII encoded and you have defined a UTF-8 signature the signature will not be applied in order to preserve the integrity of the email message.
- MPP takes care to preserver the integrity of a message when inserting a signature. Signatures are added in the message for text-formatted messages and as a new body part for multipart messages. You may use HTML code in the signatures but HTML code will not display properly in text-formatted emails.
- How to use a text file for signatures:
- Define the name of the text file in the text box next to ‘Append contents of file,’ and then save the configuration.
- Edit the file
Add MPP Header to Scanned Emails
- If enabled MPP adds one header for each content module that scans a message:
- X-Scanned-By: MPP/Cloudmark http://www.messagepartners.com
Custom header added to every message
- The custom header can be tailored to suit your needs and it is added to each message.
Maximum File Size to Scan
- Maximum size to scan and action if file exceeds this value. There is a corresponding configuration option in Advanced->Scanners, max file size for spam engine to scan.
Maximum Recursion Level
- This controls how many levels of MIME recursion MPP should scan. There are DoS attacks with thousands of levels of recursion.
Messages Without Recipients
- Some worms and viruses will send invalid messages without recipients in the messages, this option controls how these are handled.
- Empty messages are a byproduct of worms and viruses and this option controls how they are handled.
- To enable attachment-blocking enable the radio button and click save. Upon save a new text box will appear where you can enter a comma-separated list of extensions to block: exe, mp3, com
- Attachment stripping, or body stripping, is a major feature of MPP. The idea behind bodystripping is to remove a message part such as an attachment or MIME Type and replace it with a link to the file. This is very useful for a number of applications:
- Stripping and storing large attachments from outgoing mail, thus avoiding attachment size limitations at a remote site.
- Removing reports from automated emails and placing in processing queues.
- Allowing monitors to examine outbound attachments
- Removing potentially malicious files from incoming email
- To enable bodystripping define a pattern to match , a storage path and a retrieval link temlate. For example, you can configure that all .pot files over 10MB are stripped from emails and saved on an ftp or http server and the attachment is replaced with a link to the stored file.
- Here is an example that illustrates body stripping
- User A sends 25MB PowerPoint file to Users B C and D, thus creating 4 copies of this file; 1 each for Users B, C and D and 1 in the sent items of user A.
- With body stripping enabled when user A sends the 25MB attachment it is replaced with a link to a user specific ftp directory. Users B, C and D see a link to the file, but only access the file if they need it. After the attachment is stripped it is no longer a part of the end-users email corpus and never enters their mailbox files
- Steps to Enable Body Stripping
- Define minimum file size to match, in bytes. E.g. 100000 is 100kb.
- Define regular expressions to match the name of MIME type or MIME part name to strip.
- /.*\.pdf$/ will match file name.pdf, all files with .pdf extension
- /pdf/ will match pdf anywhere in the file name
- Here is a complete regular expression list of extensions: /(.*\.gif$)|(.*\.jpeg$)|(.*\.bmp$)/i
- Specify the location where the MIME part should be stored: file:///usr/local/MPP/attachments/$FILENAME$:apache,apache,0644
- Specify the location where users should retrieve the files
- MPP takes care to insert the retrieval link into the email with the correct MIME type. Hence, there are three formats to choose – HTML, Text, External-Body. You may define the order that the file types are attempted or leave as default.
- There is the option to define both an HTML and Text version of the text that is inserted into an email after stripping.
- Body part of type $CONTENTTYPE$/$CONTENTSUBTYPE$ was stripped by MPP.
- It can be found at $LINK$
- Body part of type $CONTENTTYPE$/$CONTENTSUBTYPE$ was stripped by MPP.
- It can be found at <a href="$LINK$">$LINK$</a>
- As in other places, MPP can store this information in text files or in the configuration. Text files are suitable for longer messages. If you wish to use text files, specify the path, save the configuration, then edit the file.
|$SYSTEM_DOMAIN$||name of the current system|
|$SENDER$||full sender string (I. e. firstname.lastname@example.org).|
|$SENDER_NAME$||just sender name (i. e. name for email@example.com).|
|$SENDER_DOMAIN$||just sender domain (i. e. domain.com from firstname.lastname@example.org).|
|$MESSAGEID$||message id as extracted from message headers or, if absent, generated at runtime.|
|$RAND16$||random hexadecimal number between 00 and 0F presented as string. This number is stripping-session-wide one, i. e. it is the same for when generating URL's from STRIP_BODY_URI_SAVE and STRIP_BODY_URI_LINK options for one body part (but different for another body part).|
|$RAND256$||random hexadecimal number between 00 and FF presented as string. This number is stripping-session-wide one, i. e. it is the same for when generating URL's from STRIP_BODY_URI_SAVE and STRIP_BODY_URI_LINK options for one body part (but different for another body part).|
|$CONTENTID$||content ID of body part as extracted from headers or, if absent, generated at runtime. This is equal to Content-ID field putted with External Body method to headers of link.|
|$CONTENTTYPE$||MIME primary content type (i. e. image from image/gif).|
|$CONTENTSUBTYPE$||MIME subtype (i. e. gif from image/gif)|
|$FILENAME$||name of body part (filename for attachment) as proposed by message headers or, if absent, generated at runtime. This macro should always be present as last component of both STRIP_BODY_URI_SAVE and STRIP_BODY_URI_LINK. Also it should not be combined with other strings and macros (i. e. file:///dir/file-$FILENAME$) because of the limitation of algorithm for generating unique file names and further substitution it to link.|
|$SIZE$||size in bytes of decoded body part.|
|$LINK$||special macro available only within text and html templates. Macro is substituted with ready URL obtained by substituting other macros into STRIP_BODY_URI_LINK. This macro MUST be used in templates instead of trying to form link from other macros as for STRIP_BODY_URI_SAVE and STRIP_BODY_URI_LINK options.|
- $TIMESTAMP$, $YEAR$, $MONTH$, $DAY$,$HOUR$, $MINUTE$ ,$SECOND$
- Macro values are validated before substitution to not contain potentially dangerous characters and character combinations that might come from message as following:
- each character '/' is substituted with 'x';
- standalone strings “.” and “..” are changed to “x” and “xx”;
- all characters that must be encoded (including control characters) for URL are encoded when macro value is substituted into link URL;
The advanced section of configuration contains many important elements of MPP regarding message processing.
The core component of MPP is mppd, the daemon process of MPP. This section configures critical elements of daemon operations.
|Info||s the recommended log level and minimum required logging level for parser scripts to works|
|Debug Data||Extremely Verbose|
|Critical||Crashes and Sever Warnings|
|Error||Virus infections, errors and warnings|
Advanced Logging Options
- By default MPP logs are stored in /var/log/MPP, one file per date. MPP logs can be broken up by size, time and other variables and can also be sent to a syslog facility.
Type (or engine) to use for logging
- Dirlog - Default
- Syslog - Syslog mail facility is used
|NOTE: All of the following options only apply to dirlog style logging, not syslog.|
Base directory to put log files to
- Self explanatory, /var/log/MPP is default.
Template to generate log file name and location
- Default log name is $YEAR$$MONTH$$DAY$.log,
- Variables for custom names and locations:
|$YEAR$||periodical macro that stands for current year (four-digit integer).|
|$MONTH$||periodical macro that stands for current month (two-digit integer).|
|$DAY$||periodical macro that stands for current day (two-digit integer).|
|$HOUR$||periodical macro that stands for current hour (two-digit integer).|
|$MINUTE$||periodical macro that stands for current minute (two-digit integer).|
|$SECOND$||periodical macro that stands for current second (two-digit integer).|
|$S_YEAR$||macro that stands for log starting year (four-digit integer).|
|$S_MONTH$||macro that stands for log starting month (four-digit integer).|
|$S_DAY$||macro that stands for log starting day (four-digit integer).|
|$S_HOUR$||macro that stands for log starting hour (four-digit integer).|
|$S_MINUTE$||macro that stands for log starting minute (four-digit integer).|
|$S_SECOND$||macro that stands for log starting second (four-digit integer).|
- Examples: /var/log/$YEAR$/$MONTH$/$DAY$.log
- Log for August 1, 2008 would be stored in
Time for log file to be closed and new one started
- By default one log is generated per day. When a time value, in minutes, is entered in this option MPP will create a new log file every x minutes. MPP adds a .1 to increment log files for a single day.
- For example if new logs were created every 12 hours using the macro from the above example, they would appear this way in the log directory.
Size limit for log file to be closed and new one started
- Specify size in Kilobytes when MPP should create a new log file.
Whether to start new log when mppd is restarted
- This option will create a new MPP log each time mppd is reloaded.
- Message Tracking is a major feature of MPP that provides a compact record of every stage of message processing in a MySQL database. With message tracking you can derive detailed statistics about any message, find the result of any message that was scanned, search for message by remote relay server, message ID or many other elements.
- Message Tracking is enabled by default on the MPP Virtual Appliance and it is highly recommended for all installations that require detailed statistics about message processing.
Searching by domain
- To search by domain put the @ sign in sender or receiver, so @raemail.net will show all email for the domain.
- Wildcard delimiters are automatically added so searching for mike will search for all email to mike*.
- Message tracking queries are case insensitive, hence queries for mike will find Mike, MIKE, etc.
MPPD Configuration File
This section allows you to view or edit the XML configuration file, mppd.conf.xml, restore a default configuration or import a configuration from a TFTP server.
MPP Daemon Options
Number of threads
- This value refers to the number of threads that mppd has available to scan messages. In general two scanning threads are sufficient, however, busy sites will need to increase this value. Thread counts should not exceed 20 unless there is a very good reason to do so.
- It is recommended to look at the thread utilization statistics in Status > Monitor and Report to look for an even distribution of messages amongst scanning threads. Too many scanning threads will degrade performance while too few will do the same. Slower plug-in modules such as SpamAssassin will require more processing threads.
- These scanning threads are distinct from threads that MPP uses to process SMTP or LMTP messages such as in the Postfix Policy Server or Content Filter implementation.
Start Daemons on Startup
- This configuration option controls if MPP should start the daemons of plug-in modules on start-up. F-PROT, Clamd, Spamd and Commtouch can all be started by mppd.
- MPP will start spamd with user nobody and all SpamAssassin prefs, databases and bayse filters will be associated with this user. If you prefer to start spamd under a different user than disable this feature and start spamd as you normally would.
Unix owner and group
- Changing the user and group of MPP is an involved process as many ancillary changes must be accounted for. Please consult support if you require changing the default ownership of mppd.
Location for MPP magic number
- The ‘magic number’ is a value that mppd uses to mark messages that should not be scanned for some reason, such as alerts, forwarded messages, etc. The magic number is recomputed when mppd is reloaded.
- If you are using MySQL spam quarantine then it is advisable to store the magic value in the same database as the spam quarantine. This way, when messages are released from quarantine the correct value will be tagged in the message headers.
- If you have multiple instances of MPP only one should store it’s magic in the spam quarantine database – the instance of MPP that will scan a message that is released from quarantine.
Enable Internal Queue
- The MPP internal queue is used when messages are handed to MPP for processing by SMTP and other processes. With the internal queue enabled messages are written to a disk queue before heavy processing if resources are not available for immediate processing.
This section is a consolidated configuration center for all MPP alerts. Alert sections and conditions are self-explanatory in GUI. There is one template available for sender, receiver and admin for all alert messages, with the exception of threshold alerts.
The macros available for warning templates are:
|%FROM%||will be replaced with the e-mail address that the warning is being sent to.|
|%TO% and %BCC%||Will be replaced with the sender of the infected message if this is the template for a sender, or the receivers of the infected message if this is the receiver's template, or "postmaster" if this is the admin's template.|
|%SUBJECT%||will be replaced with some program defined subject text.|
|%REPORT%||will be replaced with scanning report and actions that have been taken.|
|%REPORT_HEADERS%||will be replaced with full mail headers of the original message.|
Scanners to Use
- Set the order of scanners in this section. Scanners appear in drop down menus if the components are installed on the system.
- This section is to configure how mppd reacts to various conditions. These options are also available in Services->Utility.
- This section controls how MPP reacts to various scanning error conditions. The action of Defer is only available with Postfix.
| NOTE:Pay close attention to the action configured for ‘when a scanning error occurs’. Scanning errors can occur for many reasons but the most common reasons are SpamAssassin timeouts and out of date Sophos monthly virus definitions (804021e error code). If the error action is set to quarantine then all messages will be quarantined on the event of a scanner error.
It is recommended to enable warning messages for admins for scanning errors and set this action to defer or pass.
Configure Scan Engine
Max file size (KB) for spam engine(s)
- Use this variable to fine tune how much data sent to a spam engine. Most spam engines only need the first few bytes of messages. Generally this variable does not need to be changed.
Scan inside archives and compressed files
- This option applies to virus scanners only.
Maximum file size to scan (in MB)
Maximum recursion level
- This setting controls how many MIME recursion levels MPP will descend into a message. This is meant to avoid certain types of DoS attacks.
Scan engine socket
- This is an important section of MPP configuration and in most cases it need not be touched. However some spamd, f-protd and clamd all support TCP sockets, hence, they can be located on remote servers.
- MPP has a round-robin protocol for load balancing amongst multiple instances of spamd, either locally or remote. To add a new instance of SpamAssassin click add new to create a new socket definition.
Scan engine timeouts
- Use this section to fine tune scan engine timeouts. This setting generally only needs to be modified for SpamAssasin, which is notoriously slow. A read and write timeout of 30 seconds (30000 milliseconds) is required to keep SpamAssasin from timing out on message scans.
|NOTE: SpamAssassin timeouts are the most common scanning errors and they are fixed by adjusting this parameter to at least 30 seconds.|
- Optimizations are settings to control the serialization of scanners. There are a few factors that control how serial scanners are handled. Scanners are processed in the order of configuration. It is generally desirable to use spam scanners first. Since the majority of email in today’s environment is spam why waste system resources on scanning spam for viruses? Of course there are many arguments why to place virus scanning first, but we recommend spam or content scanning first.
Virus Scanner Optimizations
- Use this to configure the behavior of subsequent scanners after a virus is found. Options are to skip spam scanners or content scanners.
Spam Scanner Optimizations
- Use this option to configure how subsequent scanners behave after a scanner has found a violation.
|NOTE:MPP Custom Spam Scoring allows you to create aggregate scores from spam scanner results,RBL tests, SPF tests and content filter expressions.|
ESETS Product ID
- The default value is 0×71330000.
- Threshold or Auto Blacklists are used to automatically detect senders that are out of compliance with policies that you define. Thresholds track message counts and spam counts over a sample period. If a sender violates a message rate then you can take action against them – either warn an admin or block them for a specified period of time.
- The values of a threshold are - Message Count, Clean Count, Violation Count, Sample Time and Time in Cache. Violations include spam, content violation, and access-control- list violations.
- Since many botnet attacks are from many sources, each sending the same message, MPP can group SMTP hosts by subject. So if one SMTP sender violates a threshold, then all senders that send with the same subject will be blocked.
- It is very useful to use access list violations as criteria for threshold matches when MPP is performing a recipient verification. MPP can read valid recipient lists form any mysql database or text files. MPP can issue immediate SMTP disconnects to dictionary attackers who guess too many times when this feature is utilized in combination with the Postfix Policy Server.
- Many people use MPP thresholds to find outbound spammers or heavy senders that may lead to their email servers becoming blacklisted. For applications such as these define a threshold with zero spam count, a high message rate and a short sample period. Any sender who is sending too many messages will now be found quickly.
- Valid threshold actions are block or alert. When a sender is blocked they are placed on a temporary Client-Host blacklist. If Client Host Blacklists (from Services > Connection) are configured to use MySQL then blocked hosts will be stored in a database. If you have multiple MPP instances, they can all share the same database to block the same hosts. If a database is not defined for Client-Host blacklists then auto-blacklist entries are stored in RAM. If the Postfix policy server is in use the hosts are immediately disconnected.
- Threshold alerts are unique and allow you to define the senders and receivers. On a busy mail system there will be many warning messages and generally the warnings are sent to automatically monitored mailboxes as they can very rapidly. Threshold alert text cannot be customized.
- Thresholds have their own white lists to exempt hosts from threshold checks. Whitelists are entered as CIDR IP addresses, such as 192.168.100.0/24 or as host names.
The Message Stores section is a consolidated location to define email archives and quarantine locations.
Message tracking is an important feature for MPP. Like thresholds, reject templates and per-user WBL’s it is another feature for a big deal that never materialized. Message Tracking keeps a very compact record of each message that MPP scans in a MySQL database.
Message tracking is designed to scale for very large message counts and care has been taken to store the minimal amount of information. There are many configurable options to custom tailor how MPP stores tracking data about different facilities of MPP.
Message tracking makes it possible to query message status by message id, scan state, remote relay server, domain, email address and many more variables. Message tracking also gives detailed statistics that can be grouped per-user, per-domain, per-engine, by scan state, by scan result and more.
For each MPP scanning stage and result MPP can store no information, basic or detailed statistics. If Message tracking is enabled MPP will store detailed information about parsed, quarantined, quarantine_failed, archived, archive_failed, relayed, relay_failed,queued, queue_failed and forwarded. (back to the top)
Other facilities that can be enabled are as follows in the common section of mppd.conf.xml:
- e.g. <TRACK_MAIL_TRANSFER_ACL>detailed</TRACK_MAIL_TRANSFER_ACL>
- Message tracking reports are very accurate accounts of the activity of your email system. Each entry of a message-tracking event can be expanded to show the result of every recorded processing stage.
- Valid search criteria in the advanced section are email, domain, message ID, remote relay server, date, time, message events and results. Future releases will include more reporting options. A single message can have multiple events, such as discard and quarantine. (back to the top)
- Spam, infected, disinfected, mpp_spam_score, parsed, disinfection_failed, deleted, deletion_failed, harass, rbl, rbl_Failed, unauthorized, encrypted, malformed, max)recursion, max_file_size, quarantined, quarantined_failed, archived, archive_failed, acl, aclent_wl, client_bl, client_df (deffered), client_dns_failed (could not resolve IP for hostname), group_wl, group_bl, user_bl, user_wl, error, relayed, relay_failed, queued, queue_failed, no_recipients, empty_message, forwarded, ldap_failed, magic_found
- Every message will have a parse event and most will have a relayed event.
- Passed, discarded, deferred, rejected.
Reject templates are another one of countless MPP features designed to win one big deal or another. This feature is actually very cool and one of the truly unique features of MPP. On the surface customer reject templates do what they say they do – they allow you to create custom rejection notices that contain detail about why message was rejected (550 SMTP Code). This is useful if you want to customize SMTP rejection (550) notices to tailor some business or technical need. However, Reject Templates can also be used an API to MPP, allowing MPP to function as a stand-alone SMTP scanner.
Reject notices are only useful for pre-queue features such as rejections from the Postfix policy server or Sendmail Milter interface. Sending post-queue rejection notices is very bad practice as you will be bouncing complete messages, thus creating potentially huge basckatter problems.
If you can’t reject a message before the message is queued then don’t use the reject action.
Reject Condition Templates
- Templates are constructed in two parts, a mandatory part and text customized each of the following conditions:
- virus condition
- spam condition
- unauthorized condition
- error condition
- maximum recursion condition
- maximum size condition
- acl condition
- encrypted condition
- malformed condition
- blacklist condition
- RBL condition
Reject Notice Construction
- The mandatory part of a rejection notice will always be present, while one or more conditional parts will follow after, if the corresponding condition is true. Conditional parts are separated with spaces.
- For the SMTP protocol, there is one response for all recipients. If the client initially specified multiple recipients, and they were processed differently, and different rejection text should be reported for at least one of the recipients, then rejection text will be combined as follows:
- [text for recipient 1][text for recipient 2][text for recipient n]
- Text for each recipient will be in "" brackets - Otherwise, there will be single text for all recipients without brackets.
- The LMTP protocol provides one response for each recipient, so the described scheme is not used.
- The following predefined macros can be used specified within mandatory or any of the conditional parts:
|%INTERNALID%||Internal ID of the message, this is to search in MPP logs for transaction flow.|
|%GROUP%||MPP group that was used to scan the message.|
|%STATE%||Substituted with a comma-separated list of scan states. Here's the full list:virus,spam,harass,unauth,error,max_recursion,max_size,acl,encrypted,malformed,bl,rbl.|
|%VIRUSLIST%||Substituted with a comma-separated list of found viruses.|
|%SPAMSCORE%||Spam score for the message as returned by a scanner.|
|%SPAMLEVEL%||Spam level for the message. This could be "low", "medium", "high", or empty string.|
Reject Templates – the MPP API
Reject templates can also be thought of as a type of API into MPP. When used in this fashion all actions of MPP are set to reject and a program will parse MPP rejection notices to take appropriate actions. In this way MPP can be a scanning brick that accepts messages on SMTP and hands back response codes.
In order to use custom reject templates fill in the text boxes with the text that you want to use in the rejection notices. DO NOT put file paths in the text boxes as file paths are not supported.
Since MPP will use its entire arsenal of plug-ins, embedded functions and features MPP any script or entity that can pass a message to MPP via SMTP can benefit from our functionality.
In order to use MPP in this function set <on_clean> to reject, this can be done in the GUI in Advanced->Policy Engine -> Advanced->Action for Messages. Also set every other action to reject – all spam thresholds, on virus, acl violation, rbl, acl, etc.
By setting all actions to reject your application will get a custom message back from MPP that uses your template, telling the application the result of MPP processing. So the general flow is: Your app SMTP to MPP MPP Scans MPP sends reject report Your app reacts For more details on using MPP in this fashion please send an email to Support@Mailspect.com
The Policy Engine is central to MPP and provides the capability to create distinct configuration sets that can be applied to groups of users. Groups can be identified by email address, domain, CIDR IP address and direction. Membership information can be stored in the MPP configuration file, text files or can be added as an attribute to any standard LDAP directory. Please see the appendix for details of using LDAP for policy storage.
How can I use the policy engine?
- In most cases the uses of the policy engine are simple – create different rules for inbound and outbound processing, archive email for a few people, monitor the emall of a few, etc. However, since MPP can support many thousands of policies and policy membership can be stored in a subscriber directory the policy engine can lead to elaborate configurations.
- It is very common for service providers to use MPP to query their subscriber directory to see if a user has subscribed for a service such as an archival. It is also common to classify processing options by IP address of trusted SMTP servers.
- The options of the MPP policy engine are limitless and it is really one of the most interesting abilities of MPP.
- MPP takes great care to preserve the integrity of multi-recipient email. If an email goes to two users or domain in different policies then both policies will have their preferences respected.
- As each email message gets analyzed during processing pipeline, MPP must determine the appropriate policy for the message.
Policy Match Logic
- Data sources are searched in the order described below. If the configuration parameters have multiple values, they are searched in order.
- Search through merged list of IP address entries from static LDAP, global text files and local text files.
- Search through dynamic LDAP
- Sender Domain
- Receiver Domain.
- Search through merged list of entries from static LDAP, global text files and local text files for
- Sender domain
- Receiver Domain
Customizing Policy Search Order
- Search order may be customized if you prefer to give weight to policy matching methods. Select the Policy->membership tab to configure the order by which policy selection is made.The following options are available:
|ip_list||Selects group for all recipients depending on IP address of sending client. The list consists from all IP's that that may appear in members_global_textfile, members_local_textfile, members_addresslist options.|
|ldap_recipient||Selects group for each recipient depending on dynamic LDAP query for recipient group. Ignored if LDAP is not enabled or LDAP is not dynamic.|
|address_list_recipient||Selects group for each recipient depending on searching in address list for recipient group. The list consists of email addresses that may appear in members_global_textfile, members_local_textfile, members_addresslist.|
|ldap_sender||Selects group for all recipients depending on dynamic LDAP query for sender group. Ignored if LDAP is not enabled or LDAP is not dynamic.|
|address_list_sender||Selects group for all recipients depending on searching in address list for sender group. The list consists of email addresses that may appear in members_global_textfile, members_local_textfile, members_addresslist.|
|header_filter||Selects group for all recipients depending on searching in message headers for regular expressions. Searching is done with boorex engines listed with members_global_header_filter option (see further). If match expression is found its id is directly mapped to group name.|
|content_filter||Selects group for all recipients depending on searching in message content and (optionally) attachments content for regular expressions. Searching is done with boorex engines listed with members_global_content_filter option (see further). If match expression is found its id is directly mapped to group name.|
|recipient_filter||Selects group for each recipient depending on searching in SMTP envelope recipient address for regular expressions. Searching is done with boorex engines listed with members_global_recipient_filter option (see further). If match expression is found its id is directly mapped to group name.|
|sender_filter||Selects group for all recipients depending on searching in SMTP envelope sender address for regular expressions. Searching is done with boorex engines listed with members_global_sender_filter option (see further). If match expression is found its id is directly mapped to group name.|
Default Policy and Inherited Options
- Email that does not have a specific policy match will be processed with the default policy group settings.
- When creating new policies most settings are inherited from the default policy group s such as choice of scanners, spam actions, quarantine location and all others except those identified in the next section.
Non-inherited Configuration Options
- The MPP GUI generates a list of all available policies and presents them as a drop down menu on the upper right of pages in the Services and Advanced sections. To configure options for a policy select the policy from the drop down on the upper right hand corner of the interface.
Creating New Policies
- Name the policy by typing the name in the policy drop down, next to the ADD button. This is a special drop down that allows you to type new entries
- Create the matching criteria by typing in the match criteria for sender, recipient or both setting.
- Entries are separated by commas. mpp@Mailspect.com,mpp2@Mailspect.com, raeinternet.com.
- Valid entries are email address, domains or CIDR IP Address – mpp@Mailspect.com, Mailspect.com,18.104.22.168/24
- Save and restart to have the policy take effect.
- If you require dynamic provisioning of policies use LDAP for policy storage.
- The default action for an MPP policy is to pass clean messages. However, there are interesting possibilities with other actions of MPP policies. If you expand the advanced section of this page you can change the policy page you can set the default action for a group.
|Scan||Default action, messages are scanned|
|Pass||Messages are passed and not scanned, analogous to Client Host White List|
|Reject||Messages that match this group are rejected|
|Discard||Messages that match this group are silently discarded|
|Forward||Messages that match this group are forwarded to the address specified|
- If message forwarding is used it is possible to deliver the message after forward or discard.
- The ‘action for forwarded email’ is valid for all email with action of ‘forward’. Even if the action for a group is scan, this option is valid for forward action for content filtering, ACL’s.
- MPP can consult and LDAP directory to find group membership. An entry has an object class (raeMPP) that has attributes raeMPPGroupNameRecipient and raeMPPGroupSender. MPP looks for the Email address LDAP attribute and then for the group name.
- Please see the appendix for a complete explanation of LDAP attributes.
The system section of the MPP GUI controls system wide parameters and configuration parameters of the GUI itself.
MPP is very commonly used on SMTP relays and this section allows you to configure how MPP relays mail. This section is only available when MPP is installed with the Postfix MTA. If enabled, by selecting ‘Allow MPP to control relay domains and transports’. By selecting this option MPP will modify the Postfix configuration files, saving originals, to use relay and transport files that the MPP GUI can control.
Adding a Relay Domain
- Type the name of the domain, such as messagepartners.com, the IP address of the real email server, 192.168.100.3 for example and decide if the mx record for the domain should be checked. For private networks, i.e. your own email server behind your firewall, don’t check the mx record. If you are forwarding for a domain that may move their email server and mx then check this, but generally this should be left unchecked.
Outbound Relay – IP’s to relay
- Use this option to define specific IP’s or networks that MPP should relay. This is useful for outbound relay if your real email server is on a different subnet than the MPP gateway. If you are attempting to filter or archive outbound email then add the IP address of the real email server here.
Verify recipient addresses using SMTP
- This option will verify recipient addresses using SMTP verify messages. SMTP verify are sent according to domain transport maps defined above.
- Use this option to clear out the queue of the relay. This is useful if your real email server was down for some period or if mppd was down and mail has backed up in the SMTP queue.
This section displays red, green or yellow to show if a database is online, non-functional or has in an unknown state, such as extra tables (orange).
From this page you can show, drop, empty or dump (back up) tables.
This section is to configure, monitor and create MySQL databases for use with MPP. There is a lot of functionality with this section and a few key concepts that must be understood.
- The default database is stored in the MPP GUI configuration files, it is not extracted from MPP configurations. It is a good idea to define a default database that MPP monitors.
How Many Databases?
- We recommend that spam quarantines and email archives use separate databases. While it is possible to share the database connections between all purposes it is best to keep the functions separate.
Database Functions for Spam Quarantine
- The following MPP features should use the same database if spam quarantine is in use.
- Primary Quarantine
- White and Black Lists
- Per-User Spam Actions
- Recipient ACL
- Client-Host WBL
- Magic Location
Database Functions for Archive
- To use MySQL for email archive only the archival feature needs to be set to use MySQL, however, it is strongly recommended to use the MPP virtual appliance for this purpose. If you opt to set up MPP email archival on your own it is mandatory to also install the Sphinx indexing engine, as per the appendix, to enable full-text searching.
Creating a Database
|NOTE: Read this section carefully. If you have multiple policies be mindful of the policy selection. To create a database for MPP to use follow these steps.|
- Create a blank database with your favorite MySQL tool and a user that can access the database. The database can be remote or local and multiple instances of MPP can share the same database. Here are the commands to create an empty database and a user with access rights from the localhost:
- mysql> create database spam_quarantine;
- Query OK, 1 row affected (0.00 sec)
- mysql> grant all on spam_quarantine.* to 'mpp@localhost' identified by 'mpp_password';
- Query OK, 0 rows affected (0.02 sec)
- Define the settings of database in Global MySQL Database Connection.
- Decide if this database should be the ‘Global’ database. The Global Database is an analog of MPP GUI that it uses as the default to monitor. After entering the settings select Save and Create Tables.
Applying a Database
- To apply the database settings to places in the MPP configuration check the applicable boxes in the right column of the page. For spam quarantine you should apply the same database to quarantine, recipient ACL, White/Black List, Per-User Spam Settings, Magic Location and Client-Host WBL.
- If you have configured multiple policy groups it is possible to copy the database permissions to other policy groups by selecting the functions of database (i.e. wbl, spam settings, etc.) and the name of the policy that you want to COPY TO:.
This section controls how MPP interoperates with a Mail Transfer Agent, or MTA. In most cases it is not necessary to ever change the parameters in this section as the default settings will work in most cases.
Most parameters are self-explanatory, however, the options in the Postfix section require some explanation because this represents the primary SMTP interface for MPP.
Generic SMTP Interface
- The Postfix interface is a generic SMTP interface that any application can use to submit messages to MPP. Any SMTP application can submit messages to MPP via SMTP and the socket address may be configured.
- LMTP is the best protocol choice to use with Postfix because it provides per-user status codes. So if you are using Postfix use LMTP.
ESMTP Extensions for Quarantine Transport
- MPP supports a hierarchical storage model that transports quarantine or archive email via ESMTP. To configure a listener for this model enable the mailstore selection from the List of extensions to use for SMTP/LMTP protocol option.
- In order to create a ‘listener’ for this traffic configure a remote instance of MPP to recognize the ESMTP extensions in System -> MTA Integration set MTA to Postfix and List of extensions to use for SMTP/LMTP protocol to mailstore.
SMTP Listening Sockets
- By default MPP listens on 10025, however this can be changed in the Socket type and address of the for incoming connections field. This is useful if you have multiple Postfix content filters or need MPP to listen on some other socket for an external application.
The options in MPP GUI section are largely self-explanatory and need not be changed in virtually all cases. The commonly changed fields are outlined here:
Log Files Directory
- Change this if necessary.
MPP parser time period
- By default the parser script is run every 20 minutes, change this value if you have very large log files.
MPP GUI Certificate Files
- This section allows you to upload you own certificates if SSL was enabled during the installation of MPP Server. It is not necessary to install your own cert files to use SSL with MPP Manager.
Postfix Policy Server
- The Policy Server is a very important piece of the MPP filtering architecture for its pre-queue filtering possibilities. Since MPP has both a Postfix Content Filter and a Policy Server, and they share information, MPP is the only integrated pre and post queue-filtering tool for Postfix that we are aware of.
- MPP processes spam white and black lists, spam traps, access control lists, real-time black hole lists and.
Policy Server Architectures
- The policy server can be used in 3 different architectures:
Policy Server and Content Filter Coexist
- In this model, the default, it is assumed that the Content Filter and Policy Server co-exist on the same server. This is fine for the majority of sites.
Policy Server Only
- This model is suitable for sites where there are high volumes of SMTP traffic that needs to be processed by connection services such as spam traps, black lists, client-host black lists, access-control lists and real time black hole lists. In this model MPP is only a policy server and there is no content filter configured. In this model MPP cannot support per-recipient actions.
Remote Policy Server
- In this model the policy server is on a dedicated server that is only a policy server. Postfix is not operational on this server, only MPP is accepting policy connections and responding to queries.
Policy Server Configuration
- In any usage case configuration is basically the same. The MPP configure.pl script will setup the Policy Server within MPP and the Postfix configuration files. If you decide after installation to run the policy server enabling it in the GUI will configure all components necessary.
- When MPP policy server is NOT co-existent with the content filter it is important to leave the filter string as blank. By default the filter string is mppscan, this should be set to blank when the policy server and content filter do not co-exist.
- Greylisting is only active if the Postfix Policy Server is enabled. If the Policy Server is enabled configure Greylisting in Services->Connections.
Pre and Post Queue Processing with Policy Server
- MPP has a unique capability to intelligently process pre or post-queue with Postfix using the following logic:
- At first RCPT command from client the Policy Server receives a request from Postfix with IP, sender and recipient info. MPP then checks whether IP is in Client Host Whitelist. If yes then the message is NOT routed to post-queue MPP scanning at all and is only processed by Postfix. If the message is not on the CHWL then MPP checks whether IP is in Client Host Blacklist. If yes then recipient is rejected with a 421 code, which causes postfix to disconnect from client immediately. No further processing is performed.
- If the message is not on any CHBL then the MPP policy-engine processes the message according to sender and recipient information. If the group matched has an action of "pass" then the message is NOT routed to post-queue MPP scanning. If it is "reject" recipient is rejected without disconnecting. If it is "discard" message is discarded.
- This scheme is the same for ACL’s and WBL’s except that if quarantine is required or there is a spam whitelist hit, the message is routed to post-queue processing. If blacklist violation action is to "mark header" then it is marked and is passed and NOT routed to post-queue scanning.
- If the message is not black- or whitelisted per-group then RBL’s are queried. Here message can be discarded, rejected or left for further processing depending on corresponding RBL options.
- A special case exists for messages with multiple recipients. Some actions cannot be applied on per-user basis at pre-queue stage due to postfix limitations. Those actions include pass, discard and markheader. In this case, if any of these actions appear for one recipient, and there is other non-rejected recipient, then decision is postponed until post-queue processing. For example if for one recipient group action is “pass" and for other it is "scan" then message is passed to post-queue processing for BOTH recipients because "pass" cannot be applied on per-user basis at pre-queue.