Cloudmark Home Security System 3048 User's Manual
Contents
1. 26 Envelope whitelisting Chapter 5 Whitelisting Sample whitelist configuration file Whitelist Configuration File This configuration file defines whitelisted domains IPs and headers When an item is matched the message is guaranteed to be let through unmodified Empty lines or lines where the first non whitespace character is a are ignored Type Host This type of whitelist entry applies to any kind of ip or domain name If a DNS name is provided then whitelist effectiveness is contingent on DNS being properly enabled and set up on the system this product is installed on IP subclasses and DNS subdomains are supported type host address 1 2 3 4 type host address 192 168 type host address mxl somecompany com type host address gov Type Header This type of whitelist entry effectively matches any substring or regular expression against the specified header field type header header From value gov gt type header header From value cloudmark com a Type Envelope This type of whitelist entry applies to commands on the SMTP envelope The value will match against any substring or regular expression in the specified command field of the envelope type envelope command helo value ite type envelope command mail from value ite type envelope command rcpt2. At least one micro update must be downloaded in order for the Cartridge to operate offline If the Cartridge is unable to download the next micro update the previous micro update file set is used 20 Data files Chapter 4 Micro Updates When a valid set of data files is downloaded from the micro updates service they are saved to the lt micro update cache path gt micro_updates directory with version numbers in the filenames New files are loaded from disk at startup l Delta micro updates files are read into memory and are not kept on disk Data file integrity and security The micro update files containing data are compressed and encrypted Data that is not encrypted with the correct key will be ignored As a result DNS or IP spoofing is not a concern The contents of the data files are read into memory at startup as well as each time a new file is downloaded Advanced micro update configurations Cloudmark provides flexible options for complex customer deployments If your organization has constraints that restrict you from enabling automatic micro updates downloads from the Cloudmark micro updates service the following advanced configurations may allow you to use micro updates within the constraints of your organization Using an HTTP proxy If your organization has constraints that prohibit the Cloudmark software from accessing the Internet directly you may enable an HTTP proxy to direct all traffic thro
3. will not send report This message is logged when there is a problem connecting to the configured micro updates host potentially via a proxy if a proxy is configured If this message occurs the statistics report will not be sent to Cloudmark Could not send HTTP request lt http_request_str gt lt net_error gt will not send report This message is logged when there is a problem sending a HTTP POST containing a statistics report to the configured micro updates host or proxy If this message occurs the statistics report will not be sent to Cloudmark Could not close socket for statistics report lt net_error gt report may not have been sent successfully This message is logged when there is a problem closing a network socket after sending a statistics report If this message occurs the report may not have been successfully sent to Cloudmark Could not receive HTTP response for statistics report lt net_error report may not have been sent successfully This message is logged when there is a problem reading the HTTP response after submitting a statistics report If this message occurs the report may not have been successfully sent to Cloudmark Could not determine hostname lt net_error gt using a host identified of unknown This message is logged when the cartridge could not determine the local machine s hostname The string unknown will be used in statistics reports Unexpected HTTP response for HTTP request lt htt
4. these messages are exposed This appendix explains the logging variables and the log messages in which they are found e Common variables below e INFO log messages on page 38 e WARN log messages on page 41 e ERROR log messages on page 44 e CRITICAL log messages on page 48 Common variables The following common variables are used in the INFO log messages Table 4 Log variables Variable Sample Values lt cachedir gt N A lt count gt N A lt customer id gt N A lt decode_error gt Could not decrypt data Could not decompress data Truncated content Description The directory specified by micro update cache dir possibly prepended with lt etcdir gt A message entry statistic count The customer id specified in cartridge cfg An error string related to decoding content 37 Cloudmark Cartridge Installation and Administration Guide Appendix A Table 4 Log variables Variable Sample Values lt disk_error gt Permission Denied Not a Directory No Such File or Directory lt encrypt_errnum gt N A lt etcdir gt N A lt host id gt N A lt http_request_str gt N A lt http_response_str N A gt lt mu_file gt N A lt mu_source gt N A lt net_error gt Connection Refused Timed Out Any of the standard error strings returned on Unix or Windows systems related to TCP IP networking lt seri
5. and administer the cartridge You can find out what s new in this version of the cartridge in What s new in Cartridge 3048 below The rest of this chapter introduces the technology behind the Cloudmark Cartridge e Cloudmark fingerprinting algorithms on page 2 e Cloudmark Global Threat Network on page 2 e Micro updates on page 3 e Message scoring on page 3 e Cartridge statistics on page 4 e Whitelisting on page 5 What s new in Cartridge 3048 Cartridge 3048 includes these changes e The Cartridge now keeps track of its last known viable state and returns to it upon restarting after a crash e Anew configuration key specifies an alternate port on which to download micro updates See micro update port on page 16 e Anew configuration key controls how the sending IP address is determined See use envelope for ip information on page 16 e The default value for the use ip information configuration key is now yes See use ip information on page 16 e The micro update file set now includes fsl and xrl metadata files See Data files on page 20 Cloudmark Cartridge Installation and Administration Guide Chapter 1 e A new fingerprinting scheme provides faster processing e Anew statistics field reports your unique installation ID See What statistics are collected on page 31 Cloudmark fingerprinting algorithms The Cloudmark Cartridge
6. By default statistics are reported to Cloudmark at the following URL http 1lvc cloudmark com cmstats The default hostname is microupdates cloudmark com Ifthe Cartridge is configured to use an HTTP proxy for micro update downloads statistics will be sent using the same HTTP proxy See Using HTTP proxies on page 19 29 Cloudmark Cartridge Installation and Administration Guide Chapter 6 The POST body will be of content type text plain and contain a collection of key value pairs Below is a list of the key value pairs in the POST body Table 2 POST body key value pairs Key value pair report spamdna stats version lt val gt encoding e c e stats lt compressed encrypted encoded blob gt Description This identifies that this report consists of Cartridge statistics It may be different for future possible communications from the Cartridge to Cloudmark This identifies the version of the reported statistics When Cloudmark adds removes or changes the statistics reported by the Cartridge Cloudmark will increment this version number This identifies the encoding of the statistics block e e implies that the block has been encrypted and then base64 encoded This was the default for Cartridge 3044 and 3045 e e c implies that the block has first been compressed using the compress function before being encrypted This was the default as of Cartridge 3046 All remainin
7. Cartridge with the following command 8 Installing or updating the Cartridge Chapter 2 Cloudmark Cartridge Installation gzip d c lt SpamDNA 3048 x x x lt platform gt tar gz tar xvf e For Windows installation double click the zip file then click Extract Create the etc license cfg file This file must contain the two line license text that you received from Cloudmark If you are updating an existing Cartridge installation update your cartridge cfg to the latest defaults listed in the file etc cartridge cfg sample 7 Restart the server service using the Cloudmark Authority Engine 8 Check for the following log message INFO MICROUPDATE Successfully updated lt file gt from network new serial lt serial gt There should be one such message for every micro updates file listed in The Cartridge installation package on page 7 See also the Cloudmark Authority Engine SDK Guide Installation for the Cloudmark Authority Plug In for SpamAssassin During a new installation of the Cloudmark Authority Plug In for SpamAssassin the Cartridge is installed automatically To upgrade the Cartridge for an existing installation of the plug in use the instructions below TO UPGRADE THE CARTRIDGE FOR THE SPAMASSASSIN PLUG IN 1 2 Become the superuser Switch to the SpamAssassin CMAE subdirectory cd etc mail spamassassin cmae cloudmark This path may vary in your installation Make sure you are in the
8. The disk may be full or there may be a permission error Could not read from s The disk may be full or there may be a permission error Malformed line in s The state file is malformed Could not update signatures from network malformed license id in etc license cfg The license ID included non numeric characters Could not update signatures from network missing license file etc license cfg The license file is missing Could not update signatures from network invalid license HTTP error 401 The license credentials are invalid the ID and password do not match or the ID does not exist Could not update signatures from network unexpected download failure http_code 500 A micro update file is missing Could not update signatures from network license verification failed tid 38695 License not enabled or expired The license is disabled in the LDAC Could not update signatures from network license verification failed tid 38697 License not enabled or expired The license has expired 44 ERROR log messages Appendix A Logging Could not update signatures from network authorization denied The license is not authorized to download micro updates Could not update signatures from network license verification failed tid 38977 No sessions available The number of sessions authorized for this license has been reached Could not update signatures from network microupdate file XXXX failed the int
9. includes Cloudmark s fingerprinting algorithms designed to target the most current spamming techniques Using these algorithms the Cloudmark Cartridge generates a set of fingerprints for each incoming message The Cloudmark Cartridge maintains a cache of all fingerprints that have a known classification such as spam phishing or virus fingerprints The fingerprints of an incoming message are compared to these known fingerprints and a message score is generated This list of known fingerprints is regularly updated with the latest data from the Cloudmark Global Threat Network using the micro updates mechanism See Cloudmark Global Threat Network below and Micro updates on page 3 Cloudmark Global Threat Network Cloudmark s community of millions of end users provides constant real time feedback about which messages are considered spam phishing or email borne viruses and which ones are considered legitimate The Trust Evaluation System TES assigns each user a trust level based on how well the user s feedback concurs with that of other trusted users Less trusted users have less influence over network wide message classification while the most trusted users have more influence When a sufficient number of trusted users block a certain message as junk this message s fingerprint is flagged Information about the fingerprint is distributed throughout the network to automatically block that message and all its permutat
10. latest fingerprint data used to identify abusive messages The fingerprint data is provided by highly trusted reporters and analysis by the Trust Evaluation System in the Cloudmark Global Threat Network in near real time Therefore proper use of the micro update mechanism is critical in maintaining Cartridge accuracy as new types of spam phishing and virus messages are reported l Your installation must have a valid license file in order to download and process micro updates If you do not have a license file please request one from Cloudmark This chapter includes the following topics e Micro update frequency below e Network interaction on page 19 e Data files on page 20 e Advanced micro update configurations on page 21 Micro update frequency The administrator can set the frequency at which the software downloads the micro updates in one of two ways using automatic updates This is the default setting See Automatic micro updates on page 18 e setting a specific update interval This setting is not recommended as it may compromise the accuracy of filtering See Updates at user specified intervals on page 18 17 Cloudmark Cartridge Installation and Administration Guide Chapter 4 These options are configured using the micro update interval configuration setting in the cartridge cfg file See also Chapter 3 Cloudmark Cartridge Configuration Automatic micro up
11. or when it finds a file in the micro_updates directory whose serial number is greater than the serial number of the in memory version of the source Removed partial lt mu_source gt file lt file gt This message is logged when the micro update module finds and removes partial micro update files with the INFO log messages 39 Cloudmark Cartridge Installation and Administration Guide Appendix A extension part in the micro updates directory These files may exist if the cartridge is shut down while a micro update download is in progress Successful lt mu_source gt download from network new serial lt serial gt This message is logged when the cartridge successfully downloads and decodes a micro update file from the network Removed old lt mu_source gt file lt filename gt This message is logged when the cartridge successfully removes an old file i e the one with the lowest serial number below the micro_updates directory The cartridge will keep at most 2 micro update files for a source at any one time The option micro update path has been deprecated ignoring This message occurs when a deprecated configuration option is encountered Successful lt mu_source gt incremental download from network new serial lt serial gt This message is logged when the cartridge successfully downloads and decodes a micro update file from the network Shutting down stopping micro update downloads This message is logged at shutdown
12. type is non zero all sizes will be included Count of messages whitelisted by local whitelist cfg and awl whitelist files These count the sizes of messages whitelisted by various methods in the last data period If all counts are zero the entry will not be included in a statistics report However if any size count for a given whitelist type is non zero all sizes will be included These count how many times each whitelist cfg awl entry whitelisted a message Only entries with counts gt 0 will be included What statistics are collected 33 Cloudmark Cartridge Installation and Administration Guide Chapter 6 Table 3 Statistics key value pairs Key value pair category lt cat_number gt lt count gt subcategory lt cat_number gt lt subcat_number gt lt count gt whitelist lt base64 encoded blob gt successful complete micro update downloads lt count gt successful incremental micro update downloads lt count gt failed complete micro update downloads lt count gt failed incremental micro update downloads lt count gt successful complete micro update downloads from disk lt count gt failed complete micro update downloads from disk lt count gt failed micro update serial number downloads from network lt count gt mu lt source gt serial lt complete serial gt lt incremental serial gt sigs seen lt sig gt lt engine gt lt count gt lt sig gt lt engine gt
13. 19 proxy 14 19 21 46 authentication 14 19 35 S security 21 49 Index signature files 20 SpamAssassin 9 spoofing 21 state files 20 statistics 4 14 16 29 43 47 configuration 35 T timeout 19 Trust Evaluation System 2 17 U updating 8 Ww whitelist cfg 34 whitelist cfg file 11 12 23 33 whitelisting 14 23 body 25 envelope 25 header 24 host 23 sample configuration 27 50
14. Cloudmark Cartridge Installation and Administration Guide cLoupmark 2001 2007 Cloudmark Inc All rights reserved Cloudmark the Cloudmark logo and other trademarks service marks and designs are registered or unregistered trademarks of Cloudmark Inc and its subsidiaries in the United States and in foreign countries Other brands and products are trademarks of their respective holders All product information is subject to change without notice This document may not in whole or in part be copied photocopied reproduced translated or reduced to any electronic medium or machine readable form without prior consent in writing from All examples with names company names or companies that appear in this guide are fictitious and do not refer to or portray in name or substance any actual names organizations entities or institutions Any resemblance to any real person organization entity or institution is purely coincidental While every effort has been made to ensure technical accuracy information in this document is subject to change without notice and does not represent a commitment on the part of Cloudmark Inc Cloudmark makes no warranties with respect to this documentation and disclaims any implied warranties of merchantability and fitness for a particular purpose Cloudmark shall not be liable for any errors or for incidental or consequential damages in connection with the furnishing performance or use of this manual o
15. Global Threat Network 2 2 2 ee ee 22 Micro updates ee ee ed Message scoring ee ee ee ee eee eee ee 9 Message categorization 1 6 ee 4 Cartridge statistics ee ee e 4 Whitelisting e 6 6 ee ee Cloudmark Cartridge Installation 7 The Cartridge installation package 22 Installing or updating the Cartridge a O ae eee Installation for Cloudmark Authority Engine based os 8 Installation for the Cloudmark Authority Plug In for SpamAssassin 9 Installation for Cloudmark Immunity 2 2 ee e 10 Installation for Openwave Email Mx 1 1 6 6 ee ee ee 11 Installation for Openwave Edge GX a a 1 ee ee ee ee ee 12 Cloudmark Cartridge Configuration 13 Micro Updates acu a a wok as aa ae TZ Micro update frequency ee ee ee ee ee 17 Automatic micro updates a 6 6 ee 18 Updates at user specified intervals 18 iii Contents CHAPTER 5 CHAPTER 6 APPENDIX A Network interaction 2 a ee ee ee ee ee ee 19 Using HTTP proxies ee 19 Connection timeout logic a ee 19 Datatiles s moe oe a e ee a e ZO Offline files s ew mos m ee e 20 Data file integrity and securi
16. TO INSTALL THE CARTRIDGE FOR OPENWAVE EMAIL Mx 1 Follow these steps cd AUTH_ HOME AUTH_HOME value from configuration key lt host gt authority homeDir You should see an etc and a lib directory here cp lib cartridge so lib cartridge so lt datestamp gt SINTERMAIL 1ib imservctrl stop imextserv gzip d c lt SpamDNA 3048 x x x lt platform gt tar gz tar xvf 2 Create the etc license cfg file This file must contain the two line license text that you received from Cloudmark 3 Ifyou are updating an existing Cartridge installation update cartridge cfg to the latest defaults listed in the file etc cartridge cfg sample 4 Start the extensions service SINTERMAIL lib imservctrl start imextserv 5 Check for the following log message Aug 21 12 28 06 INFO MICROUPDATE Successfully updated lt file gt from network new serial lt serial gt There should be one such message for every micro updates file listed in The Cartridge installation package on page 7 Installing or updating the Cartridge 11 Cloudmark Cartridge Installation and Administration Guide Chapter 2 See also the Cloudmark Authority Engine Extensions Service for Openwave Email Mx Administration Guide Installation for Openwave Edge Gx Before starting this procedure Cloudmark recommends backing up entire AUTH_HOME directory TO INSTALL THE CARTRIDGE FOR OPENWAVE EDGE GX 1 Stop the server SINTERMAIL 1ib ims
17. al gt N A lt sig gt N A lt whitelist_module gt N A INFO log messages Description Any of the standard error strings returned on Unix or Windows systems related to disk I O An error code related to encryption or decryption Directory containing authority cfg immunity cfg or the ConfigDirectory passed to CMAE_Init The local machine s hostname as returned by gethostbyname The contents of a HTTP request to the configured micro updates host or proxy with unprintable characters converted to The contents of a HTTP response from the configured micro updates A file in the micro_updates directory Contains one of the following values signatures signature meta data rpl signature meta data dpl or awl The serial number of a micro update file in the form of lt complete gt lt incremental gt An individual message fingerprint Indicating the specific whitelisting module Will load source s from safe set s is not safe A suspect file was found an older file will be loaded instead 38 INFO log messages Appendix A Logging Copied package microupdate files to safe set Displayed during first cartridge initialization after installation when the package files are considered safe Committed key report statistics from License Cfg value yes A configuration parameter was set from the LDAC Licensing OK session lt session number gt A new session was
18. as found in license cfg If the license is not found the value of this key is zero This key indicates whether the cartridge shut down properly during its previous run 1 if it did O if it did not This is a base64 encoded string that uniquely identifies a cartridge installation Cartridge reporting configuration The following configuration items configure the Cartridge s reporting mechanism report statistics By default this is set to yes to enable reporting If your organization has special policies that prohibit the type of reporting described in this chapter set this parameter to no However this is not recommended as it may complicate Cloudmark s technical support process in the event of a problem exclude from stats reports Use this configuration key to exclude whitelist and proxy authentication statistics from the report customer id This sets the name of your organization or your organization s CNFS login if any See also Chapter 3 Cloudmark Cartridge Configuration Cartridge reporting configuration 35 Cloudmark Cartridge Installation and Administration Guide Chapter 6 36 Cartridge reporting configuration APPENDIX A Logging Log messages are passed programmatically from the cartridge to your application See your application s documentation for information about how
19. as it can fall back to network based downloads If the cartridge is not configured to perform micro update downloads it will log an error message instead of a warning message Could not load lt mu_source gt files from disk lt disk_error decode_error gt This message is logged when the cartridge attempts to read cached micro update files from disk but fails This message will be logged only if the cartridge is configured to perform micro update downloads as it can fall back to network based downloads If the cartridge is not configured to perform micro update downloads it will log one of the below errors instead Can not remove old lt mu_source gt files lt disk_error gt This message is logged when the cartridge attempts to remove an old file i e the one with the lowest serial number below the micro_updates directory but fails The cartridge will keep at most 2 micro updates files for a source at any one time Can not resolve micro update http proxy hostname lt host gt lt net_error gt expect micro update downloads to fail This message is logged at startup when the cartridge cannot resolve the configured micro update host to an IP address This typically indicates a problem with DNS resolution at the installation Cartridge accuracy will be significantly reduced without current micro update files so the problem should be identified and solved as quickly as possible Could not remove signature lt sig gt from local set T
20. ation see Chapter 5 Whitelisting Whitelisting 5 Cloudmark Cartridge Installation and Administration Guide Chapter 1 6 Whitelisting CHAPTER 2 Cloudmark Cartridge Installation This chapter provides the Cartridge installation instructions e The Cartridge installation package below e Installing or updating the Cartridge on page 8 e Be sure to refer to the release notes of each Cartridge version for special installation instructions The Cartridge installation package The Cartridge installation package is provided in either a TAR or a ZIP file depending on your platform Before installation verify that the installation package contains all the required installation files Below is a list of the components in a standard Cartridge installation package e etc micro_updates lt dpl_version_number gt dpl e etc micro_updates lt rpl_version_number gt rplv1 e etc micro_updates lt awl_version_number gt awl e etc micro_updates lt acf_version_number gt acf e etc micro_updates lt csl_version_number gt csl e etc micro_updates lt fsl_version_number gt fsl e etc micro_updates lt xrl_version_number gt xrl e etc micro_updates lt mpl version_number gt mpl e etc micro_updates lt cats_version_number gt cats e etc micro_updates lt mfl_version_number gt mfl Cloudmark Cartridge Installation an
21. cloudmark subdirectory Remove all of the files in the etc micro updates directory The new Cartridge will download the correct files with which to re populate this directory Extract the Cartridge gzip d c lt SpamDNA 3048 x x x lt platform gt tar gz tar xvf Installing or updating the Cartridge Cloudmark Cartridge Installation and Administration Guide Chapter 2 5 If you are upgrading from Cartridge 3046 or earlier create the etc license cfg file This file must contain the two line license text that you received from Cloudmark Update your cartridge cfg with the latest defaults listed in the file etc cartridge cfg sample Restart the CMAE server bin cmaed restart Installation for Cloudmark Immunity To install the Cartridge when using Cloudmark Immunity follow the steps below TO INSTALL THE CARTRIDGE FOR IMMUNITY 1 2 Stop Immunity If you are updating an existing Cartridge installation remove all of the files in the etc micro updates directory The new Cartridge will download the correct files with which to re populate this directory Place the compressed Cartridge update file in the product home directory Decompress it e For Linux Solaris extract the Cartridge with the following command gzip d c lt SpamDNA 3048 x x x lt platform gt tar gz tar xvf e For Windows double click the zip file Create the etc license cfg file This file must contain the two line licens
22. d Administration Guide Chapter 2 e etc micro_updates lt impl version_number gt implvl e etc micro_updates states srl_set package e etc whitelist cfg sample e etc cartridge cfg sample e lib cartridge so Additional files are downloaded as micro updates For more information about these files see Data files on page 20 Installing or updating the Cartridge Follow the installation instructions for the product with which you are using the Cartridge e Installation for Cloudmark Authority Engine based products below e Installation for the Cloudmark Authority Plug In for SpamAssassin on page 9 e Installation for Cloudmark Immunity on page 10 e Installation for Openwave Email Mx on page 11 e Installation for Openwave Edge Gx on page 12 These instructions apply to both new installation and updates to existing Cartridge installations Installation for Cloudmark Authority Engine based products TO INSTALL THE CARTRIDGE FOR AUTHORITY BASED PRODUCTS 1 Stop the server service using the Cloudmark Authority Engine 2 If you are updating an existing Cartridge installation remove all of the files in the etc micro updates directory as well as the etc micro updates states subdirectory The new Cartridge will download the correct files with which to re populate this directory 3 Place the compressed Cartridge file in the product home directory Decompress it e For Linux Solaris extract the
23. d not initialize that storage If this message is logged it means that the cartridge will return a score of zero for all subsequent messages This problem should be identified and fixed immediately to achieve high accuracy 48 CRITICAL log messages A authentication 14 19 35 NTLM 14 19 B body whitelisting 25 C cartridge cfg file 10 11 13 18 19 cartridge so file 8 12 categories 4 20 34 category file 4 20 checksum files 20 Cloudmark Authority Engine 8 Cloudmark Authority Engine SDK 4 Cloudmark Authority Engine SDK CMAE SDK 3 Cloudmark Global Threat Network 2 3 17 Cloudmark Immunity 10 Cloudmark Network Feedback System CNFS 13 29 CMAE _ Score function 4 configuration 13 connection timeout 19 crash recovery 1 empty messages 13 39 encryption 30 envelope whitelisting 25 F file integrity 21 fingerprinting algorithms 2 fingerprints 2 H header whitelisting 24 host whitelisting 23 HTTP proxy 14 19 21 46 image processing depth 15 image based spam 15 imconfedit 12 imservetrl 12 installation 7 8 L license cfg 9 10 11 12 44 licensing 17 19 35 errors 41 44 logging 37 M metadata files 20 micro updates 2 3 13 15 17 39 configuration 15 frequency 17 microupdates cloudmark com 29 O Openwave Edge Gx 12 Openwave Email Mx 11 P port configuration 16
24. dates By setting the micro update interval configuration setting to auto the Cloudmark Cartridge will automatically download the latest micro update information at the interval defined by Cloudmark in the acf file When the time intervals are updated the Cartridge will automatically download a new version of the acf file By default the Cartridge checks for the availability of changes to the micro update data every minute The entire micro update is downloaded once every three hours and saved to disk so that the data can be used offline if necessary e The default time intervals as stated above may change as Cloudmark releases new acf files Cloudmark strongly recommends that customers use the auto setting for micro updates Updates at user specified intervals You can specify an interval in hours for the micro update interval setting If an update is available at that interval the full micro update will be downloaded and saved to the etc micro_updates directory The two latest versions of the micro updates data will be kept on disk Older versions will be deleted automatically l This configuration is not recommended as it may compromise the accuracy of filtering 18 Micro update frequency Chapter 4 Micro Updates Network interaction Micro updates are downloaded using standard HTTP requests If an HTTP proxy is not enabled then at the specified interval a download will be attempt
25. e cannot open or read the configured whitelist cfg file In this case no local whitelist entries will be used Malformed line lt line gt lt parse_error gt skipping This message is logged when the whitelist module could not parse a line in whitelist cfg Such lines will be ignored Examples include unknown type missing command or D D regex malformed regex missing address etc Could not save engine information lt string gt gt lt string gt When used with CMAE the cartridge is initialized with engine information such as the CMAE version string application name and so on This information is saved within the cartridge and included in statistics reports This message is logged when the cartridge could not properly save that information WARN log messages 43 Cloudmark Cartridge Installation and Administration Guide Appendix A ERROR log messages Failed to read current serials from disk The etc micro_updates states srl_set current file is unexpectedly missing Could not find source s in current serial state file The srl_set current file is corrupted Failed to write the current state file s The disk may be full or there may be a permission error Failed to remove s There was a permission error in the etc micro_updates states directory Failed to rename s to s There was a permission error in the etc micro_updates states directory Failed to open s for writing
26. e text that you received from Cloudmark If you are updating an existing Cartridge installation update your cartridge cfg with the latest defaults listed in the file etc cartridge cfg sample Restart Immunity Ifyou are installing Cloudmark Immunity for the first time complete the following additional steps Manually copy the file new_cm_egm db temp externally provided to the following location before running dbsetup pl e Immunity 2 0 2 lt immunity root gt data new_cm_egm db temp 10 Installing or updating the Cartridge Chapter 2 Cloudmark Cartridge Installation e Immunity 2 0 1 lt immunity root gt setup sql sqlite cm_egm db lt immunity root gt is typically srv immunity 9 The default cartridge cfg and whitelist cfg files will not be installed in etc To create the default configuration files copy etc cartridge cfg sample to etc cartridge cfg and etc whitelist cfg sample to etc whitelist cfg 10 Check for the following log message Aug 21 12 28 06 INFO MICROUPDATE Successfully updated lt file gt from network new serial lt serial gt There should be one such message for every micro updates file listed in The Cartridge installation package on page 7 See also the Cloudmark Immunity Installation and Administration Guide Installation for Openwave Email Mx Before proceeding with the installation instructions below Cloudmark recommends backing up the entire AUTH_HOME directory
27. ed over port 80 by default or the port configured by micro update port on page 16 If the filtering server is unable to connect to the Cloudmark micro updates service the most recent offline version of the data file in the etc micro_updates directory will be used until the next micro updates interval before attempting to download again e Micro updates cannot be downloaded if a valid license file is not present Using HTTP proxies To use an HTTP proxy set the http proxy configuration parameter in cartridge cfg to your proxy server in one of the following forms lt host gt lt port gt lt address gt lt port gt If HTTP proxying is enabled then the port specified will be used If the proxy uses HTTP basic authentication then set the http proxy basic auth parameter in the following form lt user gt lt password gt If the proxy uses NTLM authentication then set the http proxy ntlm auth parameter in the following form lt domain gt lt username gt lt password gt See also Chapter 3 Cloudmark Cartridge Configuration Connection timeout logic The micro update timeout parameter sets the timeout that will be used for all network calls Therefore this setting is not related to the actual download time just the network call Network interaction 19 Cloudmark Cartridge Installation and Administration Guide Chapter 4 Data files Offline files The Cloudma
28. ed in conjunction with Cloudmark Authority Engine SDK CMAE SDK 2 0 or later the cartridge may also provide information about each message s classification spam phishing virus and so on Consult your vendor to find out whether your implementation of the CMAE SDK supports this feature You can establish your own policies for handling spam and configure your application to take action on a message based on its spam score Such actions typically include one or more of the following e storing spam in a designated folder e flagging spam messages in the Subject field e deleting spam e returning spam to its original sender Micro updates 3 Cloudmark Cartridge Installation and Administration Guide Chapter 1 Message categorization When scoring a message with the Cloudmark Authority Engine SDK s CMAE_Score function an application can request that the cartridge return a category and a subcategory for the message Categories and subcategories are expressed as integers which are mapped to categories in the cats file See the cats file for the list of categories For example using the Authority Engine SDK the following call produces a message score and category CMAE Score CMAE Envelope Envelope const char RFC822Content size t RFC822ContentLength unsigned int ScoreOut unsigned int CategoryOut unsigned int SubCategoryOut unsigned int RescanOut char AnalysisOut If CategoryOut is 7a
29. egrity check The computed checksum is incorrect Failed to load checksum file from cache lt filename gt The Cartridge failed to load a checksum file from disk Unable to load symbol lt sym gt This message is logged when there is a problem loading the cartridge adapter It is likely that there will be problems with subsequent message classification Could not create message score Expect errors related to reading message scores This message is logged when the cartridge cannot initialize a new message score If this message is logged the cartridge will return a score of zero for this message This problem should be identified and fixed immediately to achieve high accuracy Could not free per thread storage for message scores This message is logged at shutdown when the cartridge cannot free per thread storage used for message scores This will not affect accuracy but implies a resource leak Could not update lt mu_source gt from network or disk lt disk_error net_error decode_error gt This message is logged when the cartridge attempts to update a source from the network or disk but both methods fail Cartridge accuracy will be significantly reduced without current micro update files so the problem should be identified and solved as quickly as possible Invalid micro update interval lt interval gt using default interval of 3 hours This message is logged at startup when the value for micro update interval in cartridge cf
30. ensing host has served a checksum file unencrypted WARN log messages Can not create directory lt cachedir gt lt disk_error gt Expect errors saving micro update files This warning is logged when the pathname specified by micro update cache path does not exist the cartridge attempts to create the directory and fails Can not create partial micro update file lt mu_file gt lt disk_error gt This warning is logged when the micro update module cannot open a file to hold downloaded micro update content Micro update content will still be loaded into memory but will not be saved on disk Can not write to micro update file lt mu_file gt lt disk_error gt This warning is logged with the micro update module cannot write downloaded micro update content to disk Content will still be loaded into memory Can not rename partial micro update file lt mu_file gt part to lt mu_file gt lt disk_error gt The micro update module initially creates micro update files with the extension part and renames them once all content has been downloaded and validated This warning occurs if the module cannot rename a complete file with the extension part Downloaded content is still loaded into memory Can not remove partial micro update file lt mu_file gt part lt disk_error gt Ifa micro update download is terminated before it is complete or if there is a problem validating the downloaded content the micro update module attempts to
31. ervctrl stop imextserv Switch to the AUTH_HOME directory cd SAUTH HOME AUTH_HOME lt value of key lt host gt cloudmarkeas homeDir gt You should see an etc and a lib directory here Back up the old cartridge so file cp lib cartridge so lib cartridge so lt datestamp gt Decompress the Cartridge installation package gzip d c lt SpamDNA 3048 x x x lt platform gt tar gz tar xvf Create the etc license cfg file This file must contain the two line license text that you received from Cloudmark Install the whitelist configuration file if it does not already exist cp etc whitelist cfg sample etc whitelist cfg Use imconfedit to edit the config db and make all applicable changes Update the cartridgeParameters key with values as specified in the cartridge cfg sample file Start the server SINTERMAIL 1lib imservctrl start imextserv Check for the following log message Aug 21 12 28 06 INFO MICROUPDATE Successfully updated lt file gt from network new serial lt serial gt There should be one such message for every micro updates file listed in The Cartridge installation package on page 7 12 Installing or updating the Cartridge CHAPTER 3 Cloudmark Cartridge Configuration This chapter discusses the configuration settings in the cartridge cfg file Please also refer to the Car
32. form must reproduce the above copyright notice this list of conditions and the following disclaimer in the documentation and or other materials provided with the distribution Neither the name of the University of Cambridge nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS AS IS AND ANY EXPRESS OR IMPLIED WARRANTIES INCLUDING BUT NOT LIMITED TO THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT INDIRECT INCIDENTAL SPECIAL EXEMPLARY OR CONSEQUENTIAL DAMAGES INCLUDING BUT NOT LIMITED TO PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES LOSS OF USE DATA OR PROFITS OR BUSINESS INTERRUPTION HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY WHETHER IN CONTRACT STRICT LIABILITY OR TORT INCLUDING NEGLIGENCE OR OTHERWISE ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE Contents CHAPTER 1 CHAPTER 2 CHAPTER 3 CHAPTER 4 Introduction i 6 What s new in Cartridge 3048 2 ee ee 1 Cloudmark fingerprinting algorithms 2 a ee ee 2 Cloudmark
33. g is not auto or a positive integer Invalid cache path lt path gt using default value This message is logged at startup when the value for micro update cache path in cartridge cfg is empty If the path is specified but is not a directory or cannot be created a different message will be logged ERROR log messages 45 Cloudmark Cartridge Installation and Administration Guide Appendix A Value lt val gt for enable micro updates must be yes no 1 0 keeping downloads enabled This message is logged at startup when the value for enable micro updates in cartridge cfg is not a yes no style value Can not start networking subsystem lt net_error gt expect network operations to fail This error is logged on Windows platforms when the cartridge could not initialize networking via WSAStartup Invalid http proxy lt proxy gt lt error gt not using a http proxy Examples include must be of the form host port port must be in the range 1 65536 etc This message is logged at startup when the value for http proxy in cartridge cfg is invalid Invalid http basic auth lt auth gt lt error gt not using authentication for http proxy A string describing why the authentication specification is invalid Invalid micro update timeout lt timeout gt using default timeout of 60 seconds This message is logged at startup when the value for micro update timeout in cartridge cfg is no
34. g statistics are compressed and encrypted using a symmetric key embedded into the Cartridge The blob is compressed using the zlib compress function encrypted using AES and base64 encoded The raw uncompressed unencrypted data consists of key value pairs separated by newlines By default the Cartridge communicates statistics back to Cloudmark every hour The statistic reporting interval is managed by Cloudmark Cloudmark may adjust the frequency of statistic reporting interval when assisting customers with accuracy issues 30 How statistics are reported to Cloudmark Chapter 6 Cartridge Statistics Reporting What statistics are collected Information regarding the specific Cartridge instance is reported to Cloudmark upon each Cartridge installation The following is a list of statistics that are collected by the Cartridge if reporting is enabled Table 3 Statistics key value pairs Key value pair Description customer id lt string gt The exact value of customer id in cartridge cfg It is possible that this value may be blank for customers who enable statistics but do not specify a customer id host id lt string gt The fully qualified domain name of the local machine or a sha 1 of the domain depending on the value of anonymize statistics report serial lt positive integer gt Identifies a specific report from a given Cartridge instance This is incremented for every report whet
35. gt lt count gt identified e15 sizes lt count gt lt count gt identified e16 sizes lt count gt lt count gt identified e17 sizes lt count gt lt count gt identified e18 sizes lt count gt lt count gt local header whitelist lt count gt local envelope whitelist lt count gt local body whitelist lt count gt local ip whitelist lt count gt awl header whitelist lt count gt awl envelope whitelist lt count gt awl body whitelist lt count gt awl ip whitelist lt count gt local header whitelist sizes lt count gt lt count gt local envelope whitelist sizes lt count gt lt count gt local body whitelist sizes lt count gt lt count gt local ip whitelist sizes lt count gt lt count gt awl header whitelist sizes lt count gt lt count gt awl envelope whitelist sizes lt count gt lt count gt awl body whitelist sizes lt count gt lt count gt awl ip whitelist sizes lt count gt lt count gt locallawl header host env body entry 1 lt count gt locallawl header host env body entry 2 lt count gt Description Count of messages identified by specific fingerprint algorithms in the last data period These count the sizes of messages identified by various engines in the last data period If all counts are zero the entry will not be included in a report However if any size count for a given whitelist
36. he whole header Header whitelisting is not foolproof because spammers sometimes forge or spoof header information Additionally a header may be added to a spam message when it is relayed by a trusted host and could then be passed to the recipient without being scanned for spam 24 Header whitelisting Chapter 5 Whitelisting Following are example configurations which use header whitelisting to bypass spam filtering for all addresses from the gov and doj org domains respectively type envelope command mail from value gov b type envelope command rcpt to value doj b To match an explicit email address type header header From value buser domain com b Body whitelisting Body whitelisting checks the body of the email to see if it matches the given regular expression This type of whitelisting should only be used in cases where you want to ensure delivery of a message that is considered spam by the Cloudmark community Suppose that you want to whitelist automatic email messages from a house hunting site that contains an HTML header type body regex lt h1 gt HomeHunters lt h1 gt It is important to understand that complex regular expressions when applied to the entire email body can adversely affect performance For this reason it is best to stick to very explicit match patterns that do not use too many or patterns Envelope whitelisting Envelope whitelis
37. her or not it is sent successfully spamdna version lt string gt Identifies the Cartridge version of the instance making the report cmae version lt string gt This identifies the Cloudmark Authority Engine version using this Cartridge It may be unknown meaning that the Cartridge is being used by a non CMAE application or CMAE lt 2 0 What statistics are collected 31 Cloudmark Cartridge Installation and Administration Guide Chapter 6 Table 3 Statistics key value pairs Key value pair application name lt string gt log messages lt base64 encoded block gt message sizes lt size1 gt lt size2 gt lt sizeN gt scored 0 lt count gt scored 1 lt count gt scored 100 lt count gt scored O sizes lt count gt lt count gt scored 1 sizes lt count gt lt count gt scored 100 sizes lt count gt lt count gt Description This identifies the application using this Cartridge It may be unknown meaning that the Cartridge is being used by a non CMAE application or CMAE lt 2 0 It may also be unspecified meaning that a CMAE 2 0 application chose to not provide an application name If present this contains any log messages with severity gt INFO in the last reporting period These log messages are separated by newlines and base64 encoded This shows the message sizes in kilobytes used by any statistics with names sizes These si
38. his message is logged when a downloaded micro update file contains a signature that is already present in the in memory set of signatures This warning can occur in rare 42 WARN log messages Appendix A Logging instances but a large number of warnings indicate a problem with the in memory set of signatures Could not update signature lt sig gt from local set add remove portion Incremental micro update files may contain updates to meta data associated with a signature and this log message indicates a problem updating signature meta data Unable to parse mime parts of message skipping This message is logged when the cartridge is unable to parse the mime parts present in a message In this case the network classifier will not compute fingerprints for the message and the message will be scored with a score of zero Statistics reporting is enabled but customer id is set to none Set it to an address of the form lt companyname gt feedback cloudmark com This message is logged when statistics reporting is enabled but customer id is still set to its default value of none Invalid configuration directory This message is logged when the cartridge is configured with an invalid directory containing whitelist cfg In this case no local whitelist entries will be used Could not find open config file lt file gt lt disk_error gt local whitelist will be disabled This message is logged when the cartridg
39. idge Installation and Administration Guide Chapter 3 Table 1 Micro updates configuration settings Parameter Value s Default micro update Positive integer 80 port micro update Positive integer 60 timeout report statistics yes orno yes use envelope for yes or no no ip information use ip yes or no yes information Description Connect to an alternate port on the host specified by micro update hostname Note that Cloudmark s micro update servers only accept connections on port 80 Specifies the timeout period in seconds for HTTP requests used when checking for micro updates Enables or disables the communication of message scanning statistics to Cloudmark As of Cartridge 3047 this setting is overridden by Cloudmark s back end servers Statistics reporting is now mandatory When this option and use ip information are both set to yes then the sending IP address is derived from the message envelope instead of the Received headers This is a useful option when scoring a message that does not yet contain the Received header that reflects the hop into the MTA When set to yes the Cartridge uses the sender IP address of the message as one of its fingerprints 16 CHAPTER 4 Micro Updates Micro updates is the mechanism that allows the Cloudmark Cartridge to regularly download the
40. ions for other users Micro updates provide the latest known fingerprints as determined by the Cloudmark Global Threat Network By using micro updates you protect your platform against the most current email borne threats See Micro updates below 2 Cloudmark fingerprinting algorithms Chapter 1 Introduction Micro updates Cloudmark stores message fingerprints generated though the Global Threat Network in near real time Micro updates are the mechanism that allows Cloudmark customers to download the latest fingerprint data at regular intervals Micro updates enable Cloudmark to e maintain the highest level of accuracy on spam virus and phishing messages as well as legitimate messages e handle new varieties of email threats proactively and automatically e reduce false positives e eliminate manual message analysis with a fully automated approach To maintain the highest possible accuracy the micro updates feature must be correctly configured For complete information see Chapter 4 Micro Updates Message scoring When the Cloudmark Cartridge scans a message it assigns a spam score as a percentage to indicate the likelihood that the message is an abusive message such as spam phishing or a virus For example if the cartridge assigns a message a score of 99 it means that Cloudmark is 99 certain that message is bad a score of 1 means that Cloudmark is almost certain the message is legitimate When us
41. is over speed gateway received hosts http proxy http proxy basic auth http proxy ntlm auth Value s whitelist or proxy auth or whitelist proxy auth yes orno hostname or IP address host port or ip port user password domain username password Default none no NULL NULL NULL NULL Description If present the corresponding values in statistics reports will be replaced by the string lt excluded gt When set to yes the Cartridge calculates all fingerprints before returning a result for the message When set to no the Cartridge stops when it finds one known fingerprint that matches the message or when it finds a matching whitelist entry This parameter specifies a comma separated list of hostnames or IP addresses of the email gateway hosts in your network The value should match that of the Received header in messages that enter your network from the outside If no value is configured heuristics are used instead Specifies the http proxy for connecting to the Cloudmark micro updates service Specifies user and password for HTTP proxy basic authentication A colon is not allowed in usernames and passwords Enable NTLM authentication with a proxy server A colon is not allowed in domains usernames and passwords 14 Cloudmark Cartridge Configuration Table 1 Parameter image processing depth local addre
42. lt count gt Description lt cat_number gt is the category number lt subcat_number gt is the subcategory number Key value pairs are only sent when lt count gt gt 0 Since subcategories are not always used there may be more messages classified as a particular category than messages classified in that category s subcategories This holds the exact contents of whitelist cfg as read by the Cartridge It is base64 encoded as the file may contain multiple lines blank lines malformed content and so on Count of the number of successful and failed micro update downloads Note that a micro update download includes rpl dpl awl and other files in addition to fingerprints Reports the latest serial number for all micro update data files i e fingerprints rpl dpl and so on If present this key lists th signatures that have been seen by the Cartridge in the last reporting period e lt sig gt lt engine gt names the signature e lt count gt names the number of times the signature was seen This key will only be present for installations that have enabled last seen checking via a hidden cartridge cfg option 34 What statistics are collected Chapter 6 Cartridge Statistics Reporting Table 3 Key value pair clean shutdown 01 installation id lt string gt license id lt integer gt 0 gt Statistics key value pairs Description The license number
43. nd SubCategoryOut is 0 then the cartridge has categorized the message as a virus message The following call provides more information CMAE DescribeCategory unsigned int Category unsigned int SubCategory char CategoryDescOut char SubCategoryDescOut With Category 7 and SubCategory 0 CategoryDescOut would contain an allocated string virus and SubCategoryDescOut would contain an allocated string undefined For detailed information about using this feature in your application see the Cloudmark Authority Engine SDK Guide Cartridge statistics By default the Cloudmark Cartridge sends cartridge configuration information and message scanning statistics back to Cloudmark By collecting this information Cloudmark can more effectively detect potential accuracy issues and proactively address them before there is a need for the customer to contact Cloudmark If your organization has special privacy concerns contact Cloudmark For complete information see Chapter 6 Cartridge Statistics Reporting 4 Message categorization Chapter 1 Introduction Whitelisting A whitelist is a list of trusted senders from whom you always accept email or email characteristics which indicate a trusted message This feature of the Cloudmark Cartridge minimizes the filtering of legitimate messages and allows system administrators to conveniently manage the receipt of messages from known safe senders For complete inform
44. ng MTA must connect directly to the MTA that hosts the Cloudmark product 23 Cloudmark Cartridge Installation and Administration Guide Chapter 5 Following are example configurations that allow mail from hosts with the specified IP addresses to bypass spam filtering type host address 1 2 3 4 type host address 1 2 3 type host address 192 168 32 0 24 In the first example only mail from the host at the exact IP address bypasses spam filtering The second example allows mail to bypass spam filtering if only the first three octets of the IP address match The third example shows an entry in CIDR format l A CIDR mask of 0 is invalid In the case of hostname whitelisting a domain suffix match will be performed to determine if an email should bypass spam filtering Here is an example type host address bar com In this case if the server s IP address resolves to bar com or anything bar com the email will bypass the spam filter As long as there is an exact match on the right side of the hostname then the email will be whitelisted ly is important to understand that host whitelisting does not make use of regular expressions The matches must be exact prefix matches in the case of IP addresses and exact suffix matches in the case of hostnames Header whitelisting Header whitelisting checks the header fields of incoming mail The header whitelist value is a regular expression that is applied to t
45. p_request_str gt lt http_response_str gt This message is logged when the HTTP response code from a statistics report is something other than 200 i e OK If this message occurs the report was not accepted by Cloudmark Could not start background statistics thread will not log or report statistics This message is logged when there is a problem starting the thread that periodically logs statistics and sends statistics reports If this thread does not start collected statistics will not be logged or sent to Cloudmark ERROR log messages 47 Cloudmark Cartridge Installation and Administration Guide Appendix A Could not initialize encryption expect problems reporting statistics This message is logged when there is a problem initializing the module used to encrypt statistics reports You can expect subsequent failures with sending statistics reports CRITICAL log messages lt module gt Could not initialize micro update module This message is logged when there is a problem initializing the module used to interact with micro updates servers Without this module micro update downloads and statistics reporting will be non functional both of which are extremely important in achieving high accuracy Could not initialize per thread storage for message scores Will not be able to return meaningful scores for messages Message scores are maintained in the cartridge in per thread storage and this message is logged if the cartridge coul
46. r examples herein Regular expression support is provided by the PCRE library package which is open source software written by Philip Hazel and copyright by the University of Cambridge England The GIFLIB distribution is Copyright c 1997 Eric S Raymond Cloudmark Inc 128 King Street 2nd Floor San Francisco CA 94107 USA Cloudmark Europe Ltd Carmelite 50 Victoria Embankment Blackfriars London EC4Y ODX UK Cloudmark Cartridge version 3048 Last modified March 11 2008 Jpeglib is copyright C 1991 1998 Thomas G Lane ImageMagick is copyright 1999 2007 ImageMagick Studio LLC PCRE is a library of functions to support regular expressions whose syntax and semantics are as close as possible to those of the Perl 5 language Release 7 of PCRE is distributed under the terms of the BSD licence as specified below The documentation for PCRE supplied in the doc directory is distributed under the same terms as the software itself Written by Philip Hazel lt ph10 cam ac uk gt University of Cambridge Computing Service Cambridge England Phone 44 1223 334714 Copyright c 1997 2004 University of Cambridge All rights reserved Redistribution and use in source and binary forms with or without modification are permitted provided that the following conditions are met Redistributions of source code must retain the above copyright notice this list of conditions and the following disclaimer Redistributions in binary
47. remove the partial file lt mu_file gt part This warning is logged if the partial file cannot be removed Have not updated lt mu_source gt yet accuracy will be affected until content is successfully downloaded from the network This message is logged at cartridge initialization when no suitable files are found below the micro_updates directory As these files are all extremely important in achieving WARN log messages 41 Cloudmark Cartridge Installation and Administration Guide Appendix A high accuracy it is important to wait until files are successfully downloaded from the network before passing messages through the cartridge Can not read information about micro update directory lt cachedir gt lt disk_error gt This warning is logged when the cartridge is unable to read any information about the configured micro updates directory This message will not be logged if the directory does not exist In that case it will attempt to create the directory instead Micro update directory lt cachedir gt must be a writable directory This warning is logged when the configured micro update cache directory exists but is not a writable directory Could not load lt mu_source gt files from disk lt disk_error decode_error gt This message is logged when the cartridge attempts to read cached micro update files from disk but fails This message will be logged only if the cartridge is configured to perform micro update downloads
48. retrieved from Cloudmark s licensing host lt whitelist_module gt Backend whitelist enabled with lt count gt entry entries This message is logged when a awl micro update file is downloaded and loaded that contains any entries of the specified type lt whitelist_module gt enabled with lt count gt entry entries gt This message is logged when whitelist cfg is loaded and contains any entries of the specified type Will not consider empty messages as spam This message is logged at initialization and reflects the option consider empty messages spam in cartridge cfg Micro updates enabled updating all micro update files from network and or disk before continuing This message is logged during initialization when micro updates are enabled The cartridge will attempt to load the latest micro update files from the network and from disk before initialization completes Micro update configuration This message is logged when the micro update module is initialized and gives a summary of micro updates related configuration Attempting to create micro update directory lt cachedir gt This message is logged at cartridge initialization when the configured cache directory does not exist The cartridge will attempt to create the directory Successfully updated lt mu_source gt from disk new serial lt serial gt This message is logged when the cartridge loads a new micro update file from disk it does this at startup
49. rk service currently generates new versions of the full micro updates data files at intervals designed to balance bandwidth usage with the required amount of updates Delta updates are generated at more frequent intervals to complement the micro updates files l New files will only be generated if new data is available Micro updates consist of the file types listed below metadata files The metadata files have extensions of mfl mpl rplv1 dpl awl ipl implv1 csl fsl xrl cfg or cksum txt state files The etc micro updates state directory contains files that track the state of the micro update file set srl_set safe srl_set unsafe and srl_set package signature files The signature files have the extension hs2 z aes Delta micro updates data are not updated in the hs2 z aes file Therefore the serial number of the hs2 z aes file is not updated for each successful delta micro updates download The numerical portion of the zdata txt file can be higher than the serial number of the hs2 z aes file during normal operation checksum files There is one checksum file per signature file These guarantee the integrity of the data Each checksum file also contains the decryption key for its corresponding signature file category file This file defines the categories and sub categories returned by the Cartridge See Message categorization on page 4 e This file may be overwritten by micro updates do not modify it
50. ss for m u cons micro update cache path micro update hostname micro update interval Value s none or low or medium or high IP address File path hostname Positive integer or auto Micro updates configuration settings Default high NULL lt etcdir gt micro updates microupdates cloudmark com auto Description When analyzing images the Cartridge can apply a variety of fingerprinting algorithms some more resource intensive than others The default value of high applies all image specific fingerprinting algorithms achieving the highest possible accuracy If you observe excessive CPU load while processing very high message volumes you can adjust this value to use fewer resources when processing images at the expense of accuracy Forces the Cartridge to bind to the specified IP address for all micro update connections whether direct or proxied Specifies the directory on the local server where micro updates data files will be stored Specifies the hostname to connect to when downloading micro updates l If you change this hostname you must clear the etc lt micro update cache path gt micro_updates directory at the same time Failure to do so may result in degraded accuracy Specifies how frequently in hours to check for the latest micro update if set to auto the software checks for deltas on a periodic basis 15 Cloudmark Cartr
51. t a positive integer Could not shut down networking subsystem lt net_error gt This message is logged on Windows platforms when the cartridge could not shut down networking via WSACleanup Can not start thread for background loader Will only use local micro update files This message is logged at startup if the background task that periodically updates micro update files from the network and disk could not be started Cartridge accuracy will be significantly reduced without current micro update files so the problem should be identified and solved as quickly as possible Could not encrypt statistics report error lt encrypt_errnum gt will not send report This message is logged when there is a problem encrypting a statistics report If this message occurs the statistics report will not be sent to Cloudmark Could not encode encrypted statistics report will not send report This message is logged when there is a problem encoding the encrypted contents of a statistics report If this message occurs the statistics report will not be sent to Cloudmark 46 ERROR log messages Appendix A Logging Could not construct HTTP POST for statistics report will not send report This message is logged when there is a problem constructing the HTTP POST command used to submit a statistics report If this message occurs the statistics report will not be sent to Cloudmark Could not connect to micro updates host lt net_error gt
52. ting checks the SMTP envelope of an incoming message This provides a flexible whitelisting mechanism that will match any substring or regular expression in the specified command field of the envelope Like header whitelisting spammers can forge or spoof the From field to create the illusion that the message comes from a trusted source Following are sample configurations for envelope whitelisting that bypasses spam filtering for messages containing ite in the mail from and recipient to attributes respectively type envelope command mail from value ite b type envelope command rcpt to value ite b Body whitelisting 25 Cloudmark Cartridge Installation and Administration Guide Chapter 5 Note that email address strings contain a leading less than sign lt and trailing greater than sign gt Either of the following examples can be used when searching for an explicit email address for both header and envelope whitelisting type envelope command rcpt to value lt user domain com gt type envelope command rcpt to value buser domain com Suppose that you want to configure a more complex rule to allow mail from the following domain user domain com user primary domain com user secondary domain com but explicitly exclude user machine primary domain com A suitable rule could look like type envelope command rcpt to value domain domain com b
53. to value ite Type body This type of whitelist only applies to the body of the message The value of those entries can be any kind of regex Be careful as to what is ntered here since the most complex regexes will Sample whitelist configuration file 27 Cloudmark Cartridge Installation and Administration Guide Chapter 5 affect performances type body regex CLOUD ARK 28 Sample whitelist configuration file CHAPTER 6 Cartridge Statistics Reporting The Cloudmark Cartridge collects statistics about message classification and reports these statistics to Cloudmark These statistics are used in conjunction with feedback data collected from the Cloudmark Network Feedback System CNES to provide customers and Cloudmark with visibility into filtering accuracy at customer sites las of Cartridge 3047 statistics reporting is mandatory If your organization has special privacy concerns contact Cloudmark This chapter includes the following topics e How statistics are reported to Cloudmark below e What statistics are collected on page 31 e Cartridge reporting configuration on page 35 How statistics are reported to Cloudmark Statistics are always collected at each Cartridge installation Collected statistics are not written to disk and they are not accessible at an installation
54. to indicate that micro updates will no longer be downloaded and saved The option micro update filename has been deprecated ignoring These messages are logged at startup when either of these deprecated options are present in cartridge cfg Could not connect to host lt host gt 80 addr lt addr gt lt net_error gt This message is logged when the cartridge attempts to connect to the micro updates host on port 80 but fails Ifan HTTP proxy is configured the cartridge only makes a single connection attempt to the configured HTTP proxy Network feedback whitelist hit for lt email gt This message is logged when Cloudmark s network feedback system has whitelisted a message for a specific email address These messages should only occur if your installation is submitting feedback through Cloudmark s Network Feedback System and this message is similar to false positive feedback received for that specific address Current statistics for period lt report_id gt These messages comprise a statistics report and are logged at the end of every reporting period Shutting down stopping background stats reporting This message is logged at shutdown 40 INFO log messages Appendix A Logging Statistics initialized Current settings This message is logged when the statistics module is initialized and gives a summary of statistics related configuration Connecting to licensing host without authentication Cloudmark s lic
55. tridge release notes for updates to these settings You can configure Cartridge parameters in the cartridge cfg file located in the etc directory in the root directory of your Cloudmark installation using any text editor Table 1 below lists the configuration parameters as specified in cartridge cfg All configuration parameters are case insensitive If cartridge cfg is not available the default settings are used Table 1 Micro updates configuration settings Parameter Value s Default consider empty yes orno no messages spam customer id Company name none or CNFS login download yes orno no micro updates before init enable yes orno yes micro updates Description Specifies whether messages with empty subject and empty body or a subject or body consisting only of whitespace are considered spam Specify your organization s name This helps identify your reports when they reach Cloudmark If you are a Cloudmark Network Feedback System CNFS customer use your CNFS logon name for this parameter Specifies whether micro updates are downloaded before startup Enables or disables the download of micro updates over the network from the micro update host Setting this option to no will adversely affect accuracy 13 Cloudmark Cartridge Installation and Administration Guide Chapter 3 Table 1 Micro updates configuration settings Parameter exclude from stats reports favor analys
56. ty 1 ee ee ee ee 21 Advanced micro update configurations 21 Using an HTTP proxy s 2 a ee a ee 21 Whitelisting s e a 6 a n ae m g a k23 Host whitelisting 2 6 a ee ee 28 Header whitelisting 6 6 6 ee ee 24 Body whitelisting ee ee 0 ee 25 Envelope whitelisting 2 6 1 6 6 ee ee ee 25 Sample whitelist configuration file 27 Cartridge Statistics Reporting 29 How statistics are reported to Cloudmark 29 What statistics are collected ee ee ee 31 Cartridge reporting configuration 35 LOGGING sos p mw amp amp eZ Common variables 1 6 a37 INFO log messages 6 1 6 ee 38 WARN log Messages ee ee 41 ERROR log Messages ee ee ee 44 CRITICAL log messages ee 48 Contents Index gt a a sa e ec a e a AR Contents vi CHAPTER 1 Introduction Cloudmark s gateway solutions use the Cloudmark Cartridge to deliver the latest Cloudmark anti abuse technology for your email platform This guide explains how to install configure
57. ugh the proxy server For configuration information see Using HTTP proxies on page 19 Advanced micro update configurations 21 Cloudmark Cartridge Installation and Administration Guide Chapter 4 22 Advanced micro update configurations CHAPTER 5 Whitelisting The Cloudmark product provides system level whitelisting support allowing you to pass messages automatically based on domains IP ranges envelope header or body features Whitelisting configuration settings are stored in the file whitelist cfg located in the etc directory This chapter introduces the four types of whitelisting plus a sample configuration e Host whitelisting below e Header whitelisting on page 24 e Body whitelisting on page 25 e Envelope whitelisting on page 25 e Sample whitelist configuration file on page 27 l por products based on versions CMAE SDK prior to 2 0 changes to the whitelist take effect upon a restart of the product CMAE SDK 2 0 has the ability to reload whitelists without a restart Host whitelisting Host whitelisting checks the IP address or the domain of the mail server from which a message is being received While host whitelisting is a highly effective anti spam measure it requires that the Cloudmark product have the address of the connecting MTA This usually means that the connecti
58. zes are used to break down statistics by size into buckets lt size1 gt lt size2 gt lt size2 gt lt size3 gt lt sizeN gt infinity This statistic may not be present for initial statistics reports that is when the report serial number is 1 Counts of the number of messages in the last data period with scores from 0 to 100 To save space non zero counts may not be included in a stat report These count the sizes of messages scored with scores 0 through 100 in the last data period If all counts are zero the entry will not be included in a statistics report However if any size count for a given score is non zero all sizes will be included 32 What statistics are collected Cartridge Statistics Reporting Table 3 Statistics key value pairs Key value pair identified e4 lt count gt identified e7 lt count gt identified e8 lt count gt identified e9 lt count gt identified e10 lt count gt identified e14 lt count gt identified e15 lt count gt identified e16 lt count gt identified e17 lt count gt identified e18 lt count gt identified as empty lt count gt identified e4 sizes lt count gt lt count gt identified e7 sizes lt count gt lt count gt identified e8 sizes lt count gt lt count gt identified e9 sizes lt count gt lt count gt identified e10 sizes lt count gt lt count gt identified e14 sizes lt count
Download Pdf Manuals
Related Search