 |
Version 5.1 |
|
|
E-Mail Transfer
One of the main functions of the CommuniGate Pro Server is E-mail Message transfer. Acting
as an MTA (Mail Transfer Agent), the Server accepts messages from various sources (modules,
internal components, etc.), and delivers (transfers) them to remote or local destinations
using the same or different modules.
While all submitted messages are stored as individual files in the Queue directory inside
the CommuniGate Pro "base directory", each message can be enqueued into several different queues
(if it has several recipients). Each communication module can maintain one or several logical queues.
For example, the SMTP module maintains one queue for each Internet domain.
|
|
|
E-mail Sources and Destinations
The CommuniGate Pro Server has the following set of message sources:
- The SMTP module submits messages received from mailer applications
and from other mail servers via the Internet. The Submit Address tag used: SMTP.
- The RPOP module submits messages retrieved from remote POP servers.
The Submit Address tag used: RPOP.
- The Local Delivery module submits messages generated
with the Account-level (Account and Domain) Automated Mail Processing Rules.
The Submit Address tag used: RULE.
- The Local Delivery module re-submits messages directed to 'all' addresses and
redirected to all forwarders in the specified domain.
The Submit Address tag used: ALLF.
- The PIPE module submits messages received from external applications via interprocess communication channels, and the messages
generated and stored in the special Submitted directory. The Submit Address tag used: PIPE.
- The POP module submits messages received from certain mailer applications employing the XTND XMIT protocol extension. The Submit Address tag used: POP.
- The IMAP module submits messages received from MAPI client applications. The Submit Address tag used: IMAP.
- The WebUser Interface module submits messages composed within Web browsers. The Submit Address tag used: HTTP.
- The XIMSS module submits messages composed within XIMSS clients. The Submit Address tag used: XIMSS.
- The Real-Time Applications submit messages such as incoming voicemails. The Submit Address tag used: PBX.
- The Enqueuer component submits messages generated with the Server-wide and Cluster-wide Automated Mail Processing Rules.
The Submit Address tag used: SRULE.
- The Dequeuer component generates and submits delivery notification messages.
The Submit Address tag used: DSN.
- The WebUser Interface and XIMSS modules generate and submit Message Disposition Notification ("Read Recipts") messages.
The Submit Address tag used: MDN.
- The Calendaring component generates and submits Event Request Reply messages. The Submit Address tag used: ICAL.
- The LIST module submits messages to be distributed to mailing list subscribers. The Submit Address tag used: LIST.
- The LIST module submits reply messages generated when processing administration requests. The Submit Address tag used: LSRV.
- The LIST module submits messages directed to individual subscribers (Warnings, Hello, Bye, etc.)
The Submit Address tag used: LSTM.
- The LIST module submits messages directed to the list owner. The Submit Address tag used: LSTO.
- The LIST module re-submits messages directed a Group. The Submit Address tag used: GROUP.
- The Triggers component generates and submits notification messages. The Submit Address tag used: EVENT.
- The Lawful Interception component generates and submits report messages. The Submit Address tag used: LWIT.
The CommuniGate Pro Server transfers messages to the following destinations:
- the SMTP module transfers messages to other SMTP mail servers via the Internet;
- the LIST module accepts and processes messages with mailing list postings, and with various list administration requests;
- the Local Delivery module transfers messages to local user mailboxes;
- the PIPE module transfers messages to external applications via interprocess communication channels;
- the Queue module itself transfers (discards) messages routed to NULL or ERROR addresses;
The following diagram illustrates the message flow inside the CommuniGate Pro Server:
Submitting Messages
All messages are created as temporary files. They are stored in the Queue directory as files
with the .tmp extension. A module or an internal component stores the message envelope and
the message itself in such a file and then submits it to the Enqueuer component for processing.
The message envelope is a set of text lines. Each line specifies either the message return-path,
or one message recipient address, or message delivery options.
If a module fails to compose a message (for example, an SMTP connection breaks during
message transfer), the module discards of the temporary file.
When a message is completely composed and submitted to the Enqueuer, the file extension is
changed to .msg and the message is scheduled for processing.
When the Server restarts, the Enqueuer component finds all files with the .msg extension
stored in the Queue directory and resubmits them for processing.
Open the General pages in the Settings realm of the CommuniGate Pro WebAdmin Interface, and
find the Temp File Manager panel panel on the Others page:
- Log
- This setting specifies what kind of information the Temporary Files manager
should put in the Server Log. Usually you should use the Failures
(file system error reports) level. But when you experience a problem with some module submitting messages,
you may want to set this setting to Low-Level or All
Info: in this case file i/o operations will be recorded
in the System Log as well. When the problem is solved, set the TempFiler Log
setting to its regular value, otherwise your System Log files will grow
in size very quickly.
The Temporary Files manager Log records are marked with the TEMPFILE tag.
- Recycle Temp Files
- Enable this option to improve performance of your system under heavy load: discarded
temporary file are not deleted, and they are reused.
Queue Limits and Foldering
If a Server is heavily loaded, it can contain thousands of message files in its Queue directory.
Many operating and file systems cannot efficiently process large directories.
You may want to split the Queue directory into several subdirectories, each containing a portion of Temporary and Queue files.
You may want to limit the total number of messages in the CommuniGate Pro Queue.
When this number is reached, the modules reject attempts to submit new messages.
Use the WebAdmin Interface to specify the Queue processing options.
Open the Mail pages in the Settings realm, then open the Queue page:
- Queue Limit
- If you specify a limit with this option, the CommuniGate Pro modules (SMTP, PIPE, RPOP, MAPI, WebUser) will stop
accepting new messages when the number of messages in the Queue exceeds the specified limit.
- Queue Foldering
- This setting specifies where the CommuniGate Pro Server will create new Temporary files, and,
as a result, how the message files will be stored in the Queue Directory.
If you select the 0 (zero) value, no subdirectory will be created in the Queue directory, and all Temporary
files will be created directly in the Queue directory.
If you select the 10 or 100 value, subdirectory with NN names (where NN is a 2-digit number from 00 to 99) will be
created in the Queue directory. The newly created Temporary files will be created inside these subdirectories.
The server will use 10 or 100 subdirectories, depending on this setting value.
If you select the Auto value, Queue subdirectories will be used when the total number of
messages in the Queue exceeds 5000.
Routing
When a message is submitted for processing, the Enqueuer component examines its envelope information.
Each recipient address is parsed and passed to the Router component.
The Router component decides which module or component should process each recipient address.
Enqueueing
When all recipient addresses are parsed and routed, the Enqueuer component applies the Server-Wide Rules to
the message. Then it passes the message to the modules specified with the Router component.
Communication modules do not process E-mail messages immediately, but enqueue them into the module-specific queues.
The SMTP module creates and maintains a queue for each Internet domain,
the Local Delivery module creates and maintains a queue for each local Account, etc.
The Enqueuer component can enqueue messages:
- synchronously: as soon as messages are composed and submitted, Routing and Sever-wide/Cluster-wide Rules are
applied, and the message composer component is informed if the message has been rejected (for example, if
an External Filter has found a virus in the message). For example, the SMTP incoming channel sends a negative
response to the sender of an infected message, the message is not enqueued, and no error report (bounce) messages
are generated for infected messages.
- asynchronously: the composer component immediately receives a positive response, and it can continue
processing without waiting for the Enqueuer component. For example, the SMTP incoming channel can immediately
start receiving the next message.
Most of the internal components enqueue messages asynchronously, as they cannot do anything
useful if a message is rejected with the Enqueuer component. The components receiving messages
directly from users or remote systems (SMTP, MAPI, WebMail, XIMSS) try to enqueue messages synchronously,
so if a message is rejected with the Enqueuer component, the submitting agent (a user or a remote system)
can get an error message.
Use the WebAdmin Interface to configure the Enqueuer component. Open the Queue page in the Settings realm.
- Enqueuer Log
- Use this setting to specify what kind of information the Enqueuer component
should put in the Server Log. Usually you should use the Failures
(file system error reports) level.
The Enqueuer component Log records are marked with the ENQUEUER tag.
- Enqueue Asynchronously
- Select this option to use the asynchronous Enqueuer mode for all components, including
those directly receiving messages from users or remote systems.
- Processors
- Use this setting to specify the number of Enqueuer processors (threads).
If the Enqueue Asynchronously option is not selected, the Enqueuer threads are
needed only to enqueue existing messages after the Server restart, and to enqueue messages
generated with the internat Server component, so you may want to use 2-3 threads only.
If the Enqueue Asynchronously option is selected, then each message is processed with the
Enqueuer thread. In this case, 5-10 Enqueuer threads should be enough even for a heavily loaded Server.
You should increase the number of Enqueuer processors if:
- there are very many Server-Wide Rules;
- Server-Wide Rules use the Execute action to start external programs;
- one or more External Filter (anti-virus, content filtering, etc.) programs are enabled;
- incoming load is very high, with many messages being submitted every second via the SMTP or other modules.
The numEnqueuerMessages SNMP element shows the number of messages that have been recieved, but
not enqueued yet. If this number is growing, you need to increase the number of Enqueuer processors.
- Hop Counter Limit
- When a message is being received by any host or module, it gets an additional Received: header field.
The Hop Counter is the number of Received: header fields in the message. If a message contains too many
Received: header fields, it may indicate that a message is in some sort of mail loop. This parameter specifies the
Limit for the Hop Counter - any message that has more Received: header fields than specified in this setting is
rejected by the ENQUEUER - without any attempt to deliver it to the recipients specified in the message envelope.
Delays and Suspensions
When a communication module fails to transfer a message, it uses the kernel queue management component to delay processing.
- a module can delay an entire queue: for example, the SMTP module can delay a queue created for an Internet domain, if it
cannot connect to that domain or its relays;
- a module can delay an individual queued message: for example, the SMTP module can delay a message if the receiving host rejects
this particular message (transition failure);
- a module can delay an individual recipient address in a queued message: for example, the SMTP module can delay an address if the
receiving host rejects that particular address (transition failure).
Dequeueing
When a communication module transfers a message or when it rejects a message because of a fatal error, it removes the message
from the module queue. The module composes a delivery report and passes it to the Dequeuer component.
The Dequeuer component processes delivery information. If requested, it composes Delivery Status Notification (DSN) messages
and submits them back to the system for delivery to the original message sender. When a message has several recipients, the
Dequeuer component may choose to delay DSN generation, so each DSN message can contain reports about several recipients.
When all message recipients are processed and the message is dequeued from all queues, the Dequeuer component removes the message file
from the Queue directory.
Use the WebAdmin Interface to configure the Dequeuer component. Open the Queue page in the Settings realm.
- Dequeuer Log
- Use this setting to specify what kind of information the Dequeuer component
should put in the Server Log. Usually you should use the Major & Failures (delivery reports) level.
The Dequeuer component Log records are marked with the DEQUEUER tag.
- Processors
- Use this setting to specify the number of Dequeuer processors (threads). Usually one Dequeuer
thread is enough even for a heavy-loaded server. Only if your Server performs some kind of
special message processing and has to generate a lot of DSN messages, should you use several
Dequeuer threads.
- Reporting Delay
- Use this setting to specify the maximum delay between the moment when a message was transferred or failed and the
moment when a delivery report is generated. The more the delay, the more reports can be placed in one DSN message.
A DSN message is generated immediately after the last message recipient is processed.
- On Failure, Return
- Use this setting to specify what portion of a failed message should be included into the
DSN (error report) message.
- If the sender has not specified this option explicitly,
and the headers by default option is selected, only the failed message headers will be returned;
- If the sender has not specified this option explicitly,
and the body by default option is selected, the entire failed message will be returned;
- If the always headers option is selected, only the message headers are included into the
DSN message, even if the message sender has specified that the entire message should be returned on failure;
- If the always body option is selected, the entire message is included into the
DSN message, even if the message sender has specified that only the message headers should be returned on failure.
- Copy Failure Reports
- When this option is enabled, all error messages generated with the CommuniGate Pro Dequeuer are
sent to both the failed message return-path and to the specified E-mail address.
- Send Return-Receipts to
- Positive reports (delivery reports and relay reports) are generated only if the message sender has requested them.
Use this setting to specify who can request these reports:
| everybody | reports are generated and sent whenever the message sender requests them. |
| clients | reports are generated and sent if the sender has submitted the message from a Client IP Address or if
the message has been submitted by an authenticated user. |
| authenticated | reports are generated and sent if the message has been submitted by an authenticated user
(via WebUser, XIMSS, SMTP AUTH, MAPI, etc.) |
| nobody | positive reports are not generated. |
Monitoring a Queue
Transfer modules (such as SMTP, Local Delivery,
LIST, and PIPE) maintain one or several queues for messages
to be delivered. Each module uses its own methods to build the queues (for example, the SMTP module
usually builds a separate queue for each remote domain to deliver to, while the Local Delivery module
builds a separate queue for each local Account), see the manual section describing the module for more details.
To open a module queue, click the queue name link on module Monitor page. The module ("host") queue page opens:
The table header contains the information about the entire queue ("host"):
- Module
- The link to the Monitor page of the Transfer module this queue belongs to.
- Status
- Active - this queue is being processes by the module.
Ready - this queue can be processes by the module at any time.
Delayed till time - this queue will not be processes by the module until
the time specified. The queue can be delayed because there was a queue-wide transfer operation error
(an SMTP host did not respond, or a Local Account is over its quota, etc.)
- Last Error
- This field indicates the last queue-wide transfer operation error.
The table contains a record for each message in the queue. For each enqueued message, the
following information fields are displayed:
- Message
- The message internal ID. You can use this link to open the Message Monitor page.
- In Queue
- The time the message has spent in this queue.
- Return Path
- The message envelope "Return-Path" address.
- Recipients
- The number of message addresses that should be delivered to the "host" this queue is created for.
For example, a message directed to user1@company.dom and user2@company.dom addresses via SMTP
will appear once in the company.dom SMTP queue, with the indicated number of addresses being 2.
- Size
- The message size.
- Delayed and Last Error
- If delivery failed for this particular message, the Transfer module could delay this message
individually (rather than delaying the entire queue). In this case, this field will show the time
when the module will try to deliver the message again, and the Last Error field will show the information
about the cause of the transfer operation failure.
If you have the Can Release Queues access right, and the
Queue has the Delayed status, the Host Queue Monitor page contains the Release Now button. Click
this button to remove the delay interval set for this queue and to allow the module to process
the queue immediately.
If you have the Can Reject Queues access right, and the
Queue has the Delayed or Ready status, the Host Queue Monitor page contains the Reject Host Queue button.
Click this button to reject the queue:
The specified text is used to generate DSN messages (error reports) for all messages placed into this queue.
If the Suppress Failed Delivery Reports option is selected, no DSN message is generated when
the queue messages are being rejected.
Monitoring a Message
To monitor the status of a message in the Server Queue, click the message link on
the Module queue or other Monitor page. The Message Monitor page opens:
The first part of the page shows the message attributes: the envelope Return-Path, Message-ID,
Envelope-ID (if present), message Subject, and the time the message was submitted to the Server Queue.
The second part of the page lists all active message recipient addresses (if a message address
has been already processed, it is not shown). For each address the following information is
displayed:
- Recipient
- Recipient address.
- Module
- The name of the module that will process this address. This is also a link to the
module Monitor page.
- Queue
- The name of the module queue containg this address. This is also a link to the
module queue Monitor page.
- Status
- Processing - the module is processing this address.
Ready - the module can start processing of this address at any time.
Suspended - the module has delayed processing of this message.
Delayed - the module has delayed processing of the entire queue containing this address.
- Retry In
- Time till the module resumes processing of this message or the queue containing this message.
If you have the Can Reject Queues access right,
the Message Monitor page contains the Reject Message button.
Click this button to reject all active message addresses.
Only the addresses that are not being processed can be rejected.
The specified text is used to generate DSN messages (error reports) for all rejected recipient addresses.
If the Suppress Failed Delivery Reports option is selected, no DSN message is generated when
the message addresses are being rejected.
CommuniGate® Pro Guide. Copyright © 1998-2007, Stalker Software, Inc.
