Knowledge Base Home

Call Support


Search the Knowledge Base

Skip to end of metadata
Go to start of metadata

Environment

Netmail Archive (all versions)

Synopsis

When executing archive jobs, Netmail Archive runs multiple threads, each thread processing a single mailbox from the list of users assigned to that job. Each archive server in a Netmail Archive cluster can run up to 10 threads by default, although this maximum can be changed (if memory and CPU allow) via the Node settings in the WebAdministration console:

 

As a result a single Archive job can run many concurrent threads across a cluster.

In some scenarios it may be necessary to limit the maximum number of threads that can connect to any single GroupWise POA or Exchange Client Access Server. For example, it may be necessary to limit the maximum number of running threads to avoid potential overloading of network links to remote mail servers, or to avoid overloading the overhead on the mail server itself (some implementations of GroupWise, especially on Windows, have been known to crash the POA if a large number of SOAP threads are established).

Netmail Archive provides a method for controlling the maximum number of concurrent threads that can connect to any single mail system server during a particular job.

This article will explain how to do this.

Solution

In order to limit the number of concurrent threads that run against a mail server being processed by an archive job, additional settings must be added to the ClusterConfig.xml file. The default location of this file is the C:\Program Files (x86)\Messaging Architects\Config\ directory.

 

Note:

If there are multiple archive servers in a Netmail Archive cluster, it is only necessary to edit the ClusterConfig.xml file on the Master Archive node because all changes will be automatically replicated to the other nodes in the cluster. To trigger replication, increment the value of the GWOpenConfig id located at the top of the file:

 

Load balancing setting lines should be added towards the end of the ClusterConfig file, in the ThreadPoolDefinition section; the necessary lines should be placed between the closing of the Admin section (labeled </Admin>) and the closing of the final ThreadPoolDefinition section (labeled </ThreadPoolDefinition>) as indicated in the following example:

</PO>
</Admin>
<LoadBalancers>
<LoadBalancer id="0" type="MA.JobManager.POLoadBalancer" bin="MAClusterManager" agenttype="0" triggername="*">
<Parameters><![CDATA[<Param><Parameter name="MaxPo" value="3"/></Param>]]></Parameters>
</LoadBalancer>
</LoadBalancers>
</ThreadPoolDefinition>
</GWOpenConfig>

 

The only settings within the Load Balancer settings which should be customized are:

LoadBalancer id – used to define Load Balancers, and must be unique when using multiple Load Balancer definitions (see later scenario examples below).

triggername – this setting defines which Archive Agents (and their jobs) that this parameter is applied to. A wildcard setting (“*”) will match all Archive Agents. To implement a load balancer for a specific job, enter a more specific triggername value; for example, a value of “Label1” will only apply this load balancer’s setting to jobs which contain the text string Label1 anywhere within their name.

MaxPo value – the value listed for the MaxPo parameter will define the limit (across the entire cluster) for the maximum number of threads running against any single mail system server by any single archive job matched by the triggername.

 

NOTE: In order for any setting changes to take effect, the Netmail services need to be restarted.

 

The operation and usage of these settings is best illustrated with some examples as provided in the two scenarios below.

 

SCENARIO 1 – Global Load Balancing
Company A is running a single daily archive job to archive users from all of their ten GroupWise Post Offices. The job executes on all five of the Archive nodes in their Netmail Archive cluster. They want to ensure that no more than 10 concurrent threads will run against any single post office during this job’s or any archive job’s execution.

To achieve this, the Load Balancer settings should be as follows:

<LoadBalancers>
<LoadBalancer id="0" type="MA.JobManager.POLoadBalancer" bin="MAClusterManager" agenttype="0" triggername="*">
<Parameters><![CDATA[<Param><Parameter name="MaxPo" value="10" /></Param>]]></Parameters>
</LoadBalancer>
</LoadBalancers>

This is set to apply to all Archive jobs (by use of the wildcard “*” triggername, and will restrict operation to a maximum of 10 concurrent threads against any single POA.

 

NOTE:

The number of available archive nodes is irrelevant to these settings - it is the maximum number of allocated threads that is relevant. The default limit of 10 concurrent threads applies, regardless of the number of nodes. However, since the time required to archive different user accounts varies, and Netmail Archive allocates threads across the cluster in a way designed to balance the total workload, the distribution of threads for a single POA cannot be predicted. When a job starts, it processes many users from a single Post Office and it is likely that two threads will be launched on each of the five archive servers for users on that POA, to a maximum of 10. As the job progresses, the thread distribution may change, but will always be limited to a maximum of 10 threads (the default, unless, of course, the maximum number was configured to be higher).

 

Once a job reaches the maximum number of allowed threads for a particular POA, that job’s user list will be scanned until a user from a different POA is found, and a thread will launch for that user (as long as the maximum thread capacity assigned to the cluster is not exceeded, which by default is 10 threads per server). If the load balancer limit is reached for all Post Offices that a job is servicing, no more threads can be launched. As such, care should be taken not to specify a MaxPo value which is too low, as this could result in underutilization of the Archive nodes.

 

SCENARIO 2 – Setting Specific Post Office Limits
Company B has four central GroupWise Post Offices and another Post Office at a remote site at the end of a low bandwidth connection. They wish to restrict the number of threads that connect to the remote Post Office during their nightly archiving to two threads to avoid saturating the bandwidth of the link; a scenario like this would also apply to cases where the remote Post Office is on a server with minimal hardware specifications so the number of running threads should be limited to avoid excessive overhead on the server itself.

They also want to restrict connections to the central Post Offices to a maximum of 20 threads.

In this scenario, two different limits must be applied to different Post Offices, so two Load Balancers must be configured in the ClusterConfig.xml files, and archiving must also be split into two jobs (different limits cannot be applied to different post offices within the same job). The configuration file would be edited as follows:

<LoadBalancers>
<LoadBalancer id="0" type="MA.JobManager.POLoadBalancer" bin="MAClusterManager" agenttype="0" triggername="Remote-PO">
<Parameters><![CDATA[<Param><Parameter name="MaxPo" value="2" /></Param>]]></Parameters>
</LoadBalancer>
<LoadBalancer id="1" type="MA.JobManager.POLoadBalancer" bin="MAClusterManager" agenttype="0" triggername="Central-PO">
<Parameters><![CDATA[<Param><Parameter name="MaxPo" value="10" /></Param>]]></Parameters>
</LoadBalancer>
</LoadBalancers>
 

By creating two archive jobs, “Remote-PO-Archive-Job” and “Central-PO-Archive-Job,” different limits can be applied to each job since the triggername values of the load balancers will be compared to the job names and the appropriate limit applied.

As long as archive jobs that process the remote post office contain the string “Remote-PO” in the job name, those jobs will always be limited to 2 concurrent threads. Similar naming rules will apply when naming archive jobs associated with the Central Post Offices to allow 10 concurrent threads.

 

Summary Points

• Since multiple Load Balancer settings can be added to the ClusterConfig.xml file, each Load Balancer must be allocated a unique LoadBalancer id number.

• Where multiple Load Balancers are used, a specific archive job will use the first Load Balancer listed in the ClusterConfig file that has a triggername value that matches its job name. As such, care should be taken in the ordering of Load Balancers and triggername usage.

• The MaxPO value setting applies as a limit across the entire Netmail Archive cluster and is not applied per archive server. A value of 10 does not mean that each archive server will run a maximum of 10 threads against any mail server, but rather that a maximum of 10 concurrent threads will run across all archive nodes against any mail server.

• Finally, it is important to note that the Load Balancer applies on a job-by-job basis. If two jobs run concurrently (both of which are subject to a Load Balancer and are processing the same mail servers), then the limits apply to each job and as such, there may be twice as many threads running against a mail server.

Notes


Help us improve!
Is this article helpful?
Is it well written?
Is the content complete?