Child pages
  • Configuring External Logging

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Migrated to Confluence 5.3
Noprint


Section
Column
width40%
Panel

 On this page:

Table of Contents
maxLevel3
Column
width60%

 

This section deals with setting up the [log] section of your configuration.

Anchor
obscuring
obscuring
Obscuring UUIDs in Logs

The log.obscureUUIDs parameter enables you to determine whether or not entire UUIDs display in logs.

For example:

With log.obscureUUIDs = false (the default), the entire UUID displays as follows:

<183>2011-11-15 02:08:36,300 SCSP DEBUG: REQUEST: GET 172.16.0.33
<None>/358a76f06ffe7e4128d5e6c467a2a96d?alias=true (request:3087553833379006269
connection:25579392) 172.16.0.32 14/11 20:08:37.065

With log.obscureUUIDs = true, 20 of the 32 characters of the UUID display, as follows:

<183>2011-11-15 02:08:36,300 SCSP DEBUG: REQUEST: GET 172.16.0.33
<None>/358a76f06f...c467a2a96d?alias=true (request:3087553833379006269
connection:25579392) 172.16.0.32 14/11 20:08:37.065

Anchor
configuring
configuring
Configuring the Logging Host

The log.host option enables Netmail Store log messages to be sent to a central syslog, rsyslog, or syslog-ng server. The examples here show the specific Netmail Store items that need to be added to the UNIX configuration files for syslog or syslog-ng. The actual configuration files will likely contain additional information for logging messages from other hosts and other logging facilities. See syslogd reference and syslog-ng reference for more information.

To configure your Syslog server:

1. Set the syslog or syslog-ng daemon options to enable logging from a remote host.

  • syslog

Edit /etc/sysconfig/syslog to the SYSLOGD_OPTIONS to the following:

SYSLOGD_OPTIONS="-r -m 0"

  • syslog-ng

An example is shown in the next step.

2. Configure other logging options.

  • syslog

This example shows a sample configuration with the standard syslog program. When editing the syslog.conf file, remember that facility.level must be followed by a Tab character to separate it from the destination specification. See the syslog.conf manual page for more information.

# /etc/syslog.conf
local6.* /var/log/castor.log

  • syslog-ng

This example shows a sample syslog-ng configuration. See the syslog-ng.conf manual page for more information.

a. In the filters section, add the following:

#CAStor filter
filter f_castor {facility(local6);};

If a filters section does not exist, add one after the source statement and before the destinations section.

b. In the destinations section, add the following:

#CAStor destination
destination d_castor {file("/var/log/castor.log");};

c. Add the following log statement:

#Caringo-specific additions #
log {source(s_net){ udp();}; filter(f_castor); destination(d_castor);};

Note

Note: The statement (source(s_net){ udp();}; is the default remote service source for syslog-ng. If you use a different remote source name, replace (source(s_net){ udp();}; with the name you used.

d. Enter the following commands to restart the service with your changes:

sudo /etc/init.d/syslog-ng stop
sudo /etc/init.d/syslog-ng start

3. Set Netmail Store logging options in the node or cluster configuration file.

  • log.host. The fully qualified host name or IP address of your syslog or syslog-ng server.
  • log.port. The server port.
  • log.level. Set the log level, as described in the table below.
Log LevelMeaning
0No logging
50Critical messages and announcements
40Errors, critical messages, and announcements
30Warnings, errors, critical messages, and announcements
20Info, warnings, errors, critical messages, and announcements
10Debug, info, warnings, errors, critical messages, and announcements (the most verbose log level)

Examples:

• Fully qualified parameters:

log.host = 192.168.0.100
log.port = 514
log.level = 40

• Section/unqualified parameters:

[log]
host = 192.168.0.100
port = 514
level = 40

Configuring an Rsyslog Server

You can configure an rsyslog server running Red Hat® Enterprise Linux® or CentOS operating system to accept incoming syslog messages from Netmail Store.

To configure the syslog server:

1. Log in as a user with root privileges.

2. Execute the following command: vim /etc/rsyslog.conf

3. In the rsyslog.conf file, comment out the following lines to accept inbound UDP connections on port 514:

  • $ModLoad imudp.so
  • $UDPServerRun 514

4. Edit the file so the timestamp and IP address of incoming syslog messages appear.

a. Locate the following text:

#### GLOBAL DIRECTIVES ####
Use default timestamp format
$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat

b. Change this text to the following:

#### GLOBAL DIRECTIVES ####
$template myFormat,"%timestamp:::date-rfc3339% %fromhost-ip%
[%programname%]\9 %syslogseverity-text%\9: %msg%\n"
$ActionFileDefaultTemplate myFormat

5. (Optional) Create a log file for each Caringo product by configuring a log file per logging facility:

local5.* /var/log/caringo/cr.log
local6.* /var/log/caringo/castor.log

6. (Optional) Create a log file based on any desired string in the log message using the :msg parameter.

For example, to create a log file that only includes messages with the word "Trims", use the following format:

:msg,contains,"Trims" /var/log/caringo/trims.log

The result would match the following messages:

2012-05-25T16:00:51.625861-05:00 10.1.1.153 [21] debug : 00:51,602 HP
DEBUG: Trims decidable locally / trims needed: 0/0
2012-05-25T16:00:52.507941-05:00 10.1.1.153 [21] debug : 00:52,484 HP
DEBUG: Trims decidable locally / trims needed: 0/0

7. Check iptables and Security-Enhanced Linux (SELinux) to verify that you are not blockinginbound port 514.

8. Restart the rsyslog process by executing the following command:

service rsyslog restart

Configuring an External Time Server

Precisely synchronized time is critical to the integral processes in the Netmail Store storage cluster, such as versioning, lifepoints, and updates. If the nodes in a storage cluster are not synchronized with each other, you may experience unexpected results.

Guidelines for Time Servers

Netmail Store requires extremely precise clock synchronization to prevent data loss. Follow these guidelines to ensure adequate synchronization:

  • Use Trusted NTP Servers: Be sure to use trusted NTP servers, whether they are dedicated hardware solutions in your internal network or external, public servers.
  • Synchronize client systems: Netmail Store does not synchronize the client system clocks. Best practice is to synchronize these clocks with the NTP servers as well. However, client system configuration is not required.
  • No virtual deployments in production: Clock drift in virtual environments may prevent Netmail Store from being able to properly sequence updates; therefore, virtual deployments are not supported for production use, only for test and development clusters.

Configuring a Node with NTP

To configure a node to use NTP:

  • Implement the timeSource parameter in the node or cluster configuration file by setting the network.timeSource parameter to one or more IP addresses or host names.
Note

Note: If the nodes do not have network access to public NTP servers, they will eventually time out waiting for a connection and automatically restart. Ensure that the timeSource parameter is set correctly or the nodes have network access to the NTP pool servers.

For best results, set the network.timeSource parameter to one or more IP addresses or host names based on the DNS server configuration in the network.

  • If the node is not configured to a DNS server, set the network.timeSource parameter to one or more IP addresses:

network.timeSource = 192.168.0.20 192.168.0.50 192.168.0.110

The dnsServers parameter must be set to a valid value.

  • If the node uses DHCP that provides a DNS server to resolve host names, set the network.timeSource parameter to one or more host names.

For example, you may decide to use NTP pool servers. The NTP Project recommends using pool servers that are close to your servers' time zone as described on their Help page. To view the page in a language other than English, go to www.ntp.org and click the "How do I use pool.ntp.org?" link.

To use U.S.-based pool servers, use the following parameter value:

network.timeSource = 0.us.pool.ntp.org 1.us.pool.ntp.org 2.us.pool.ntp.org

Configuring a Node without NTP

Never run a storage cluster in a production environment without using NTP. However, in demonstration or development environments where there is no internal or external NTP server available, you can choose a minimum of one or a maximum of two nodes as the master clock and synchronize the clocks in the remaining nodes to the master clock in one of those nodes.

Warning

Warning: If you create a Netmail Store storage cluster without an external NTP time source, ensure that the BIOS clocks in all new nodes are set relatively close to the correct GMT time before they join the cluster. All Netmail Store node clocks are set to GMT (not local time) and do not change for daylight saving time.

To implement a cluster without using NTP, set the following parameter in the configuration file of any single node in the cluster:

network.timeSource = system

All other nodes in the cluster will attempt to synchronize their clocks to this node.