Netmail Store has an Overlay Index, which provides increased scalability in a storage cluster. When enabled, the Overlay Index tracks object locations in a shared cluster-wide index, minimizing multicast traffic in the cluster network.
The Overlay Index creates a dynamic master index in RAM from the local node indexes of all existing objects in the cluster. When an SCSP request for an existing object is sent to a Netmail Store storage cluster, the Overlay Index locates the nodes containing the targeted object and directs the request to an appropriate node without multicasting to locate it. This process minimizes the multicast traffic and associated processing in your storage cluster.
The knowledge of existing nodes in the cluster is refreshed regularly and the Overlay Index evaluates whether changes to its model of the cluster are required every 5 minutes, ensuring the Overlay Index will be optimally spread across all available nodes in the cluster as new nodes are added or existing nodes are removed from the cluster. Adjustments to the Overlay Index's distribution due to changes in the nodes participating in the Overlay Index, particularly immediately after a cluster reboot, happen quickly but are not instantaneous. Distribution adjustments should not affect client activity, however.
The Overlay Index is enabled by default for clusters with 32 or more nodes. The Overlay Index can be disabled (or later re-enabled) without a cluster reboot with the read-write
overlayIndexOn SNMP OID. Similarly, the minimum number of nodes required before the Overlay Index is used can be modified with the read-write
overlayMinNodes SNMP OID.
Note: Lowering the required number of nodes below the default of 32 may hurt throughput performance for small-object reads.
The Overlay Index also enables an optional feature called Optimize 404, which is enabled by default. With the Overlay Index enabled and fully populated and Optimize 404 enabled, Netmail Store provides faster
404: Not Found responses for missing objects. Optimize 404 can be disabled without a cluster reboot with the read-write
overlayOptimize404 SNMP OID.
Before implementing the Overlay Index, review its features and considerations:
Enables the Health Processor to more quickly discover and trim over-replication generated by failed volume recovery (FVR) and erasure coding recovery (ECR), which maximizes the disk space on your cluster nodes.
Requires additional RAM index slots to hold the additional index information. If your current node indexes are full, you must add additional RAM and reboot the nodes in order to take advantage of the Overlay Index.
Reduces the multicast traffic in your storage cluster network.
Due to the extra processing required to maintain the index, does not provide scale benefit in smaller clusters and may negatively affect read throughput performance if the minimum number of nodes is lowered below the default of 32 nodes.
The Overlay Index requires additional index slots in cluster node RAM to store the additional index information. If there are not enough nodes or RAM resources to hold the additional index information, the Overlay Index will not be populated and the cluster will continue to use multicast to locate objects. See thefor RAM requirements.
To determine the current state of the Overlay Index at a given time, check the value of the
indexOverlayStatus SNMP OID. The status of the Overlay Index will be one of the following:
- Disabled: When the overlay is turned off.
- Uninitialized: When the overlay is first activated.
- Operational: When population of the Overlay Index is in process.
- Authoritative: When the node servicing the SNMP query has determined that the Overlay Index is fully populated for the entire cluster. The Optimize 404 feature will only work when the Overlay Index is authoritative.
During normal operation, the state of the Overlay Index will toggle between "authoritative" and "operational" as the structure of the cluster changes with new nodes being added or existing nodes being removed. The status may also indicate a variety of reasons why the Overlay Index is not operational, such as insufficient nodes or inadequate Overlay Index space.