Netmail Store ships with erasure coding disabled by default. If licensed in your cluster, you can select the encoding level for erasure-coded objects for each individual object or all objects in the cluster.
Note: To obtain a license file with erasure coding enabled, contact your sales representative.
Erasure Coding by Cluster
To erasure code all cluster objects, set the following options in the cluster or node configuration file to enable the cluster to erasure code objects based on their size (in bytes):
ec.minStreamSize. Sets the minimum size, in bytes, for an object to be automatically erasurecoded.
ec.encoding. Sets the number of data and parity segments to be used when erasure coding objects (for example,
Erasure Coding by Object
To erasure code individual objects, use the
?encoding=k:p query arguments with an SCSP
UPDATE method or specify an encoding tuple in the lifepoints for the object (reps=5:2). This process erasure codes the object at the specified encoding level.
Adding the optional
erasureCoded=yes argument will force the object to be erasure-coded, regardless of the cluster setting or object size.
EC Disk Footprint
The amount of disk space (or footprint) used for erasure-coded objects depends on the ratio of data to parity segments in the specified encoding. Use the following formula to roughly calculate the disk space that you can expect to see used by an EC object with one set of erasure segments:
k+p total segments / k data segments * ObjectSize = total object EC disk footprint
A 1 GB object with 5:2 encoding has this footprint: (5 + 2) / 5 * 1 GB = 1.4 GB (vs. 3 GB for replication)
A 3 GB object with 5:2 encoding has this footprint: (5 + 2) / 5 * 3 GB = 4.2 GB (vs. 9 GB for replication)
Note: Some system metadata is written with each EC segment, adding ~16 bytes per segment.
Methods Affected by Erasure Coding
These are the SCSP methods that are affected by erasure-coded objects:
|A new object written to the storage cluster is erasure-coded if it meets the EC criteria.|
APPENDs with version 6.0 are optimized to write a new set of EC segments with the appended data and update the manifest for a previously erasure-coded object, instead of rewriting the whole object to include the appended data as with replicated objects. Data appended to an object that was not previously erasure-coded can cause the object to become erasure-coded if the additional appended data pushes the object size over the configured
Instead of rewriting the entire object to update the metadata,
|Returns the |
The cluster automatically retrieves all segments so a
|Netmail Store deletes both the manifests and the segments for the object.|