What is it?
Netmail Store is a branded storage solution from Caringo Inc (originally called 'CAStor' and since renamed to 'Swarm'). Much like a NAS it is ultimately an array of disks connected to the network for the purposes of storing data.
Unlike a NAS, it does not:
- Require costly or proprietary hardware
- Require a NAS head to bottle-neck operations
- Present any sort of file-system (unbrowsable)
- Provide any additional services such as authentication, encryption, etc…
It does, however:
- Provide multiple points of contact for clients
- Use a universal protocol for access (HTTP)
- Provide resiliency at multiple levels
- Maximize functional storage of the platters
More information: http://www.caringo.com/resources/faqs.html
How do I interface with it?
Netmail Store speaks a subset of the standard HTTP 1.1 protocol, called SCSP. Any application which is HTTP compliant (such as any modern browser) can interface with Netmail Store.
How does it work?
When data is written to a node, a copy is immediately made somewhere else. The number of replicas you wish to keep is configurable (default:2). The replicas are distributed as diversely as possible across the nodes (or subclusters) in order to minimize the possibility of data loss during a disaster.
Any node can be used to read or write from, regardless of where the data is finally stored. Leveraging the redirection capabilities of HTTP, the nodes will automatically steer the requests to whichever is node is hosting the data, most idle, or has the most capacity.
The nodes synchronize amongst themselves using IP multicast. Consequently it is highly recommend that:
- All nodes of a cluster be placed in the same subnet
- That subnet be dedicated to Netmail Store nodes
- IGMP snooping be disabled on the switch/router connecting the nodes
What’s a CSN?
In order to perform its tasks, Netmail Store nodes require some services be available in the environment. For example,
The Cluster Services Node (CSN) serves to centralize these facilities into a single box, and creates a single point of administration. It can also automate certain tasks, such as upgrades, by hosting boot images and PXE-booting nodes in a rolling fashion. Lastly, it can serve to link two separate CAStor clusters across a WAN/LAN for data replication scenarios (for DR purposes).
How do I upgrade it?
If you are booting off USB keys, then the procedure can be found at: Upgrading Netmail Store with USB keys
If you have a CSN, then the procedure can be found at: Upgrading Netmail Store software using a CSN
How does it integrate with Netmail Archive?
The Netmail web console has a place to specify the CSN connection & credential information. On the tree in the left-hand side, all the way at the bottom, select Storage -> Location tab. This will bring the CSN console into the Netmail console, thus enabling administration from a single portial.
To use the Store cluster as a target to read/write data, you must specify it in the Archiving->Storage tab. Create a Storage Device, select the Netmail Store type, and provide:
- A name you wish to use to identify this cluster
- The IP address of any node or CSN
- The port to connect to (default is 80, for HTTP traffic)
- The number of nodes in the cluster
The address of the node used can also be a DNS name, and round-robined among the different IPs of the cluster, however round-robin DNS can make troubleshooting more difficult.
How do I monitor it?
Netmail Store can be monitored 4 ways:
- via the Swarm web console on port 90
- via the web console on the CSN on port 8090
- via the integration in the Netmail Diagnostics
- via SNMP MIBs
How can I see what data is stored on it?
Object-based storage is not like a traditional file system in that it cannot be browsed or navigated. Each file stored therein returns a universally unique ID (UUID) when it was first written to the cluster, and that UUID is required to read the file back. Consequently a client will need the exact UUID of the data they want to access, otherwise the data/contents remain obscured. In the context of Netmail Archive, the audit files are the location where these UUID’s are kept, hence why they cannot also be kept within Netmail Store.