A tenant corresponds to a corporation or an organizational unit and consists of the following Netmail Store components, which the Admin Console creates for you:
- A domain, which is a container for buckets and objects.
- An administrative bucket, named _administrators, and its user list. Users in the administrative bucket user list are referred to as domain managers, and they are responsible for maintaining the user lists for the administrative bucket and for the domain.
- A domain protection setting. The domain protection setting determines what realm has post privileges in the domain.
The terms user list, realm, and domain manager are defined in Terminology Related to Tenant Security.
On this page:
Terminology Related to Tenant Security
The following are basic terms related to Netmail Store security:
- Authorization list: List of SCSP operations that users in a security realm are allowed to execute. The authorization list is specified by the Castor-Authorization header. An authorization list can be associated with a domain, bucket, or named object.
- User list (also referred to as a security realm or a realm): List of user names and passwords that are hashed using the algorithm defined for Digest Access Authentication. A user list can be associated with a domain or bucket. Domain managers are responsible for managing realms and authorization lists for the domain.
Netmail Store uses the following roles to determine who can perform different types of actions in the cluster:
- Cluster administrator: The cluster administrator (you) is responsible for creating tenants and domain managers and for the overall maintenance, management, and monitoring of the cluster. The list of cluster administrators is maintained using the administrators parameter in the cluster or node configuration file.
- Domain manager: Maintains the domain manager user list, and determines which realms can create buckets in a domain. Users who are not in any realm defined by the domain manager can perform only actions that require no authentication.
- Application developer: Responsible for creating content in the domain (that is, buckets and objects for which they have privileges).
About the Default Cluster Domain
The default cluster domain is an optional domain whose name exactly matches the cluster name. Every unnamed object and every named object that has no domain explicitly defined for it belong to the default cluster domain. Creating a default cluster domain enables your cluster to use the following features:
- POST authentication for unnamed objects.
- Disambiguation of cluster names in a roll-up cluster. The domain name is stored as part of the object's system metadata. Provided every domain name you manage is unique, there will never be a collision between named objects from different clusters if those clusters are combined (such as in a disaster recovery cluster).
- Convenience when working with objects. Application developers do not have to explicitly set a domain name as the Host in the request to perform operations on an object (such as create, update, or delete).
Note: Every unnamed object belongs to the default cluster domain and, if you use security, POST permissions are supported for unnamed objects on the default cluster domain only. There is no reason to create a default cluster domain if your client applications use unnamed objects only without POST authentication.
Security Privileges for Administrative Operations
The following table shows the privileges required to perform administrative operations in a domain, bucket, or in the objects contained by them.
Manage realms (that is, user lists)
put, post, or change in the domain or bucket. The user list for the domain is administered by the domain manager. User lists for buckets are administered by authorized users in the domain.
Create buckets in a domain
post in the domain. The Admin Console enables you to set post permissions as “protection settings” for the domain.
Each protection setting is specified as a Castor-Authorization header in the form:
Castor-Authorization: domain-name/ _administrators, post=domain-name
where the domain in post=domain-name is blank if you choose the “all users” protection setting.
Create named objects in a bucket
post in the bucket.
Note: Security privileges are not inherited from container objects to the objects contained by them. In other words, a realm that is authorized to create a bucket is not automatically authorized to create objects in the bucket.
Rules and Recommendations for Managing Tenants
The following guidelines must be observed when creating domain for tenants:
- You must create at least one domain in your cluster to use named objects.
- You must properly set the clusterSettingsUUID parameter in your node or cluster configuration file. For more information about the clusterSettingsUUID parameter settings, see Cluster Actions.
- All domain names must be unique among all tenants and all clusters you manage.
- It is strongly recommended that all domain names be IANA compliant (for example, cluster.example.com).
- If you already have a cluster name that is not IANA-compliant, create an IANA-compliant domain name and create of all your named objects in buckets in that domain.
- It is strongly recommended that you set up a default cluster domain (that is, a domain name that exactly matches the name of the cluster). Every object that has no domain explicitly defined for it belongs to the default cluster domain. However, if your client applications maintain use objects only without POST authentication, you should not create a default cluster domain.
Note: If a domain or bucket is deleted without first deleting the objects it contains, the objectsare not deleted; however, the objects cannot be retrieved because their container is missing. For example, if a bucket that contains objects is deleted, the objects cannot be retrieved. To work around this issue, see Restoring Domains and Buckets.
Domain Naming Rules
When you create a domain name, observe the following rules:
- It must begin with a number or letter.
- It is recommended to be an IANA domain name (see also RFC 1034).
- Allowed characters: alphanumeric, including underscore (_), period (.), and hyphen (-). However, a valid name cannot end with a hyphen character, contain two successive periods, or have a hyphen and period adjacent to each other. Examples: cluster.example.com, my-cluster.example.com, my_cluster.example.com. It is strongly recommended to not use a non-IANA domain name like domain or cluster_example_com.
- It cannot contain a comma (,), colon (:), space, or slash (/) character.
- It must be a valid UTF-8 byte sequence.
- It cannot be an IPv4 or IPv6 IP address.
- It is case insensitive.
Note: To use named objects in your cluster, you must create at least one domain. Unnamed objects do not require a domain because you can access them using a UUID.
Adding, Editing, or Deleting Tenants
To add, edit, or delete a tenant:
1. Connect to the Admin Console.
2. In the top navigation bar, click Settings.
3. In the Cluster Tenants section of the Cluster Settings page, click Add Tenant to add a tenant domain or Edit to edit an existing tenant domain. To delete a tenant domain, select the checkbox next to its name, and click Delete and then Submit.
Note: If you delete a domain that contains buckets, the buckets and any objects they contain are not deleted; however, they become inaccessible. To work around this issue, see Restoring Domains and Buckets.
4. Enter or edit the following information:
Domain Name field
Enter a fully qualified IANA-compliant name to identify this domain. For example, cluster.example.com. The domain name must be unique among all clusters you manage. If you have not already configured a domain name that matches your cluster's name, the cluster name displays in this field. Creating a domain with the same name as the cluster sets up a default cluster domain. Renaming an existing domain is supported only using the Admin Console.
All protection settings enable domain managers to maintain their own user list and the domain user list. The difference between protection settings is which realms can POST to the domain (that is, create buckets).
Click one of the following:
Each protection settings is specified as a Castor-Authorization header in the form:
Castor-Authorization: domain-name/_administrators, post=domain-name
where domain-name is blank if you choose the "all users" protection setting.
Domain managers have the ability to manage their own user list and user lists in the tenant's domain. To add a new administrator, click Add Domain Manager. To edit an existing manager, click Edit next to the administrator's manager. Note that domain manager names can consist of ASCII characters only and cannot include a colon character (:).
Note: If Custom Policy displays for Protection Setting, a tenant administrator has altered the Castor-Authorization header for the _administrators bucket. If you are trying to troubleshoot an issue with users being able to access objects in a domain, try setting the protection setting back to its default. Examples of modifying the Castor-Authorization header are shown in Using Override to Resolve Authorization Specification Issues.
5. If you are adding or editing a domain manager, enter or edit the following information:
- User ID: Enter a name to identify the domain manager. Domain manager names can consist of ASCII characters only and cannot include a colon character (:).
- Password: Enter the domain manager's password.
- Re-Enter Password: Verify the password by entering it again.
6. Click Submit.
7. If prompted, enter an administrative user name and password.
Note: The following error indicates that a previous renaming of this domain has not yet expired from the content cache:
ERROR! realm name 'domain-name/_administrators' already exists.
In other words, suppose a cluster administrator renamed cluster.example.com to domain.example.com. If you attempt to create cluster.example.com before the name has expired from the content cache, the error displays. In that case, wait a few minutes and try again.
Other Cluster Administrator Tasks
Using Administrative Override
Administrative override means bypassing the Castor-Authorization header as well as the Allow header, enabling you to perform any SCSP operation on an object. To use administrative override, you perform the SCSP operation using the admin query argument and authenticating with credentials from the CAStor administrator realm. For example, you can:
- Delete an object that has delete restrictions because of security settings, Allow header, or both.
- Get the user list for an object or upload a new user list to an object (for example, when a user forgets his or her password).
- Restore access to an object that has an incorrect Castor-Authorization header. (For example, access to an object is limited to a realm that has no users.)
- Access a deleted object.
Important: Do not change the Castor-Authorization header or user list on any object if the current header uses owner@ or @owner syntax object because when you do, you (the cluster administrator) become the object's owner and users in other realms cannot change the header later. Alternatively, ask the object's current owner to change the Castor-Authorization header or user list.
Using Override to Delete an Object
For example, to delete an object named hello.html from cluster.example.com/mybucket as the default administrative user, you can use the following command:
curl -X DELETE --location-trusted "http://172.16.0.35/mybucket/hello.html?
admin" --anyauth -u "admin:ourpwdofchoicehere" --post301 -D admindelete.log
If the object is not in the default cluster domain, you must pass in the domain name as the Host in the request.
Using Override to GET or APPEND User Lists
If a user forgets if they have access to an object, you can use administrative override to GET the user list to verify whether or not their user name is in the list. If a user forgets their password, you can use administrative override to update the user list with the user's name and with a new password. Although you can update a user list using PUT, it is recommended that you use APPEND, because PUT replaces the user list, which might disable other users' access to the same object.
To GET a user list:
curl --anyauth -u "your-username:your-password" --location-trusted
"http://node-ip[/bucket-name]?admin[&domain=name]" [-D log-file-name]
You must specify domain=name to GET the user list for a domain. To GET a user list for a bucket, pass in the domain name as the Host in the request.
To APPEND a user list:
1. Create the user list. A user list (also referred to as a security realm or realm) is a collection of user credentials, each of which includes an MD5 hash using the HTTP Digest authentication algorithm. You compute user list or realm from the string username:realm:password. There are a variety of programming functions and utilities available, including md5sum; and htdigest, which is provided as part of the Apache distribution.
- On Windows, avoid creating a password file with a reserved extension, such as .com.
- If either the password file name or your realm name includes spaces, enclose the name in double quotes.
Important: The realm name must exactly match the name of the domain or bucket.
To create a user list for the cluster.example.com domain, enter
htdigest cluster_example_com cluster.example.com username.here
To create a user list for mybucket in the same domain, enter
htdigest cluster_example_com_mybucket cluster.example.com/mybucket username.here
2. APPEND the user list:
curl -i -X APPEND --data-binary @user-list --anyauth -u "yourusername:your-password" --location-trusted "http://node-ip[/bucketname]?admin[&domain=name]" --post301 -D [-D log-file-name]
You must specify domain=name to APPEND the user list to a domain. To APPEND a user list to a bucket, pass in the domain name as the Host in the request.
3. Confirm the user list using GET as shown in the preceding bullet.
Note: In the event the same user is listed more than once in a user list, Netmail Store uses the last (that is, the most recent) entry.
Using Override to Resolve Authorization Specification Issues
This section discusses how to resolve issues with authorization specifications that render objects inaccessible. You can perform these tasks to reset the authorization specification for any object, even an object for which an authorized user name and password are not known. To resolve this issue, you must PUT to the object the user list and the authorization specification using the admin query argument, authenticating with your cluster administrator credentials.
Important: Do not use this procedure if the current Castor-Authorization header uses owner@ or @owner syntax because the CAStor administrator realm becomes the owner of the object and, as a result, no other realm can change the authorization specification later.
1. Create the user list.
A user list (also referred to as a security realm or realm) is a collection of user credentials, each of which includes an MD5 hash using the HTTP Digest authentication algorithm. You compute user list or realm from the string username:realm:password.
Important: The realm name must exactly match the name of the domain or bucket.
For example, to create a user list for a domain:
htdigest cluster_example_com cluster.example.com sample.username
To create a user list for mybucket in the same domain:
htdigest cluster_example_com_mybucket cluster.example.com/mybucket sample.username
2. HEAD the current value of the Castor-Authorization header for the object.
curl --anyauth -u "your-username:your-password" --location-trusted "http://node-ip[/bucket-name]?admin[&domain=name]" [-D log-file-name]
You must specify domain=name in a HEAD for a domain. If the HEAD is for a bucket, the domain name is required as the Host in the request if the domain is not the default cluster domain.
Important: If the Castor-Authorization header includes @owner or owner@, stop. GET the user list for the object and confirm whether or not any realm has post or change privileges to the object. (For example, if the Castor-Authorization header includes change=@owner, any user in the object owner's realm can modify the object.) Ask one of those users to modify the Castor-Authorization header. If no user can modify a Castor-Authorization header that includes @owner or owner@, you can either take ownership of the object permanently by continuing with the next step or you can remove the Castor-Authorization header and allow another user to modify it. To remove the Castor-Authorization header header, use PUT or POST with the admin query argument and your credentials to upload a user realm to the object but do not specify a Castor-Authorization header header in the request.
3. PUT the updated Castor-Authorization header and user list to the object.
curl -X PUT -i --post301 -H "Castor-Authorization: authorizationspecfication" -H "Castor-Stream-Type: admin" --location-trusted "http://node-ip[/bucket-name[?domain=domain-name]&admin" --anyauth -u "your-username:your-password" --data-binary @user-list [-D log-file]
You must specify domain=name in a PUT for a domain. If the PUT is for a bucket, you must pass in the domain name as the Host in the request if the domain is not the default cluster domain.
For example, suppose a bucket named testbucket in the domain cluster.example.com has become inaccessible due to either a bad authorization specification or user list. The example that follows shows how you can restore access by changing the Castor-Authorization header and uploading a new user list. The example makes the following assumptions:
- cluster.example.com is the default cluster domain so it does not need to be passed in as the Host in the request.
- The object's Castor-Authorization header does not contain either owner@ or @owner.
- The cluster administrator uses the default user name (admin) and password (ourpwdofchoicehere).
- Commands are sent to a node whose IP address is 172.16.0.35.
4. Create the user list.
htdigest -c cluster_example_com_testbucket cluster.example.com/testbucket john.jones
5. PUT the new Castor-Authorization header and user list.
curl -i -X PUT --post301 -H "Castor-Authorization: post=cluster.example.com/testbucket, change=cluster.example.com/testbucket, view=cluster.example.com" -H "Castor-Stream-Type: admin" --location-trusted "http://172.16.0.35/testbucket?admin" --data-binary@cluster_example_com_testbucket -u "admin:ourpwdofchoicehere" -D fixbucket.log
6. Ask a user in the user list to confirm they have access to the object by uploading a new user list to the object or by changing the Castor-Authorization header.
Working With Inaccessible Objects
In the event a domain, bucket, or named object is not accessible by usual means, you can access it directly using the cid= query argument. Examples of when this might be useful:
- The domain or bucket has been deleted.
- The domain has been duplicated in a disaster recovery cluster; in other words, there are two domains with the same name in the same cluster.
Note: This procedure enables you to access the object but not to recover it. To recover accidentally deleted domains and buckets, see Restoring Domains and Buckets.
To use this procedure, you must know the value of the object's Castor-System-CID header, which identifies the object's parent. For example, if an object named photo1.jpg is not accessible, you must know the value of its Castor-System-CID header. If you did not already record this information or store it, you can find it in any of the following ways:
- Debug-level system logs record the value of Castor-System-CID every time the object is accessed.
- You can use the Content Router Enumerator component to find it. The Enumerator iterates through all objects in a cluster and returns information about those objects. First, add a Content Router filter rule to search for streams where the value of the Castor-System-Name header is the name of the inaccessible object. Then, using the SDK, instantiate a metadata enumerator subscribed to the rule channel you created in the preceding bullet to obtain the object's metadata. In the metadata returned for the object, look for the value of the Castor-System-CID header. For more information about using the Enumerator, see . . For information on creating Content Router rules, see
After you find the value of the object's Castor-System-CID header, access the object using the cid=CID-header-value query argument. To access a named object using a web browser, enter the following URL in the browser's address or location field:
For example, to access an object named file.html with a CID of 55aba17ad53c61782d7dd0afa8dd2f7d, enter