EMC Centera/CAS/OBS

What is EMC Centera?

The EMC Centera is the world's first magnetic hard disk-based WORM data storage device, providing Write Once Read Many functionality to applications that require data to be stored on a non-rewriteable, non-erasable storage medium.

By using traditional magnetic hard disks as its storage medium, the EMC Centera offers greater performance over other archival media types such as optical and tape. EMC developed the Centera to address the storage of fixed content data, the fastest growing data type today.

What is Fixed Content Data?

Fixed content data is any digital asset that is created once, never modified, and must be retained for reference throughout its life cycle and retention period as required by regulatory agencies.

The recent explosion in the creation of fixed content has created a demand for a new category of storage devices designed to provide fast, secure on-line access to this data with long-term availability. The EMC Centera represents this new category of data storage devices known as Content Addressed Storage (CAS).

What is Content Addressed Storage?

Content Addressed Storage is a method of data storage that stores and retrieves a data object by its content address within the storage system, rather than by its actual file name at some physical location.

The benefit of a content addressable approach to storage is that an object is stored in such a way that it is authenticated and unalterable. In addition, objects cannot be deleted prior to the expiration of its defined retention period.

 

How Content Addressed Storage Works

When an application delivers a data object to the EMC Centera, the API calculates a 128-bit “claim check” that is uniquely derived from the objects binary representation. The metadata for the object, which includes filename, creation date, etc., is inserted into an XML file called a C-Clip Descriptor File (CDF), which in turn has its content address calculated. The Centera repository then stores the object and a mirror copy.

Once two copies of the object and CDF are stored in the repository, the Content Address is returned to the application. Future access to the data object occurs when the application submits the CDFs Content Address to the Centera repository via the API. The data is then returned back to the application. The Centera file system architecture eliminates directory structures, pathnames, and URL references to filenames and only uses the C-Clip Content Address as a reference.

How EMC Centera Provides WORM Functionality

The C-Clip Content Address of a data object assures the authenticity of that object. If an object is retrieved and altered, the Centera API produces a new CDF with a new content address for the altered object. The original object remains in its original form at its original content address and is still accessible by its original address.

This feature of Centera provides a level of versioning integrity that standard file servers and operating systems cannot provide. Additionally, Centera features an operational mode where an object cannot be deleted prior to the expiration date of a defined retention period. These non-rewriteable and non-erasable properties of the EMC Centera give the Write Once Read Many attributes required for compliance with SEC 17a-4, Sarbanes-Oxley, HIPAA, FDA, and many others.

Centera Hardware Architecture

Comprised of Redundant Arrays of Independent Nodes (RAIN), every node in a Centera contains a CPU, network interface, and 3TB of raw storage, and is interconnected with all other nodes in the cabinet via a private LAN. Each node executes an instance of CentraStar, the Centera operating software, in one of two operational modes to act as either a storage node or an access node.

The storage nodes provide the physical storage of data objects and C-Clip Descriptor Files and the access nodes provide the means for interaction between the application server and the storage nodes. Throughput and storage requirements of the application will determine how many access nodes vs. storage nodes must be configured at the time of installation of the EMC Centera.

Fault Tolerance

The EMC Centera is based around a “no single-point-of-failure” platform and can be serviced in a non-disruptive manner. Every component of the Centera has built-in redundancy. This includes hard drives, power supplies, AC power connectors, cooling fans, and network adapters and associated cable interconnects.

When drives fail, CentraStar, the Centera operating software, will transparently remove them from the cluster. Objects are regenerated from the current mirror to a new mirror to ensure that a fully redundant mirror copy of the content is always available. Data integrity checking runs in the background and continuously recalculates the content addresses of all the objects and compares the calculations to the content addresses originally stored in the C-Clip Descriptor File.
Scalability

A single Centera 19” rack cabinet can hold 8, 16, 24, or 32 nodes to provide 5.4TB – 43.2TB in mirrored mode, or 18.3TB – 73.4TB in parity-protected mode. For scalability beyond 73.4TB, multiple Centera cabinets can be configured as a single cluster, offering hundreds of Terabytes of total storage capacity in a single Centera storage pool.

When new storage units are added to the cluster and powered on, they are automatically “auto-discovered” and join the cluster. No reconfiguration or downtime is necessary to add capacity.

NetApp Telnet : Too many users logged in! Please try again later

You can log out the session blocking your log on attempt by running the following command.  
You can just execute the following command to remotely logout the user and free up the session.

rsh (hostname)  -l root:(password) logout telnet

NetApp - Theory of Aggregate

NetApp's Aggregate is a DataONTAP feature that combines one or more Raid Groups(RG)  into a pool of disk space that can be used to create multiple volumes in different flavors. Newly added disks are assigned to the spare pool and a new aggregate or an existing aggregate that requires more space will then be pulled from the spare pool.

Unlike other Storage arrays where you create LUN from a specific RG, DataONTAP creates volumes within the available space in the aggregate across multiple RG. DataONTAP dynamically stripes the volumes across the aggregates within the Raid Groups. When disks are added to the aggregate they will either go into an existing RG that is not yet full or into a new RG if all the existing groups are full. By default RG are created as RAID-DP(dual parity drives) with stripes of 14+2 for FC/SAS disks and 12+2 for SATA disks. If you want to separate your "Vol0" in a separate aggregate, usually its aggr0 with a minimum of 3 disks for RAID-DP.

Aggregates have snapshots, snap reserve & snap schedule just like any other volumes. Aggregates may either be mirrored or unmirrored. A plex is a physical copy of the WAFL storage within the aggregate. A mirrored aggregate consists of two plexes;(A plex may be online or offline unmirrored aggregates contain a single plex. In order to create a mirrored aggregate, you must have a filer configuration that supports RAID-level mirroring. When mirroring is enabled on the filer, the spare disks are divided into two disk pools. When an aggregate is created, all of the disks in a single plex must come from the same disk pool, and the two plexes of a mirrored aggregate must consist of disks from separate pools, as this maximizes fault isolation.

An aggregate may be online, restricted, or offline. When an aggregate is offline, no read or write access is allowed. When an aggregate is restricted, certain operations are allowed (such as aggregate copy, parity recomputation or RAID reconstruction) but data access is not allowed. 

Creating an aggregate

Aggregate has a finite size of 16TB raw disk with 32 bit DataONTAP. By default, the filer fills up one RAID group with disks before starting another RAID group. Suppose an aggregate currently has one RAID group of 12 disks and its RAID group size is 14. If you add 5 disks to this aggregate, it will have one RAID group with 14 disks and another RAID group with 3 disks. The filer does not evenly distribute disks among RAID groups.

To create an aggregate called aggr1 with 4 disks 0a.10 0a.11 0a.12 0a.13 from a loop on fiber channel port 0a use the following command:

netapp> aggr create -d  0a.10 0a.11 0a.12 0a.13

Expanding an aggregate

To expand an aggregate called aggr1 with 4 new disks use the following command:

netapp> aggr add aggr1 -d  0a.11 0a.12 0a.13 0a.14

Destroying an aggregate

Aggregates can be destroyed but there are restrictions if volumes are bounded to the aggregate. Trying to destroy an aggr with volumes will throw an error but this can be overridden with the -f flag. It is recommended to go over each and every volume and destroying it before using the -f option to avoid any potential data loss. Aggregates can be destroyed much like any volumes but it needs to be taken offline.

To destroy an aggregate aggr1 use the following command:

netapp> aggr offline aggr1

netapp> aggr destroy aggr1

DataONTAP Logical View - LUNs, Deduplication, and Thin Provisioning

 This is one of the coolest storage model for NetApp that explains the placing of disks, Raid Groups, Aggregates, Flex Volumes, NAS and SAN. Thanks to Dr. Dedupe for sharing this piece of information. 

NetApp Fractional Reserve

When we snapshot the filesystem, we take a copy of the file allocation tables and this locks the data blocks in place. Any new or changed data is written to a new location, and the changed blocks are preserved on disk.

This means that any new/changed blocks (D1 ) in the active file-system are written to a different location - this overwrite space is called as fractional reserve.

Fractional reserve ensures that there is space available for overwrites and the default fractional reserve space is 100% but this can be changed and generally set to "0" value.

So the reason a LUN will be switched offline if the fractional reservation area is set to 0, is that the filer needs to protect the existing data that is locked between the active file system and the most recent snapshot, plus any additional changes that happen to the active file system. If the volume / LUN / frac-res and snap reserve are full, then this space is not available and the filer needs to take action to prevent these writes from failing. The filers guarantee no data loss, but with no space free and no-where to write the new data, it has to offline the lun to prevent the writes from failing.

So fractional reservation is in constant use by the filer as an over-write area for the LUN. Without it, you need to make sure that sufficient space is free to allow for the maximum rate of change you would expect. Defaults are good, but trimming down on these you need to monitor the rate of change and make sure the worst case scenario is within a buffer of free space that you allow.

Pocket Survival Guide NetApp

Setting Up a Filer:
1. Check version and licenses
(telnet) license to list
(telnet) license add (key) to add a license
(FilerView) Filer -> Manage Licenses
2. Setup Network Interfaces (set up single-mode and multi-mode vifs or ifgrps)
NOTE: You can't modiy a vif once it is created. I usually have to delete the vif and start over to modify it. This also means network service will go down on that port group!
(telent) ifconfig -a to list the interfaces
(telnet) ifconfig command to manage the interfaces
(telent) To manage vifs in ONTAP 7: vif command, in ONTAP 8: ifgrp
(telnet) You can also run setup, will probably require a reboot!
(FilerView) Network -> Manage Interfaces -> i.e. vif1a or vif1b
3. Enable SSH on the filer
(telnet) secureadmin setup ssh
(FilerView) Secure Admin ->SSH -> Configure -> Generate Keys -> OK -> Apply
4. Set Snap reserve on Aggr0 if not MetroCluster
The default for this is 5%. I have seen some set this to 0% if not using MetroCluster. I set it to 3% because I have seen issues if set to 0% and NetApp support likes at least a little reserve at this level for disaster recovery
(telnet) snap reserve -A aggr0 3
(FilerView) I don't think this is possible from FilerView
5. Resize Vol0
(telnet) vol size vol0 15g for 2000 series (20gb on 3000 series)
(telnet) vol size vol0 to check
(FilerView) Volumes -> Manage -> Click on vol0
Click Resize Storage
Click Next -> Click Next -> Enter New Volume Size -> Next -> Commit -> Close
6. Iscsi & FCP status
(telnet) iscsi status to check
(telnet) iscsi start to enable
(telnet) same commands for fcp
(FilerView) LUNs -> Enable/Disable ->Check Enable Box
7. NTP Setup
(telnet) to list all time options: options timed
(telnet) options timed.servers 0.us.pool.ntp.org,1.us.pool.ntp.org,2.us.pool.ntp.org (or enter your time servers for your site)
(telnet) options timed.proto ntp (if not already set to ntp)
(telent) options timed.log on (if you would like the updates to go to console and log)
(telnet) options timed.enable on
(FilerView) Filer -> Set Date/Time -> Modify Date/Time
Choose Time Zone -> Click Next
Choose Automatic -> Click Next
Change Protocol to ntp -> Click Next
Enter ntp servers 0.us.pool.ntp.org (1&2) - or any other ntp server you have
Click Commit -> Click Close
8. Enable and Test Autosupport
(telnet) to list all options: options autosupport
(telnet) options autosupport.from (userid) - sets the userid autosupport is sent from
(telnet) options autosupport.mailhost (host ip or name)
(telnet) options autosupport.to (user1,user1)
(telnet) options autosupport.enable on
(telnet) options.autosupport.doit test - generates a test autosupport
(FilerView) Filer -> Configure Autosupport
Change Autosupport enabled to Yes
Enter mailhosts
Enter To: fields -> Click Apply
Click Test Autosupport on the left -> Click Send Mail
9. Check Cluster Failover
(telnet) cf status – Check Cluster Failover Status
(telnet) cf partner - Lists the partner head
(telent) cf monitor – more details on failover
(telnet) cf takeover - This will reboot the host you are taking over!
(telnet) cf giveback - when partner is ready to receive back
(FilerView) Cluster - Click the buttons! :)
10. Disk Commands (Assign Drives to Controller)
Typically I would assign the odd disks to the first controller and the evens to the second controller. I have also assigned the majority of drives to one controller and a minimum (3-4 drives) to the other controller to maximize capacity
(telnet) disk show –n - show disks NOT owned
(telnet) disk show –o (controller name) - show disks owned by a controller
(telnet) disk show –v - show all disks owned and not owned
(telnet) disk assign (drive#)
(telnet) disk remove_ownership (drive#) (This is a priv set advanced command!)
(telent) disk zero spares
(FilerView) - I don't think this is possible from FilerView
11. Aggregate Commands (Assign Drives to Aggregate)

In RAID-DP environments, if there are 2 hot spares, I often add one back in. You already have 2 drive failure with RAID-DP and this way you have one more drive capacity
(telnet) aggr status -v - shows status with volume information
(telnet) aggr status –r - shows which drives are in the aggr already including raid groups
(telnet) aggr options (aggr) - shows all options for the aggr (raid groups, raid type, etc.)
(telnet) aggr show_space -m OR -g
(telnet) aggr add (aggr) –d (drive#)
(FilerView) Aggregates -> Click on the aggregate
Click Show Raid to see existing disk configuration
Click Add Disks to add new disks and follow the wizard
12. Volume Commands (Create Volumes From Aggregates)
(telnet) vol status - shows all volumes and options
(telnet) vol create (volumename) -s none (aggr) (size) - create volume with no guarantee
(telnet) vol create (volumename) -s volume (aggr) (size) - create volume with volume guarantee
(telnet) vol size (volume) - check the volume size
(telnet) vol size (volume) (size) - set volume to new size
(telnet) vol autosize (volume) on - turn on volume autogrow, the volume will grow by default in 5% increments until 20% growth achieved by default
(telnet) vol autosize volume -m (new max size) -i (new increment amount) - notice that even though the defaults are 20% max and 5% increments, if you change the value the new values have to be in space (MB or GB)!
(telnet) vol autosize (volume) - reports the current setting
(telnet) vol options (volume) - list the volume options
(telnet) vol options (volume) guarantee volume OR none - sets the volume guarantee to volume or none
(telnet) vol options (volume) fractional_reserve (value) - sets fractional reserve to (value) but only works if volume guarantee is volume.
(telnet) vol options (volume) try_first snap_delete OR volume_grow - sets the value to which method a volume will use when it runs out of space. It will either grow the volume or delete snap shots. The defaults is to grow the volume.
(FilerView) Too much here to list. Just click on volumes and poke around. The following aren't possible in FilerView that I can tell: volume autosize, volume try_first option, changing fractional reserve
13. Check and Modify Snapshot Settings as Needed per Volume

NOTE: Disable the snapshot schedule on the volume if SnapManager is protecting the volume
(telnet) vol options (volume) nosnap on - Turn off the SnapShot Schedule but leave the schedule in place. I do this to disable snapshots in favor of the following command
(telnet) snap sched (volume) 0 0 0 - Disable SnapShots by modifying the schedule. I like the first option better.
(telnet) snap reserve (volume) (#) - Sets the snap reserve to #% of the volume
(telnet) snap autodelete (volume) - shows the snapshot autodelete settings
(telnet) snap autodelete (volume) on - turns on snapshot autodelete (Check the settings! Too many default settings to list here!)
(FilerView) Too much here to list. Just click on Volumes -> Snapshots and poke around. The snap autodelete option is not possible in FilerView that I can tell.
14. Set DeDupe Settings per Volume
DELETE EXISTING SNAPSHOTS FIRST ON THE TARGET VOLUME!
Note: For the sis command you need the full volume path, /vol/vol0 for instance
(telnet) sis on (full volume path) - enables dedupe on that volume
(telnet) sis start (full volume path) - runs dedupe on that volume now
(telnet) df –sh - To check space saving on DeDupe
(telnet) sis config and sis status to check settings
(telnet) sis config -s sun,mon-fri,sat@0,3-9,12 (full volume path) - The -s option allows you to change the schedule. You can list the days separated by commas or a range by dash, same with hours.
(FilerView) This is not possible in FilerView
15. Cifs setup (set up ntp first!)
(telnet) cifs setup
OR (Filer View) CIFS -> Configure -> Setup Wizard
16. To Set-up a CIFS Share (Add Volume First!)
(Filer View) CIFS -> Shares -> Add
(Filer View) CIFS -> Shares -> Manage -> Change Access
17. NFS Setup (Add Volume First!)
Need to Add NFS Export Permissions
18. LUN Setup (Add Volume First!)
Need more information here
(telnet) lun setup - wizard to create the lun and map the igroup all in one
(telnet) lun set reservation (full path to lun) enable OR disable - disable space reservation to thin provision the LUN
(Filer View) LUNS -> Add (Name Lun with aggr.LUN file convention)
Add an igroup and assign initiators
LUNS -> Manage -> No Maps
Other Common Information Commands That I Still Need to Add and Document:
Sysconfig

snap reclaimable
snap delta
aggr show_space (aggr) -g
shelfchk
Wrfile /etc/hosts
Rdfile /etc/hosts
Rdfile /etc/rc
rdfile /etc/messages

Priv set advanced
Led_on (drive#) (get nmber from disk show) (need priv set advanced first)
(priv set) disk shm_stats
(need to add deswizzle status command)
sysstat
statit (disk utilization)
stats (performance data)
reallocate measure or
reallocate start
to see raid groups: sysconfig -r OR aggr status -R
fcstat device_map - command to show shelf layout
sysconfig -d
interface alias process

storage show
vif favor

EMC Techbook - FCoE

EMC Clariion FAST

CLARiiON Fully Automated Storage Tiering (FAST) moves data to higher performance or cost effective storage to meet demanding application service levels and to increase storage efficiency.

CLARiiON FAST enables applications to always remain optimized by eliminating trade-offs between capacity and performance. As a result, you are able to lower costs and deliver higher services levels at the same time.

EMC Announces FAST Upgrades, Including Flash-Based Cache For Clariion, Celerra

CLARiiON, Celerra, get FAST, new Unisphere management tools

EMC debuts Unisphere, FAST for Clariion, Celerra

Clariion @ EMC World 2010

News Summary:

• New advanced storage efficiency technologies for EMC CLARiiON® and EMC Celerra® storage systems —next generation of EMC® FAST (fully automated storage tiering), innovative FAST Cache capabilities and block data compression—enable customers to lower costs and improve performance.
• New EMC Unisphere software delivers a simple approach to midrange storage management with task-based controls, customizable views, flexible reporting, and advanced self-service capabilities.
• CLARiiON and Celerra storage systems support VMware vStorage APIs for Array Integration (VAAI) and new vCenter integration enables VMware administrators to easily provision and protect virtual machine storage.
• These new EMC midrange technologies are key enablers of private clouds, helping customers manage their explosion of digital data with new levels of simplicity and efficiency.

EMC VPLEX

EMC VPLEX makes Virtual Storage a reality with its ability to federate information across multiple data centers. Virtual Storage enables new approaches for delivering IT as a flexible, efficient, and reliable service. The combination of Virtual Storage and virtual servers is a critical enabler for the journey to the private cloud.

Leverage distribution federation

With VPLEX's unique distributed federation, your data can be accessed and shared among locations over synchronous distances. The VPLEX architecture combines scale-out clustering with distributed cache coherence intelligence to enable data mobility between EMC and non-EMC platforms within, across, and between data centers.

Virtual Mobility

The combination of EMC VPLEX and VMware VMotion enables you to effectively distribute applications and their data across multiple hosts over synchronous distances. With Virtual Storage and virtual servers working together over distance, your infrastructure can provide load balancing, real-time remote data access, and improved application protection.