Are you running out of room on your file servers? Just about all of us are at one point in time or another. After all, you can never be too rich, too thin, or have too many megabytes of disk space. Perhaps it is time that you considered buying into technology called Hierarchical Storage Management, or HSM. Here is the idea: rather than just purchasing more disk drives and filling them up, you start looking carefully at how your end-user population stores its data. I remember back in the good old DOS days when network administrators could keep track of their networks merely by seeing the daily incremental storage on their servers. Since files were small, these daily changes were also small. Unfortunately, those days are over. Creating a small four-color piece of clipart can occupy 50 megabytes, so storage has become more of an issue. Your HSM software keeps track of how often your users access various files and categorizes them according to their age of last access. You can also set what are called high water marks (think of tidal actions here) so that when your magnetic disks fill up past a certain level migration starts automatically, and you never have to see that "disk nearly full" message ever again. Older files that haven't been touched by any human hands for months can be moved to an archive on tape. And files that only see action at certain times (such as at the end of the month or once a quarter) can migrate to an optical disk. The reason why you would want to do this is that both tapes and optical disks are cheaper on a per-megabyte basis -- figure on around a buck a megabyte for magnetic, mere pennies per meg for tape and optical. As the files are moved off the disk drives of your front-line file servers, a special indicator file, called a stub, is put in its place. Thus, when an end-user does a directory, she or he thinks that the file is still on the server. If a user were to try to open that file, the HSM software recalls it from its archives and returns it to its right place on the magnetic disk. The user may have to wait a few seconds while this process happens, but otherwise the process of finding a migrated file isn't any different from finding one that is still living on your magnetic disks. And the stub files don't take up much room on your hard disks. That's the theory. HSM products have been around for mainframes for years, and have finally come to LAN systems. We'll get to the actual practice soon enough. Testing HSM products is no small matter, when you begin to really think about it. Since you are examining server behavior, you don't necessarily want to do this on a production server with real data and real users. However, you are also examining the behaviour of a large group of users as well. This means that your tests need to have some dose of reality to them -- so you want to be working with a copy of actual data on a test server at least initially. It also means out of necessity that any HSM tests will be fairly time consuming, since you'll want to make sure that the file migrations happen according to the rules you've established and that data just doesn't disappear into thin air. The more I thought about this technology, the more worried I was that things would work properly. After all, it is alot to ask that one's data remains intact. We are all scared of putting our data on some black box and then hope that we don't end up in a pickle when our users try to get their files back. I didn't think about these and many other factors when I first began discussions on HSM technologies last October with the Chicago-based Treasury and Capital Markets group of a large Dutch bank's holding company called ABN-AMRO Bank. The folks in the division's IT department very graciously agreed to have me work with them these many months on three different HSM technologies: from Avail Systems of Boulder, Colo., from Micro-Design International (Winter Park, FL), and from Cheyenne (Rosyln Heights, NY). They picked the HSM products, based on their own needs. We'll examine the first two products in this series and report on Cheyenne at a later date. The bank has a huge data storage habit (this particular division has over 40 gigabytes of files stored on several NetFrames running NetWare), and the IT staff was concerned: "We are definitely up against a brick wall with our storage capacity," said Craig Lenz, a senior network integrator with the bank. NetFrame, as you know from reading my column [11] on superservers, charges a premium for their disk drives and they are the sole source for them as well. "We want to avoid adding expensive hard drives because we know we will need some sort of HSM some day. We believe that the rate of our growth in our disk storage, along with the changing nature of our network data, (i.e. more faxes, graphics stuff, voice annotation, etc.), will only make the storage requirements grow more rapidly," said Lenz. "We have traders that like to see lots of historical data for their algorithms," said David Price, an IT manager for the bank. "We just increased our server's storage, and I wanted to put some solution in place that would last for another couple of years," he said. Next week I'll tell you about their selection criteria and the gear we actually tested. col 18 If you don't have a lot of time on your hands, don't think about testing any HSM products. Working with these products, which migrate files from your standard magnetic disks to near-line tape and optical drives, takes a great deal of elapsed time and energy. I first started working with ABN-AMRO bank's Treasury and Capital Markets group last fall with this technology. Since that time, the staff at the bank was busy trying out two different products from Avail Systems and Micro Design International. Quite frankly, I was impressed with the level of effort and their professionalism. I've been to about a dozen different sites since my odessey began, and these guys are one of the best gang that I've worked with. The first thing that David Price, the leader of the IT group testing these HSM technologies, did was to put together six criteria for selecting his ideal HSM product: 1. Simplicity. Fewer features is better than more inclusive and complex. "We want to be up and running after a disaster with as little expertise and hassle as possible," said Craig Lenz, a senior network integrator at ABN-AMRO. The bank wanted a system that relatively unskilled operators could manipulate, even at the expense of more features. 2. Capacity. They wanted something with at least 40 gigabytes of storage, that could handle their most demanding users. 3. Easy backup and recovery. The bank wanted to be able to make copies of the optical media and be able to integrate the HSM routines into their existing backup/recovery procedures for their magnetic media. 4. Hardware support. Being a bank, they were picky about the ultimate optical and other assorted hardware they wanted to use. Having an HSM product that supported a wide range of hardware was important, so they could choose what optical jukebox and tape drives they wanted. 5. Bullet-proof. The bank needed to be able to get access to its data even if migration routines weren't working. This means that products that offered a single point of failure, or those that used their own data formats, would not be their best choices. 6. Archiving. The bank needed the ability to archive media as the HSM storage devices filled up. Their preference was optical drives rather than tape, given the longer life of optical storage. Price and his crew identified the products that I mentioned. Now we had to go about getting the stuff in for evaluation. As those of you know who are regular readers, my deal with the companies whose products I review is for them to give the sites their products free and clear, with a limited time of support as part of the deal. For these products, we had to bend this rule somewhat, since the level of hardware involved here is huge: the jukeboxes (so-called because they can swap many different optical platters on demand) can run upwards of several tens of thousands of dollars. However, we negotiated a long-term loan from HP for one of their jukeboxes (it holds 16 plattters for a total of 40 gigabytes of storage). We used this jukebox for testing the Avail software. MDI wanted to send us their own jukebox that they OEM from HP, which was actually the same model, just with MDI's nameplate on the front. And, the Avail product requires a DAT tape drive in addition to the optical jukebox, and so we used an HP JetStore 6000 tape drive as well. Just ordering the software was an exercise in careful study, an indication that this market is not quite ready for prime time. Avail has the most complex pricing scheme of the three vendors: besides paying for their software, there are extra charges depending on the kind of optical jukebox you order (ranging in price from $744 to $4480, obviously the more capacity you need the more you pay) and the kind of tape drive you use ($1200 to $2039). And add to this another $329 for each additional server that you have on your network, since you have to load server-based drivers to handle the migration for each server that you want to enable the HSM technology. MDI is much simplier, but still has two separate software modules that you need to buy. Once we ordered the gear, our next step was to analyze our existing storage patterns to try to determine how we would set up the various migration scenarios and get a feel for what would happen once we turned these products loose on the network. What first stuck the bank's evaluation team was that there wasn't anything in these three products that could tell you on a directory-by-directory level any details about file aging and usage. Neither Avail's NetSpace Advisor nor MDI's Storage Analysis, which they both give away for the asking and both run on an ordinary Windows workstation, offer any of this information. Both products provide all sorts of statistics for your entire NetWare volume as a whole, but not for each directory. "We wanted to determine which of our users were the worst offenders in terms of hogging disk storage," said Lenz. They ultimately settled on a product called Space Hound Pro from FineWare Systems (loco TK) to do this analysis. Next week I'll review how the products were installed. col 19: Patience and persistence are two qualities that are going to be required if you want to install any of the two HSM products that I tested at a Chicago-based bank. We found all sorts of problems during this phase of our project. Neither the products from Avail Systems or Micro Design International went in smoothly, indicating to me a certain immaturity in this marketplace. Avail's NetSpace was the first product we installed. It took two days to get it setup but four weeks to tune it properly. Our first problem was that the DAT drive that HP loaned us arrived DOA. Luckily, the folks at the bank had a spare Mountain DAT drive that Avail's software would work with. Next, in order to use the Avail product we had to install a second SCSI card in the NetWare server. One of my biggest bug-aboos with Novell is this second SCSI deal: if you use a SCSI card for your magnetic disks, Novell wants you to install a second card to attach anything else to your server. It has something to do with performance and timing issues, but it amounts to a major pain in the neck, and our crew at the bank would agree with me. "We tried all sorts of SCSI cards in our test Compaq Proliant server before we found one that worked," said Craig Lenz, one of the people at the bank testing the HSM products. Lenz went through three Adaptec cards (a 2742, 1510, and a 1522) and two from Compaq (a 6620 and a 710) before he found that the latter actually worked with the Avail software. Once we found the right SCSI card, we then had problems installing the Avail software. We quickly abended our server (meaning that when we loaded the software, the server would just roll over and die, resulting in the red-switch treatment to bring it back to life). After lots of technical support help from Avail, we found the culprit: Novell's own supporting NLM software! You have heard this tune from me before: Novell is sometimes its own worst enemy when it comes to producing its file server software. The problem is that the NetWare OS requires several NLMs. Novell has a nasty habit of updating these NLMs, and many products don't work with older versions. However, with Avail, we had a new twist on this scene: their software didn't work with the NEWER versions of CLIB, NWSNUT, and AFTER311 NLMs. Our crew at the bank had to downgrade to earlier versions of these NLMs before the Avail software would work. That's a big problem, given that no one wants to go forward into the past. We had two other problems: because we got the HP jukebox directly from HP, they had included a product code in the boxes' firmware that the Avail software wasn't used to seeing. This took some careful work with an on-site visit from an Avail rep. And, Avail had originally sent us an evaluation copy of their software which expired after 45 days. The bank wanted to test the real thing, meaning more time lost in doing a reinstall. By comparison, our installation of MDI was relatively hassle-free but still took some time: "What would have taken an hour for the installation took an elasped two weeks because of mismatched software components between EZ and SCSI Express," said Lenz. Next week I'll tell you what we actually found out when it came time to use the products. Product data: NetSpace v.3.0 $2749 plus "capacity kits" for optical storage media [$2129] plus "capacity kit" for tape drive [$1860] plus additional server drivers at $329 each Avail Systems 4760 Walnut St. Boulder, Colo. 80301 303 444 4018 800 962 8245 303 546 4219 (fax) Hardware: (for Avail) Optical Jukebox 40 T (40 GB capacity) Jetstore 6000 4 GB DAT tape drive HP DAT drive Mountain Network Solutions SCSI Express v1.4.6C $ TK price EZExpress v1.0.1 $ 2995 541MX Optical Jukebox $16,400 Micro Design International 6985 University Blvd Winter Park, FL 32792 407 677 8333 800 228 0891 407 677 8365 fax Server Hardware for MDI and Avail Compaq Proliant running NetWare 3.12 ( one 2 GB volume, three 100 MB volumes) col 20 The hardest thing about testing hierarctical storage management products (HSM) is figuring out what you are trying to test. These products allow for migrating unused or lightly used files from expensive magnetic to cheaper tape and optical drives, and do so in some orderly fashion. The trick is figuring out where your data is when your users need it, and how it will be backed up. We tested two products from Avail Systems and MicroDesign International at a Chicago division of a Dutch-based bank holding company called ABN-AMRO bank over a period of several months. Avail is a full-featured HSM product, while MDI's is simpler and has fewer features. We had several issues with the Avail product, called NetSpace: First, it has crummy notification when recalls fail. It puts an entry into its log, and also sends a message via the 25th line and Novell's SEND command. The user notificatiion isn't terrific either: users get a DOS error code rather than something in plain English. If you don't check your logs or your supervisor isn't logged on at the time, you'll miss out. Neither of these notifications are ideal, but are somewhat better than the MDI products: they have no user notification whatsoever. Even under normal circumstances, notification isn't where it could be: you have to load Avail's TSR on each workstation for users to receive a "please wait, file being recalled" message to popup. Without this software, users might think their machines have hung up while the file is being recalled. Second, Avail uses its own file system. This means that if you want to restore a migrated file, you have to use their software to get it back. That is less than comforting than MDI's solution, which writes the migrated files to a standard NetWare-formatted volume. However, the Avail system doesn't require any extra RAM needed to support the additonal NetWare volumes (A general rule of thumb is that you'll need in megabytes of RAM about 7 times the gigabytes of storage volumes -- this means that to add 40 gigabytes of storage in NetWare format, you'll need 280 megabytes of RAM on your servers!) Third, finding a file can be extremely complex. It isn't easy to determine where migrated files are located -- they can be on tape, magnetic, or optical media. "Avail's product is extremely complex. I may need an 'Avail Certified Engineer' trained to run the product. I am afraid to think what would happen after a disaster here and how long it would take to restore our files," said David Price, the IT manager for the bank that I worked with on these evaluations. I agree with Price. Avail is a multi-headed hydra: it has separate server console screens and 11 different NLMs to install on the server. Let's move on to the MDI software. First off, the software, which is a combination of EZ Express and SCSI Express, doesn't have either high or low-water marks to control the migration of files. You want to be able to set these so your disks don't fill up, for example. Avail has this control and is quite flexible. MDI doesn't have any way to monitor your magnetic volumes to determine how full they are either -- again, something that Avail provides. MDI doesn't support NetWare 4.x, although it is coming soon according to company representatives. Finally, as I mentioned earlier, we had to install plenty of RAM in server to handle the migrated volumes, because they have Novell formats and are thus mounted along with magnetic ones. As you can see, Avail has more features and functions than MDI, but you pay the price of complexity. The biggest issue for the bank was the lack of any archival layer in the software: meaning that you couldn't migrate files to an optical disk and then remove that disk offsite.The bank wanted to be able to keep this migrated disk around to recover from any unlikely disasters, or to just have a spare set of their data offsite in an nice and simple package. Without this archival layer, it is difficult for the bank to figure out where their files might be located on either the Avail or MDI systems. Neither product integrated well with the bank's existing backup scheme, again making either locating files or disaster recovery very difficult to accomplish. Price said it best: "Based on what we've seen so far, HSM isn't a very mature market yet. Customers don't necessarily know what they want, and vendors aren't that consistent." As I mentioned a few weeks ago, we had tested a third product from Cheynne Software. I'll hope to have the results of these tests at a later date.
Click here to return to the
previous
page
David Strom
Port Washington, NY 11050 USA
US TEL: 1 (516) 944-3407