Storage

January 6, 2005
Leading issues, opportunities in storage networking Part - I

Geetanjali Wadhwa & Pradeep Chakraborty

Storage networking is among the hot topics today. In this first part of a two-part report, Convergence*plus looks at the leading pain points in the storage industry; the best practices; the opportunities in the storage and services market; intelligence in storage networks and distributed storage; converting market potential to reality in SMBs; challenges facing successful ILM; future of IP-SAN, as against SAN, NAS and Fiber Channel; and how network attached processing (NAP) can help the storage industry.

Part two of this report, to be published in the February 2005 issue of Convergence*plus, will look at whether one size fits all and in which direction is storage infrastructure headed; successfully combining database and storage to prepare for future data growth; need for on-demand storage in an on-demand world; convergence in the storage industry; how SATA can improve TCO; current state of the storage industry in India, and how some leading vendors perceive the way ahead.

BANGALORE AND NEW DELHI -- Kumar Malavalli, the leading storage networking guru, an industry entrepreneur and technologist, co-founder and CEO, InMage Systems, co-founder, Brocade, and chairman, Cyrca, made a presentation titled "Protection of 'Crown Jewel' - Information", at the Storage Networking Summit held recently in Bangalore. Data protection was a subset of data continuity, which involved the process of securing and backing up data, so it could be recovered during a disaster, he said. There was a need to classify information, and then match and place that information transparently at the right class of storage. This would enable policy driven tiered storage with automated migration.

He added there was a need for a unified solution that combined remote replication, continuous data backup and information tiering; increased the availability and faster backup and restore, and aligned to business events for both local and remote data continuity; and a significant reduction in TCO with little impact and risk on host. Users today demanded data recovery with little or no human involvement. Consolidated data protection was inevitable and would happen in just a matter of time. The storage-networking guru stressed on making data continuity the mantra for protecting data. In brief, he focused on successfully combining database and storage to prepare for future data growth.

The very next week, the industry was abuzz with the news of the Symantec, VERITAS merger. According to published reports, Symantec Corp. and VERITAS Software Corp. announced that they had entered into a definitive agreement to merge in an all-stock transaction. Based on Symantec's stock price of US $27.38 at market close on December 15, 2004, the transaction was valued at approximately US $13.5 billion.

The leader in storage software and the leader in security software will provide enterprise customers with a more effective way to secure and manage their most valuable asset, their information. The combined company will be uniquely positioned to deliver information security and availability solutions across all platforms, from the desktop to the data center, from consumers and small businesses to large organizations and service providers. The combined company will operate under the Symantec name. John W. Thompson, chairman and CEO of Symantec, will continue as chairman and CEO of the combined company. Gary L. Bloom, chairman, president and CEO of VERITAS, will be the vice-chairman and president of the combined company. The board directors of the combined company will include six members of Symantec's current board and four from VERITAS' current board. Based on IDC data, the total market opportunity for the combined company is US $35 billion approximately, and is likely to grow to US $56 billion by 2007. This merger is perhaps, the best example of security and storage going hand in hand, or, the intersection of storage and security, discussed in a separate article on information security. Going back, both data and data continuity require storage, among the fastest growing segments in the country today.

Leading pain points in storage networking
Storage is omnipotent. It has been doing the rounds since the advent of computing. It got into prominence recently, and is now buzzing with activity. However, like most segments, it has its own pain points. Before getting into the issues involved with storage, let us look at the leading pain points.

B. Chandrasekar, country manager, India, Intransa Inc., said the escalating costs of 'acquisition' and 'management', which were leading to a lower RoI; competing technologies leading to complexities in storage management; and standards not being applied consistently; were the leading pain points. P.K. Gupta, chairman, SNIA India and director strategic development-Asia Pacific, EMC Software, listed ten points from the customers' perspective. These are: cost (price and TCO); managing growth and meeting capacity needs; inability to manage storage assets and infrastructure; lack of integrated and/or interoperable solutions; increasing complexity of storage infrastructure; poor service and support; lack of desired functions and features; finding the right solutions and justifying expenditures; undelivered promises; and lack of robust automation for provisioning.

Anand Bhalve, vice chairman SNIA India and director Engineering, Spartan Labs Pvt. Ltd., added that the storage industry had gone through a phase where the major pain point was interoperability. Initiatives like plug-fest, organized by SNIA, had given a platform to the vendor community to get together and uncover interoperability issues. The storage industry had made significant progress and needed to continue doing this. The End User Council (EUC) of the Storage Networking Industry Association publicly released its top 10 'pain points of storage' at SNW in Phoenix early in 2004, most of which were touched by Gupta.

Gupta added that new ones had joined old drivers, contributing further to the need for well-managed and interoperable storage. "Data growth is projected at about 80 percent a year. Much of this growth is in unstructured data - file system data. In the previous years, we saw growth in structured, transaction-based application data. While this type of data continues to grow, we now see unstructured data, such as records of business, content and email growing 80 percent faster than structured data. Complexity is a big issue as the number of products and solutions available to manage data proliferates. This is compounded by the lack of skilled people who are available to manage the storage infrastructure," he added.

Consolidation was said to be the way ahead to address problems of out of control growth and complexity. However, it was also a potential challenge as the tools necessary to do storage and server consolidations were emerging. In 1999, 52 percent of the companies indicated they were planning to do some storage and server consolidation. That number jumped to 79 percent in 2004. Gupta added that while the statement of 'interoperability' as an issue was not visible much nowadays, by no means has this problem been solved. "The bottom line is, all of these problems contribute to the number 1 customer issue, which continues to be cost. Growth, complexity and lack of resources - all of these things contribute to increased costs of doing business," he noted.

Phillipe Cazaubon, director-sales, Asia Pacific, Overland Storage, quoted IDC from Storage Magazine, April 2004, that backup speed - 90 percent; TCO 75 percent; restore accuracy 74 percent; and support cost 65 percent, as the leading pain points. Arun Rawtani, country technology services manager, EMC India and SAARC region, added that the biggest challenge in India was educating the market on the advantages of networked storage and getting organizations to look at storage as a building block of their enterprise information infrastructure. Besides cost, key pain points included getting qualified storage administrators who were in short supply; storage management interoperability for cross-vendor products; data protection (business continuity/disaster recovery), and rapid growth of data (scalability).

Amod Manjrekar, CTO, Renaissance Softech, the master reseller of Brocade in India, pointed out that the customer was not aware of what he wanted to buy or whatever recommended to him by SIs was right or wrong. Every OEM has different set of solutions and product to address a single problem, which leaves a customer in a fix as to which was the right solution for his organization. Next, interoperability issues prevented customers from having vendor neutral solutions.

Sharad Srivastava, country manager, Seagate India added that the storage industry currently seemed to be doing well due to huge growth in consumer electronics. The leading pain point was managing the delicate balance of supply and demand. The ability to predict market demand, both timely and accurately, would save the cost of producing excessive drives.

Sanjay Kharade, principal consultant, Cisco Systems, India and SAARC, noted that information and data had transformed to become the most important corporate assets in today's business environment. With most organizations deploying data-intensive enterprise applications like CRM, ERP and emails, there was an explosion of information and data, and hence, a need to manage it effectively. IT managers faced issues such as difficulty in managing large, disparate islands of storage from multiple physical and virtual locations; complexity of maintaining scheduled backups for multiple systems and difficulty in preparing for unscheduled system outages; inability to share storage resources and achieve efficient levels of subsystem utilization; shortage of qualified storage professionals to manage storage resources effectively; confusion over the justification of the plethora of storage technology alternatives, including appropriate deployment of Fiber Channel, Internet Small Computer Systems Interface (iSCSI), Fiber Channel over IP (FCIP), and Infiniband, etc.

According to Durga Prasad Allada, senior consultant-storage practice, Satyam Computer Services Ltd., the leading pain point from the end user perspective was the storage management issue. Managing growth to meet the growing capacity needs by an enterprise, besides the complexity of the storage infrastructure added to it. Some other key aching areas included cost (i.e., the cost to procure, run and manage the storage infrastructure or TCO), which was still very high. Next, non-availability of interoperable solutions that are capable of seamless integration was a priority area. Another area was the lack of effective and well-educated channels for creating confidence and acceptability of the technologies by end users and market. Finally, the availability of feature richness and robust automation in storage solutions. Independent storage architecture and consultancy services providers like Satyam could play a very important role in removing these pain points as they have an understanding and perception of these issues from both the sides and could help find answers to scalable, robust and manageable networked storage architectures, he noted.

Best practices in the storage industry
If pain points exist, companies and vendors would have surely evolved best practices. Bhalve said the best practice was really a process evolved within an organization based on experience in dealing with storage and related infrastructure. These organizational experiences, when shared across end-users having similar requirements, could result into best practices. These best practices could be applied to different categories, including systems implementation, storage reliability, data recovery, etc. In India S.R. Balasubramanian, vice president-IT, Hero Honda, is representing SNIA India EUC. Bhalve encouraged IT managers, data center managers/administrators, CTOs and CIOs to get Balasubramanian for coming up with best practices.

Gupta from SNIA added that the best practice was a proven industry standard method or technique to best meet a specific requirement. Best practices may be unique or a combination of several practices. "For SAN best practices, always check for compatibility between SAN components. Multi-vendor may be alright, but dependencies get complex quickly. Always run a pilot before deploying. Various host bus adapter cards may not work together. Persistent binding for tape devices is must," he added. Business practices should be tested at least once a year. "Do not trust backup configuration until you have tested a recovery. Always cross check hardware and software with the compatibility guides on the public Web site. Read various other guides like the disaster-recovery guide and the performance-tuning guide," he suggested. Some best practice principles are:

• Deploying enterprise-wide data protection/storage management, rather than departmental storage management;
• Implement storage management software in a highly available environment providing increased accessibility of data; Cost of downtime is very high. For example, if an ERP system is down for one hour, it costs about $750,000.
• Implement storage management software to leverage existing IT infrastructure, which should be capable of seamlessly integrating into newer storage technologies like NAS and SAN. Storage management hardware and software should be scalable to take care of new technologies such as NAS, SAN, iSCSI, etc.
• Reduce cost of ownership by implementing storage management with a 'lights-out' and 'hand-off' approach to contain the operational costs;
• Have an intuitive, user-friendly, Web-based GUI that is manageable from anywhere;
• Have efficient disaster recovery capabilities. Enterprises need to have efficient disaster recovery capabilities for data and applications to recover businesses in the wake of building, managing, and supporting enterprise customers. Solutions may start from traditional tape backup to serverless backups to data replications to WAN clustering based on recovery from days to minutes and availability from 0 percent to 100 percent.
• The storage management software should support multiple OS (NT, Win2K, Linux, Netware and various flavors of UNIX), applications (ERP, Lotus Notes, MS Exchange, etc.,) and databases (Oracle, DB2, Sybase, Informix, etc.);
• Implement a storage management solution for high performance, including streaming of multiple streams of data to backup devices.

According to EMC's Rawtani, the recommended best practices include identifying all the interfaces of the storage network; creating a separate infrastructure for the out-of-the band management and control terminal interfaces to the storage network; maintaining a formal set of company best practices for storage security; protecting data in flight and data at rest; using dedicated user IDs for storage network maintenance access, and enforcing the use of strong passwords, either by policy or by configuration. Use of separate credentials again for infrastructure configuration functions; defining zones containing the smallest possible number of components, and using different zone sets for different system loads, such as the off-hours backup time; using all available LAN security tools, e.g., VLANs, Ipsec, etc. restricting access to infrastructure configuration functions and always using hard zones as against soft zones; only installing software and firmware on storage network components from authorized sources, but never doing so when a device was connected to a production storage network. Where possible, configure storage devices to not accept firmware upgrades via the storage network interfaces; always changing the default passwords before the equipment was connected to a production storage network; monitoring the storage environment; and stopping external attacks: DOS, viruses, etc.

Seagate's Srivastava said that the ability of the hard drive manufacturer to manage the supply chain had significantly helped lower the production cost. The relationship with component suppliers was tight-knit in the hard drive industry. Besides meeting the stringent performance and reliability requirement, component suppliers needed to ensure timely shipment. The close working relationship with its supplier was the key factor, which accounted for the success of the storage industry.

Satyam's Allada added: "One definition I heard on best practices is that 'Best practices are identification and validation of thoroughly tested methods that allow industries to learn from others' attempts, avoiding costly and time-consuming mistakes and duplications of effort". So best practices is a journey, rather than a goal of a constantly improving methodology. The storage industry has reached some milestones and hence also some best practices evolved. Yet, a lot more has to be achieved and a greater distance to be covered as the storage industry starts maturing and growing further. Real-life scenarios, user experiences, expertise of technology companies and contributions from representative bodies like SNIA, all play a vital role in identifying and validating the methodologies that can be best practices." Business needs assessment and technical requirements analysis for any storage deployment was a practice that must be put in place for the best RoI. A well-defined change management process and a well-documented storage and IT infrastructure, architecture and design would help manage resources well.

Cisco's Kharade listed efficient utilization of resources; deployment of scalable, storage network infrastructure to grow with future requirements; multiple system access to multiple storage devices, regardless of location; consolidation of multiple low-end systems into centralized, high-end systems; reduced administrative overhead with centralized storage management; and secure SAN deployments, including role-based management and extensive fabric security services, as the best practices. Manjrekar at Renaissance added that a proper study of the existing infrastructure was very important for both vendor and customer. Procuring what was required was the most appropriate best practice.

Huge opportunities in storage and services market
Having looked at the pain points and the best practices, we now need to examine the opportunities existing in the storage and services market. Intransa's Chandrasekhar said a lot of industry analysts were looking at the storage and services market growing 14 percent on a yearly basis. It was the only IT segment that was even growing during the downturn.

Seagate's Srivastava highlighted the consumer electronics (CE) market was one of the fastest growing segments for disc drive makers. Driven by a host of new storage-based devices such as digital video recorders, MP3 players and digital cameras, disc drive shipments in the CE market were likely to grow from 17 million in 2003 to 55 million in 2006, according to TrendFOCUS, a market research firm. He noted: "That growth is especially robust in the Asia Pacific region, where Seagate is working with several major customers to develop new, innovative applications that are extremely reliable and able to meet the performance levels consumers expect." Renaissance's Manjrekar agreed the market was big, yet untapped, and services would to grow along with storage sales.

Cisco's Kharade pointed out that the network storage business was likely to grow by over 50 percent in the current fiscal. This optimism was shared by IDC, which predicted that the overall storage systems market would grow to US $250 million by 2005 at a CAGR of 76 percent. The mood was upbeat in the NAS and SAN markets as more business verticals were willing to embrace these solutions to enjoy long-term benefits. As per IDC, NAS products would account for 32 percent and SAN products 49 percent of the market in the total storage pie. SMEs were adopting these solutions as well. If figures are anything to go by, there is no doubt that India's network storage market would grow at a healthy pace in the next couple of years.

According to Gupta the global storage market was predicted to grow to US $50 billion in 2004 to about US $90 billion in 2007. NAS would grow from US $1.5 billion in 2002 to US $4.3 billion by 2006, as per IDC, and SAN would reach US $84 billion in 2007, as per Pioneer Consulting. Software and service would be a major chunk of all this. Bhalve said the Indian storage market was set to grow at 17 percent from US $134 million to US $425 million during 2000-2005. The Indian disk storage industry alone was likely to grow from 2,258TB in 2002 to 52,700TB in 2008. This meant tremendous potential for resellers, system integrators, etc. He added: "Two years back, the storage service provider (SSP) was expected to be a big thing in the US! However, due to lack of trust by end users, it did not take off as expected. These days, there are lots of small sprung-ups that are doing one-line backup service and it is catching lot of attention."

EMC's Rawtani said the double-digit growth in storage revenues was a result of trends and opportunities such as information growth, increased demand by SMEs, regulatory compliance like the Sarbanes-Oxley Act, HIPAA, Basel II, and RBI and SEBI guidelines, which mandates companies to store data for a few years, and enterprise applications like ERP and CRM, which were driving growth in a big way.

Satyam's Allada confirmed that the market was responding to new storage technologies and investments in consolidated, networked storage infrastructure across enterprises was growing as well. Demand for storage was almost doubling every year globally. In India, storage capacities in enterprises had grown at a good pace annually since 2002 till date. IDC forecasts indicated storage capacity in disk storage shipments rising from 2Petabytes in 2002 to 10Petabytes in 2005 and going up to 52Petabytes in 2005.

Major players no longer talked of storage as a box and their product lines did not stop at storage hardware. The portfolio included hardware, management software and goes beyond to information management and disaster recovery solutions. With need for availability of data on demand, often in real time, significant information management concerns in recent times were related to storage. These included backup policies, data security, regulatory compliance, QoS, assurance of service by public and government bodies/agencies, large private companies, educational institutions, law enforcement agencies, utility firms, etc. The storage services market was an area with immense potential. Storage companies and SIs had only recently started focusing on services. "The storage services market and storage as a service are two areas, besides standards based technologies that will open up new avenues. Besides, storage management services outsourcing was fast becoming a reality," added Allada.

Converting SMB potential into reality
While on the topic of opportunities, SMBs are said to have lot of potential. The challenge is to convert that potential into market reality. SNIA's Gupta said that with costs going down and awareness levels going up in SMBs, the market potential was very high. SMB customers had started realizing the opportunity cost of not implementing right storage and disaster recovery strategies. "Whatever is enterprise today, it was SMB yesterday and market potential is market reality for the storage business. We feel the same applies for the SMB today," he said. Renaissance's Manjrekar said that there must be knowledge initiatives from the various vendors to make their products and technologies nonproprietary so that everyone was aware of the upcoming trends and was less influenced while taking decisions.

Satyam's Allada noted that the growing attractiveness of the SMB market in USA and other countries like India presented a very good opportunity. Its need for storage (not capacities) as a resource and associated pain points were common with the large enterprises as well. SMB's market potential could be converted to market reality by adoption of right products from new storage technologies. According to him, adoption of iSCSI as a storage solution architecture in an SMB would offer the advantages of storage network at an initial lower investment and a faster RoI. The iSCSI-based products integrated very easily into the SAN fabrics as storage needs grew and they become a part of the larger storage network seamlessly. Similarly, availability of storage services offerings tailored to SMB space would convert the market potential into a reality very fast as the SMB storage needs could be met by lower TCO and in turn, a higher RoI through innovative service offerings like hosted storage services, remote backup and recovery services, data management services and outsourced storage management services. These offerings addressed the problem of lack of storage skills in SMB space effectively as well.

Cisco's Kharade said that organizations should help SMBs create value that extended to their customers and, ultimately, to competitive differentiation and the bottom line. Cisco recently launched its SMB Class Solutions for Indian SMBs. Bhalve noted: "If you are able to show value that a 'SAN solution' brings to the table, based on requirements of a particular business, you should be able to convert the potential into business. In other words, it is necessary to educate the end user to realize the potential into business." Chandrasekhar said SMB customers, unlike large enterprises, did not look at a technology's viablility alone to solve their business needs, but looked at other attributes like monitoring and management tools. IT was not their core competency and they did not want to invest their scarce resources into building it.

According to Seagate's Srivastava, mid-size companies required the enterprise storage solutions that enabled basic online SCM activities as well as nearline storage of non-critical data and backups. "As IT budgets shrink, storage solutions have to be cost effective and easily integrated into existing infrastructures with minimal changes or modifications. General lack of IT personnel drives mid-size companies to look for storage solutions that are easy to deploy and maintained," he pointed out. Further, the market potential of ILM could be potentially realized in the SMB space by communicating its benefits clearly. ILM solutions for SMBs should be perceived to be easily deployed and maintained even by non-IT personnel. They must be assured that investment in new IT infrastructure would be protected. What will drive acceptance levels is ultimately the value that SMBs received.

Overland Storage's Cazaubon said its REO was a great example of how SMBs could take advantage of new technologies to emulate enterprise-class deployments. The iSCSI allowed customers to deploy IP SANs, which were very similar to FC SANs, but at a tenth of the cost and were much easier to manage. He said: "REO takes full advantage of iSCSI capabilities and is a cost-efficient way to provide inexpensive near-line storage to SMBs."

Rawtani said the SMBs would drive growth for information storage and management solution in India. According to IDC estimates, the Indian IT industry was projected to grow at a CAGR of 21 percent till 2006. The SMB segment would drive a significant proportion of IT market growth in 2004. According to IDC [in a multi-country report, September 2003], approximately 10 percent of small organizations in India had installed SAN, 4.9 percent were in the process of implementing it, and 8.6 percent were considering implementing a SAN solution. EMC estimated that the mid-tier storage market would more than double by 2007. EMC is betting big on the mid-tier space and is investing in educating, training and certifying its partners to make available the best-of-breed storage solutions to the mid-tier customers. He added: "ILM is another trend that is fast catching up in India and we are seeing some signs of adoption in SMBs as well. EMC's vision is to help our customers get the maximum value from their information at the lowest total cost of ownership at every point in the information lifecycle."

Intelligence in storage networks and distributed storage
There has been a lot of talk about intelligence in the storage network. In that case, is the industry ready for distributed storage as well? Chandrasekhar said customers were ready for distributed storage for a long time, starting from the days of the Open Systems Forum (OSF) in the late eighties. Not seeing a true 'distributed storage environment with centralized management' had given rise to a lot of skeptics about this topic. Manjrekar noted that at some point of time customers would have to take into consideration that managed storage was the need of time as the Terabyte expanded to Petabyte.

Overland's Cazaubon agreed that distributed storage had been used for several years in form of NAS, for example. "More complex solutions such as Fiber Channel SAN have found a niche in the enterprise, but SMEs cannot afford those. The iSCSI is now a reality and many SMEs will be able to benefit from these deployments," he added. Cisco's Kharade added that storage systems and SANs had gone through different and well-documented revolution over the last few years. There had been an accelerating move away from DAS toward SANs. "Storage systems have become increasingly more sophisticated providing additional functionality such as replication, snapshot, mirroring, SRM and even policy-based management. Storage applications and intelligence are starting to show up in the SAN fabrics in the form of appliances and intelligent switches. For instance, virtual SAN (VSAN) technology partitions a single physical SAN into multiple VSANs. Cisco's VSAN capabilities allow the Cisco SAN-OS to logically divide a large physical fabric into separate isolated environments to improve Fiber Channel SAN scalability, availability, manageability, and network security."

Intelligence in a storage network was happening, confirmed Allada. Technology companies and storage user communities saw it as a need and accepted it as a requirement. The proof was in the pudding. "The much talked about and strongly desired intelligence in storage networks would really make the difference only if the products with this intelligence built in are available for deployment with an easy integration in the production and live environments. Today's ebusiness infrastructures create information like in a chain reaction. There is a whole lot of competitively valuable company-wide information in all these enterprises. Use of this information creates more information and that information creates some more information. With no proper planning, management and organization, huge piles of information scattered across the enterprise was creating islands," he added. A report from the Meta Group states: "Users must move aggressively to overlay a strategic enterprise storage architecture on to their exploding capacity requirements."

Intelligence in storage networks constituted many components and primarily, the hardware, the software and the services. It was an integrated model or approach to storage with appliances, applications and people. Intelligence in storage network is the need of the hour and storage architectures being considered and deployed in present times are well on track in this direction. The demands on storage today are higher security and data protection, good disaster tolerance and faster recovery times in the event of disaster, simplified storage management and resilience to growing capacity needs besides hardware independent information reach and sharing. The intelligence in storage network provides all these. Product agnostic, intelligent storage management appliances are an example of Intelligence in storage networks.

Satyam's storage solutions practice has built a pool of competencies and skills in the last few years and has invested in building proof of concept and IP in this arena. Development of its SatStor Storage management and virtualization appliance was a showcase of its efforts and work in the intelligence in a storage network. Allada added that customers were ready for distributed storage. It was the storage of a heterogeneous nature and nomenclature distributed across a fabric and the user community was ready for this architecture as they could maximize RoI on storage and minimize the TCO with the intelligence built in switches and the management software available today.

Gupta spoke about the trend back toward recentralization of resources during this decade. As there was a greater focus on availability and protection of corporate assets, many IT departments were consolidating data to provide increased service levels for protection, security, availability and access. It was particularly true for fixed content information (unstructured). "Perhaps, one of the biggest changes on the horizon is the move to grid computing and/or utility computing. The ability to bring together servers, networks and storage in a way that makes the deployment of these resources efficient, transparent and delivered 'on-demand' makes the case for recentralization of storage clear. Provisioning and providing 'capacity on demand' continue to be one of the top pain points for end-users, and the ability through grid or utility computing to make efficient use of resources, easily and transparently, is something we will begin to see occurring over the next few years," he said.

EMC's Rawtani said the prevailing architecture had transformed from DAS to NAS over the last three years. This created the foundation for the next phase of architectural evolution: intelligent storage networks. The need for intelligent storage networks was driven by members of the storage community seeking to ease the management, deployment and consolidation of small SAN islands into larger SANs. Many storage administrators want a unified way to replicate and seamlessly migrate information from one storage array to another - even if the arrays are from multiple manufacturers.

To address the emerging storage infrastructure needs, new capabilities were added to intelligent storage arrays, including greater consolidation and scalability through dramatic increases in storage performance, as well as advanced functionality, such as local and remote data replication. Such advancements help drive the storage networking evolution by solving challenges older architectures could not. Today, many users have intelligence distributed at all levels of the storage hierarchy - in hosts (path management software, HBAs, device drivers, etc.), storage networks and storage devices.

Bhalve added it was possible to put a lot of intelligence in each one of the storage components, especially switches and routers that sat in between data paths. He said: The important issue is how well the application can exploit the capability of intelligence built in these components. Some level of intelligence already exists today in major router/bridge offerings (like copy manager)." Seagate's Srivastava tied distributed storage with information life cycle management (ILM). ILM was defined as a standards-based approach to automating data center based on business requirements and business processes. The fundamental aspect of any ILM strategy was to "distribute" information to different tiers of storage as the value of information changed over time. Moving storage to an appropriate platform helped organizations cut storage cost by more than half. "Moving to a consolidated storage architecture is a multi-step process. Storage managers must understand what storage is being used for, and who is using it before planning a networked storage implementation," he added.

Challenges for successful ILM implementation
Srivastava added that successful ILM depended largely on how effectively companies were able to create classifications for all their data and inventory of their IT assets, determine which types of storage each machine used, and design a beginning to end-of-life strategy for their data given the storage available.

"Complexity of enterprise data storage management has increased due to growth of office data contained in word processing files, spreadsheets and presentations. The high degree of variance in data types, as well as the rate of growth and value to the business, adds to this complexity. Extracting value from the larger stores of information has become increasingly dependent on a company's ability to manage information such as who created a file, when it was last accessed and so forth," he added.

"Ultimately, what organizations need is a single platform that can measure business value of unstructured files based on their location, attributes, content and business semantics. This platform must deliver the ability to distribute the enforcement of file policies to the file owners, i.e., end users who create, share and use the files on a daily basis. And finally, it must minimize the work required by either IT or the end user to implement the solution. The bottom line is: only through a systematic approach to managing unstructured files as a corporate asset can organizations realize the full benefits of ILM," noted Srivastava.

Intransa's Chandrasekhar said a true ILM deployment needed a lot of elements of the storage infrastructure to work in tandem seamlessly and getting their cost effectively was the true challenge. Integration costs were not getting any less expensive, and hence, the need for more tools to help the IT folks. Gupta noted that ILM was a relatively new term for an old concept - managing information from the time it was created till the time it was deleted. "Unfortunately, storage vendors have made this term their own, and end-user confusion is the norm. SNIA is coming up with a standard definition for ILM, "A new set of management practices based on aligning the business value of information to the most appropriate and cost effective infrastructure."

According to him, ILM aligned business processes with IT solutions through the definition of appropriate service levels and policies. It spanned data and information management services, as well as the storage, compute and network infrastructures. "Few vendors already have solutions available and you will see more and more coming in next 12 months. The challenge is to classify the information according to its value and them manage it accordingly. The ultimate goal is reduced complexity," said Gupta.

Cazaubon felt that the main challenge was for companies to be able to clearly identify and qualify their own data/information. Several companies (such as banks) had actually been using the ILM concept for many years, although the name did not even exist. ILM was mainly a marketing ploy designed by a few vendors who were eager to lock customers into a one-vendor-only type relationship. While storage vendors can easily design software and hardware solutions that match the customer's requirements, identifying and qualifying information rests with the customer itself. Bhalve noted: "If you look at the ILM framework defined by SNIA it is very clear that any ILM solution will be driven by business requirements like age of data, value and other criteria. For building a foolproof solution, making the multivendor equipment and software work together seamlessly is the primary challenge. The other challenge could be related to a lack of understanding of various business segments for which the ILM solution is developed."

Kharade said that one of the most important principles of ILM was that data should reside in the most appropriate place based on its characteristics. "The problem is that when information continues to grow, one has to go through some kind of process to move up all the records. The challenge here is the new regulations and compliance rules. Now you need a mechanism to operate and protect the data smoothly. Another challenge is to manage all the information according to its value at every stage of its life - from creation and protection to archiving and disposal. It calls for application-level integration directly into the storage infrastructure to allow for policy-based, proactive management of the relentless growth of both structured and unstructured data," he said. According to him, IT departments faced major challenges in meeting information protection, availability and accessibility requirements of these business-critical systems in face of rapidly expanding user loads and exponential data growth.

Rawtani said companies faced three key challenges related to information management: strong growth of information, information playing a more important role in determining business success, and finally, information changed in value, and many times not necessarily in a linear fashion. Understanding the value of information should be at the heart of managing information in general. According to him, tiered storage was the first step of ILM. The next step was to automate it using software. EMC calls this as automated networked storage (ANS). Unified management can help monitor, configure, and tune the storage infrastructure. Services, solutions and partners comprised the next step.

Satyam's Durga Prasad Allada said ILM was an approach and a methodology. It offered an efficient way of managing a variety of content from traditional files to digital media content in an IT infrastructure. The promise of ILM was to match the right, required content with the appropriate storage resource to improve the RoI and enable better access, performance, and utilization. ILM required many building blocks, though it was still evolving itself. Backup and recovery were among its building blocks of ILM. A good ILM strategy can be put in place with laying out the plan as a foundation for efficient and effective management of information throughout its life.

A good ILM strategy needed round-the-clock data access and should be able to support heterogeneous environments. Right data at the right time on demand made ILM most effective. The other key challenges that need to be addressed for a successful ILM strategy, said Allada, were ILM policy for an organization, logical mapping of physical location of files into an abstraction layer, and that ILM itself may not have evolved as a strategy completely in the industry today.

IP-SAN to co-exist with others; grow faster in India
IP-SAN is another key trend emerging within the storage industry. The reason for the interest in IP storage networks is the promise of networking storage over an IP network, which offers greater efficiency along with cost savings. Rawtani said: "At EMC, our philosophy of being pipe and protocol-agnostic means we strive to support all of the storage networking transports and protocols. Our strategy is to qualify IP bridge and gateway solutions that will allow existing as well as new storage arrays to participate in an IP-SAN. We will also provide native IP connectivity for our storage products that will provide a scalable and cost-effective means to network storage on IP networks."

Chandrasekhar, who's firm is a leading player in IP-SANs, added that technologies such as Token Ring, ATM, FDDI, etc., which were hailed as the future of networking at different times, had fallen off on the way, in comparison with Ethernet, aka IP. "Ethernet is all too pervasive and has only become more robust in the last few years. Given this backdrop, IP-SAN is all set to make a significant mark in the storage market in the next couple of years," he predicted.

Commenting on the future for IP-SAN, as against SAN, NAS and Fiber Channel, Gupta said all would be there; it is not one vs. the other. IP SAN was picking up as low-cost storage networks for SMBs. However, NAS and SAN markets were growing quite fast as well. "Even in India, NAS/SAN implementations have gone up to 43 percent from 15 percent in the last two years. Enterprises are still using NAS/SAN and Fiber Channel for performance and security reasons. Moving forward, they will have the mix of all for lowering cost of deployment and operations," he stated.

Bhalve said Fiber Channel SAN (FC-SAN) had been around for a while, and was is a seasoned technology. Recently, an 8Gbs fabric standard had been announced and it worked with both copper and optic interfaces. "IP-SAN has tremendous potential and theoretically is future proof in terms of speed. However, it has to mature before enterprises start deploying them. Current deployments could be focused toward connecting FC-SAN islands. It may become first choice for companies that are implementing SAN for the first time. In India, IP-SAN may catch up because we do not have too much legacy to deal with," added Bhalve.

Allada agreed that IP-SAN was not going to displace or kill FC-SAN. It was leading to a co-existence and convergence of these technologies. IP-SAN would play a long-term role but not replace FC-SAN soon enough. It would take time for IP-SAN mature to that level. Besides, there were factors like security in an IP-based network, performance levels at which the FC-SAN operates, and features are expected in large storage networks. IP-SAN as a technology, needed to address these issues before it could really be seen as a viable alternative to FC-SAN. The results of the work going on in the 10 Gigabit IP-SAN area may change the scene a lot and make IP-SAN nearly replicate the performance of today's FC-SAN. However, the race was on with all that seemed to be happening in the 4Gbps and 10GbPS Fiber Channel domains.

"In the short term, SAN is still going to be a data center choice and IP-SAN will be a preference in small, networked storage environments. IP-SAN may be used more for connecting remote SANs. NAS will wane and IP-SANs will wax. We will see more and more IP-SAN environments fitting in well in the traditional NAS environments. Features of IP-SAN like ease-of-use with no agent support requirement, block-based data access that fits all applications, very high performance that reaches up to eight times and more than the traditional NAS performance and high scalability are going to be key factors in making IP-SAN a choice over NAS," commented Satyam's Durga Prasad Allada.

Overland Storage's Cazaubon said there was a lot of misunderstanding in terminology. There are two major types of SANs, one very complex and expensive based on fiber channel cabling and infrastructure (FC-SAN) and a simple and inexpensive based IP and existing LAN infrastructure (IP-SAN). He added: "IP-SAN is not positioned against SAN, it actually is a SAN. IP-SAN is positioned against FC-SAN and NAS. Its main advantage is that IP-SAN would continue to be well accepted, as it is an inexpensive alternative to FC SAN." IP SAN would prove to be a viable alternative or compliment to SMEs who were planning to consolidate their data storage.

As per Seagate's Sharad Srivastava, IP-SAN or iSCSI was based on IP. As Ethernet was widely deployed globally, iSCSI enabled businesses to tap data management and flexibility benefits of networked storage without high costs and complexities typically associated with traditional FC-SANs. With iSCSI, companies could cost-effectively deploy data center storage with sophisticated management capabilities for distributed environments. Hence, iSCSI had a bright future. It also enabled truly affordable disaster recovery, backup, and secondary storage solutions. The iSCSI protocol is likely to accelerate transition from direct-attached storage to networked storage by reducing costs and complexities associated with SAN deployment and making it feasible to deploy SAN storage solutions in places where FC-SANs were not cost-effective.

Cisco's Kharade said large enterprises and vertical markets like financial institutions, ITeS and service providers - who had huge amounts of data requirements, were slowly realizing the benefits of IP-SAN. In addition, IP data transport networks have been built to support the front-end and back-end of IP application servers and their associated storage. He added: "Unlike IP, Fiber Channel cannot be easily transported over lower bandwidth, long-distance WANs in its native form and therefore, requires special gateway hardware and protocols. Use of iSCSI over IP networks does not necessarily replace a FC network, but provides a transport for IP attached hosts to access FC-based targets. IP network infrastructures provide major advantages for interconnection of servers to block-oriented storage devices. Primarily, IP storage networks offer major cost benefits as Ethernet and its associated devices are significantly less expensive than the Fiber Channel equivalents. Besides, IP networks provide enhanced security, scalability, interoperability, and network management over a traditional Fiber Channel network."

NAP should work well with storage grid computing
The advent of network attached processing (NAP) is another interesting trend. Large data centers generally have hundreds of applications running on thousands of servers. Managing and maintaining the applications gets more challenging and costly each year as more applications are brought online and legacy applications are migrated to new platforms. This led to a need for reliable, predictable, flexible and scalable compute infrastructures for driving these new business applications. Hence, the birth of NAP!

According to Azul Systems, a leader in NAP, it delivers compute power as an enterprise resource that can be tapped by the existing servers and operating environments running Java and J2EE platform-based applications without requiring the expensive management overhead historically associated with scaling today's enterprise applications. Azul has compute pools that are comprised of ultra-high capacity compute appliances. Based on multi-core silicon technology, each compute appliance contains up to 384 coherent processor cores and 256 gigabytes of memory in a fully symmetrical multiprocessor (SMP) system. How beneficial can NAP be for the storage industry?

According to IDC, the quest for business value for each workgroup or enterprise application would drive the need for dynamic IT at the infrastructure level with virtualization a key trend, noted Srivastava. At the infrastructure level, systems developers and independent software vendors were shifting offerings to a utility computing focus. NAP offered unlimited compute power for Java and J2EE platform-based applications built on top of today's leading application server platforms. "This new approach does not require any application-level modifications, binary compatibility requirements or OS dependencies. NAP provides massive, shared compute capacity that can dynamically respond to the changing workloads, thus eliminating capacity planning at the individual application level," he commented. Cisco's Kharade shared the same view.

Gupta said that NAP, which had roots in similar changes that occurred in storage and networking, would provide massive, shared compute capacity that could dynamically respond to the changing workloads, eliminating capacity planning at the individual application level. He predicted: "This will work very will with the storage grid computing concept in coming year. I guess, it is very good for the storage industry. Japan already has taken the lead in this year."

Bhalve added there were initiatives for building storage processors and interconnect bus technology like Infiniband. This would certainly help consolidate the interconnect/ IO technologies. However, one would have to be patient till it received wide acceptance. If it takes off, it would help in improving interconnect speed and consolidation of some IO technologies. Chandrasekhar noted that with the advent of 10GE, the networking bandwidth would soon surpass compute power, which will make it necessary for deployment of the NAP to help bridge the gap in processing capability and available bandwidth on the networking pipe.

Overland's Cazaubon said NAP was more about processor virtualization that allowed users to run multiple applications on a pool of computing appliances, rather than running them on the individual servers. Virtualization engines, such as Vmware, could be considered variants of this concept. "Virtualization is already widely used in storage to create virtual pools and/or devices (such our VTD in REO). I am not sure how NAP could help further since it addresses mainly processing power, not storage resources," he added.

Satyam's Durga Prasad Allada said: "NAP is changing the way the compute server resources are being used today. In this market space, the focus is on the aspect of providing computing power on-demand and computing resources outsourcing. If we observe the NAP paradigm in the compute server environments, it has its origin in the way storage evolved as a network-based resource, as in NAS. NAS offers a network wide storage resource availability and utilization and takes away the constraints of server attached storage. A similar concept of NAP, with respect to compute power, is fast catching up as a way to get the best out of the computing resources."

He added that NAP brought into focus the aspect of storage on demand and storage resources outsourcing. It allowed individual storage demands to be catered from a single common pool of storage based on a defined policy. The physical storage infrastructure could be modified without the applications and users utilizing that resource know about it or getting effected by the changes made.

To be continued...