NetApp Bolsters Backbone To Boost Intelligent Data Infrastructure

Intelligence is proliferating. As the oft-referenced rise of artificial intelligence and machine learning penetrates every tier of the global IT stack, it will surface in both enterprise and consumer applications at all levels. We’re now talking to our smartphones and smartwatches and AI is doing better than we had previously expected. We’re also injecting additional validation mechanisms to make AI smarter and more accurately aligned to specific use cases, especially inside enterprises that use a proprietary data set (which is pretty much all companies) with functions like retrieval augmented generation to sharpen up the much-hyped functions we can now gain from generative AI.

But below all those shiny treats, there is infrastructure… and that’s getting smarter too.

Intelligent Data Infrastructure

Specifically styling itself as an (or, indeed, the) intelligent data infrastructure company, NetApp has announced new work in its collaboration with industry partners designed to accelerate AI innovation. By providing the intelligent data infrastructure required to make generative AI work, NetApp insists it is helping organizations tap into what is obviously one of the most important developments for business and IT in the last decade.

Generative AI intelligence has of course made plenty of headlines, largely as a result of its human-like ability to work inside enterprise and consumer applications to perform functions such as summarizing large amounts of information, responding to human voice or text questions and its all-round ability to generate (the clue is in the name) content. But rather like all computing machines in general, AI is comparatively dumb until it is fed, instructed and subsequently watered for ongoing health.

Unified & Integrated Gen-AI Stack

NetApp CEO George Kurian has promised to deliver on what his firm has called an expansive approach for this era of data intelligence. Telling us that a large part of the AI challenge is a data challenge, Kurian has laid out a vision for how his firm will create and deliver an intelligent data infrastructure to ensure relevant data is secure, governed and always updated to feed a unified and integrated generative AI stack.

“When we talk about how firms work effectively with data today, a good data strategy is the first thing, deep domain knowledge is also core, the ability to test learn and adapt quickly is also central… and then, firms need to move all that information into what we could call a robust data ecosystem,” said Kurian. “But moving from siloed data into an era where unified data management can be achieved is a challenge for some companies, but many have done it well (especially in the regulated industries such as in life sciences and healthcare) and it is in these scenarios that AI can be most effectively applied to the data assets across an organization.”

What Is A Storage Operating System?

Many of the company’s product updates centralize around NetApp ONTAP, the organization’s operating system for unified storage. If our computer has an operating system (Windows, Linux, Chromium or other) and our smartphones also have an OS in the form of Android or Mac OSX, then it should be perfectly reasonable for enterprise technology departments to need a storage operating system, but what is it and what would it do?

A storage operating system could be described as a centralized and unified management tier dedicated to provisioning space for files, managing access privileges for data, initiating & performing backups, classifying data into “hot or cold” zones depending upon its frequency of usage and subsequently moving data to the most cost-effective location depending on its need, managing processes within the storage tier such as memory allocation and copying file and directory locations to serve the software applications that need them.

In terms of product updates (by which we generally mean enterprise software services, which are packaged as products within the organization’s platform portfolio), NetApp has begun the Nvidia certification process of NetApp ONTAP storage on the AFF A90 platform with Nvidia DGX SuperPOD AI infrastructure, which will enable organizations to use leading data management capabilities for their largest AI projects.

There is also the creation of a global metadata namespace to explore and manage data in a secure and compliant fashion across a customers’ hybrid multi-cloud estate to enable feature extraction and data classification for AI.

“We are also now offering a directly integrated AI data pipeline, allowing ONTAP to make unstructured data ready for AI automatically and iteratively by capturing incremental changes to the customer data set, performing policy-driven data classification and anonymization, generating highly compressible vector embeddings and storing them in a vector DB integrated with the ONTAP data model, ready for high scale, low latency semantic searches and retrieval augmented generation inferencing,” noted NetApp, in a technical product statement.

Innovation With Information

“Organizations of all sizes are experimenting with GenAI to increase efficiency and accelerate innovation,” said Krish Vitaldevara, SVP for platform at NetApp. “NetApp empowers organizations to harness the full potential of generative AI to drive innovation and create value across diverse industry applications. By providing secure, scalable and high-performance intelligent data infrastructure that integrates with other industry-leading platforms, NetApp helps customers overcome barriers to implementing GenAI. Using these solutions, businesses will be able to more quickly and efficiently apply their data to GenAI applications and outmanoeuvre competitors.”

A disaggregated storage architecture here enables full sharing of the storage backend, which maximizes utilization of network and flash speeds and lowers infrastructure cost, improving performance while economizing rack space and power for very high-scale, compute-intensive AI workloads like LLM training. This architecture will be an integral part of NetApp ONTAP, so it will get the benefit of a disaggregated storage architecture but still maintain ONTAP’s resiliency, data management, security and governance features.

Across all its native cloud services, NetApp says it is working to provide an integrated and centralized data platform to ingest, discover and catalog data. NetApp is also integrating its cloud services with data warehouses and developing data processing services to visualize, prepare and transform data. The prepared datasets can then be securely shared and used with the cloud providers’ AI and machine learning services, including third-party solutions. NetApp will also announce a planned integration that allows customers to use Google Cloud NetApp Volumes as a data store for BigQuery and Vertex AI.

Lessons From Babylonia

CEO Kurian used a speech this year to talk about NetApp’s efforts to build intelligent services into its data platform, so that, really, the company that was once thought of as a data storage company, can now validate itself as an intelligent data services organization.

“Think back to how far we have come,” said Kurian, urging us to remember a little history. “The way we use AI to track and analyze human behavior has its roots a long way back in pre-biblical times when the first census was carried out in ancient Babylonia. Fast forward to Europe in the 1700s when the first formalized modern-age census was conducted – we have been analyzing human location, movement and all wider behaviour for centuries. Now that we have the computational power of modern machines and the algorithmic power of modern artificial intelligence, the time is right to work with a powerful intelligent data infrastructure.”

Where data storage used to be, well, just storage. NetApp is building more functionality into this tier of the enterprise IT fabric so that it is not just stored, it is managed, governed, protected and optimized throughout an organization’s operational lifecycle.

Want to know what the next big thing will be in AI? Ask a Babylonian.