Long-requested forward-looking feature set bolsters metadata intensive Big Data capabilities
Beaverton, OR — July 10, 2013 — Open Scalable File Systems (OpenSFS), the premier non-profit organization supporting and coordinating the High Performance Computing (HPC) open source file system community, is announcing the general availability of Lustre® 2.4 file system. The new set of features extends the capabilities of Lustre significantly, signaling Lustre’s growing usage beyond traditional HPC environments.
As with previous releases, OpenSFS was deeply involved in funding major enhancements, sourcing and defining requirements, and supporting community vetting and testing activities.
“The Lustre 2.4 release includes both features that are long-requested and ones that further build the foundation as a key HPC and Big Data technology. OpenSFS is proud to continue to provide strong financial support for the Lustre file system,” said Galen Shipman, OpenSFS Chairman. “This new release will further enable Lustre to tackle a wide variety of challenges, most notably metadata intensive Big Data workloads. These workloads are increasingly prevalent across a variety of domains such as social media, oil and gas exploration, genomics, and traditional HPC, where tens, hundreds of thousands, or even millions of files must be processed concurrently. This is a big step forward for Lustre.”
Lustre is a parallel distributed file system for large-scale cluster computing utilized today by 7 out of 10 of the TOP10 supercomputing sites and over 60% of the TOP100. OpenSFS continues to play a pivotal role in Lustre roadmap discussions and feature releases. OpenSFS sponsors development of new features through the OpenSFS Technical Working Group (TWG) and coordinates community development and release discussions through the OpenSFS Community Development Working Group (CDWG). Moving beyond this current release, OpenSFS’s request for proposal (RFP) W4570, for example, will facilitate Lustre file system feature development, parallel file system tools development, and address Lustre technical debt and parallel file system incubators.
Lustre 2.4 Major Features
- Distributed Namespace (DNE) feature, also known as “Clustered Metadata,” providing horizontal scalability of metadata performance.
- Popular file system ZFS is now integrated easily as an option for a backing file system, allows easy scaling out of backend systems and improved manageability.
- LFSCK feature improved to allow scanning and verifying of internal consistencies.
- An intelligent Network Request Scheduler (NRS) providing improved application performance predictability.
- Improved support for high-data rate environments with large (4MB) RPCs.
- Client-side support updated for the latest Linux 3.6 kernel while still being fully interoperable with popular Lustre 1.8 clients.
A full list of Major Features is available at: http://goo.gl/pqjE8
Lustre 2.4 release is available for download immediately here: http://goo.gl/nF4Sr
Open Scalable File Systems is the premier participant-supported, vendor-neutral, and non-profit organization supporting and coordinating the High Performance Computing (HPC) open source file system community. We aggregate community resources as the center of collaborative activities. Our technical Working Groups bring in experts from many fields and backgrounds to contribute to roadmap discussions. OpenSFS-sponsored events create opportunities to network directly with the smartest open source file system engineers and business leaders. Our membership provides leverage for messaging and awareness. https://opensfs.org/
* Lustre is a registered trademark of Xyratex Technology Ltd.