Talks & Seminars
Design of Parallel File System on a Networked Intel Systems
K.Baswaraj, CSE Dept., IIT Bombay
Date & Time: August 23, 2002 04:30
Venue: IMPACT Lecture Theater
Design of parallel file system for N/W Intel workstation Cluster based parallel computing is of great interest.Clusters can be set up easily using commercially available hardware and they provide a cost-effective discretion to high-priced supercomputers. In a cluster environment, parallel programs are typically written using MPI and one node is used as the I/O node. A file is read at the I/O node and the data is distributed to other nodes using a MPI broadcast. To write to a file, the data is gathered at the I/O node from other nodes. The I/O operations unfriendly affect the performance in two ways. First, the I/O node becomes a bottleneck. Second, a communication is required along with the I/O operation. Many scientific and multimedia applications are I/O profound. Overall, the I/O presents a performance problem that needs to be addressed to take better advantage of cluster based parallel computing.A high performance I/O capability for clusters needs to address the following four main requirements. 1. It must provide high I/O bandwidth to ensure that the application does not become I/O bound. 2. It should provide an interface to simplify the parallel programming for I/O. 3. It must provide huge disk space to store very large files. 4. It should take advantage of aggregate memory pool of the cluster to minimize page swapping. Clusters provide a unique scenario to support high performance I/O. Typically, each computer in a cluster has a local disk. The disks together provide massive storage. Since each computer can access its local disk there is an opportunity to perform I/O operations in parallel. The parallel I/O can provide very high throughput. When I/O operations are distributed across computers, each computer can also use its local memory as an I/O buffer. The performance can be substantially improved by using multiple nodes as I/O nodes. There is an opportunity to use every node as an I/O node and achieve complete parallelism in I/O. Moreover, this type of parallel I/O capability scales with the number of machines in the cluster. This paper presents a parallel file system for Networked Intel workstation called PFSNIW for cluster environment. The Main constitutions of PFSNIW are 1. A file distribution & Gather scheme in which a single file can be decomposed and,distributed across the multiple computers & Gathered back. 2. The read/write scheme in which Part of distributed file can be read/modify. 3. Unix like commands to operate on Distributed file(s). (e.g. deletion, listing) 4. PFSNIW provides virtual storage system by treating the gross of disks as one large disk. It allows very large files well beyond the storage capacity of a single disk. 5. PFSNIW allows parallel I/O to access the same or different files. All nodes can perform disk operations in parallel to access a file. Each node can access the part of the file stored locally on its disk. 6. PFSNIW assumes the File permissions similar to Unix to maintain consistency of a file. decomposed part of a file belongs to a unique user considered the owner. Only the owner can modify its part of the file. Each node can access the data from the part of the file stored on its local disk. The I/O bottleneck is eliminated because the I/O operations (Read/write/modify) are done in parallel and not restricted to one node. If the files are decomposed then there must be transparency in accessing these files from different machines, which is perfectly handled by PFSNIW.
Speaker Profile:
List of Talks


Faculty CSE IT
Forgot Password
    [+] Sitemap     Feedback