A New Approach for Data Retrieval from Hierarchical File System to Handle Priority Satellite Data Products Dissemination

C Pradeep, ANSV Murali Krishna, G Prasad

Abstract: Satellite Data is being acquired by various Indian Remote sensing satellites and being widely used for various applications. Applications include resource management, disasters, crisis-management and relief support. These applications demand immediate and timely satellite data products support which is of paramount importance in order to make predictions, decision making during disasters like floods, Earthquake, landslides, forest fires, etc. and are considered as Priority products. These Priority products are to be supplied within an hour from the time the request is received from the order processing system. The entire process of ordering, processing the requested data imagery and disseminating the product to the requestor is entirely digitalized. The work flows and data processing is handled by a chain of software applications running on top of the respective IT infrastructure chain in the IMGEOS data center. SAN-large-scale data storage which is one of the major components of IT infrastructure is shared by different operational entities functioning in the organization. The resource sharing model implementation has an impact on processing and dissemination of Priority data products. One of the factors for the delay in data dissemination is translated as the delay in data retrieval from the underlying data storage systems viz., Disk and tape storage medium. This paper presents a customized software framework developed for the efficient data retrieval of the priority data sets from the wide range of archived data store. This is envisaged by lodging multiple files under single data request and enables special priority at the retrieval queue to reduce the request queue latency, which has helped in maintaining the overall Turn-around-Time (TAT) for the product generation and dissemination. Additionally, it improves retrieval requests queue length and improves the health of storage hardware.

Keywords: DM, IMGEOS, HSM, IT, LTO, LUN, LVM, MD, MSM, NRSC, OS, RF, SAN, TAT, TSM, WOF