Data has been growing at a fast pace, and it has been impractical to keep all of them online or in transactional system.
Motivations for data archiving includes; Improving performance of primary storage or RDBMS system, Reduce storage cost and maintain historical data typically for audit purposes, or government mandated compliance. We can categorize data as Hot (To be required for daily operations in transactional system), Warm (valuable to the business, but is not essential for daily transactional systems) and Cold (may have very little business value, but may be required to retain for legal or auditing purposes) data.
To be competitive historical data is an essential tool for businesses to meet increasingly stringent regulatory requirements, manage risk and perform predictive analytics. Traditional practice of archiving old data on tape or on a separate RDBMS systems makes business analytics challenging, if not impossible.
The modern approach to data storage and historical data analytics utilizes technologies like big data Apache Hadoop clusters to enable partitioned access to historical data and eliminates many challenges with data archiving.
The modern approach gives you the option to reduce cost by keeping rarely used data on less expensive but higher density storage. Unlike traditional data archives the data is still accessible, applications can easily retrieve and process the archived data.
Once you decide on the modern archival platform to meet your archival storage needs, there are further challenges in implementing the modern archival system like moving data from source to archival system, organizing the data, incremental data loads, defining the archival frequency etc.
With hTRUNK platform the data can be archived almost from any source based on the required frequency. The platform provides a set of components to connect to any source system, enables quicker data organizing on Big Data system, maintains the metadata across the archival system to leverage the capabilities of the Apache Hadoop to the core.
Key Befefits of hTRUNK platform