SAND Searchable Archive
REVIEWER: Young-Mun Cho, e-banking division manager for National Agricultural Cooperation Federation (NACF), South Korea.BACKGROUND: A national financial services institution established in 1961, NACF currently has more than 4,000 branches throughout South Korea. It operates the largest electronic network system in the country, offering telebanking, e-banking, mobile/PDA banking and a variety of automated services.
PLATFORMS: IBM p670 server with dual 1.1 GHz CPUs and GB of RAM; IBM AIX 5.2.
PROBLEM SOLVED: NACF processes a very large volume of e-banking transactions daily, and this is increasing steadily as our customers become more comfortable with Internet banking. Currently, these transactions produce at least 12GB of log data per day, and as much as twice that at month's end. All this data must be kept available for two months so that call center staff can access it if necessary to respond to customer support requests. Previously, when older data was needed, the relevant data set had to be restored from a backup cartridge into the source system, where a simple text search was used to locate the desired information. This was an unacceptably labor-intensive and time-consuming process that normally took two to three days. However, it was not practical to keep such large amounts of historical data available in our main data warehouse system: not only would it require a very large amount of disk space and negatively affect warehouse performance, but the loading and indexing times involved would be prohibitively long. A solution based on SAND Searchable Archive is currently used to store two months of transaction log data - nearly a terabyte - in a very compact (approximately 50GB) archive that can be directly searched via a simple application using standard SQL. Results are now returned in minutes rather than days. The archive process is fully automated and accommodates changes to archived objects (i.e., addition or removal of fields in the logs) over time with minimal administrative effort.
PRODUCT FUNCTIONALITY: SAND Searchable Archive has enabled implementation of a powerful information life cycle management solution for managing inactive data that we need to retain for access over the long term. It provides an impressive compression rate of 85-95% for transaction log data, along with the ability to directly query the compressed data sets using standard SQL. If required, the compressed data can be quickly restored back into the source DBMS. In addition, because the system is highly scalable, we are well prepared for future growth of the data being stored as usage increases. This allows us to maintain only our frequently accessed data in the warehouse, where we can concentrate on providing fast query response for analytic users. In general, we anticipate the following benefits from the system: 1) savings on system extension costs, 2) savings on disk costs, 3) savings on application development costs (because all log types can be queried using SQL) and 4) reduced operations overhead (because of simplified administration and flexible architecture).
STRENGTHS: Product strengths include high compression rates, the ability to query data directly without decompression and the ability to change structure of the source data (add and drop columns) without requiring restructuring of the archive.
WEAKNESSES: The management tools provided with the product use a character-based interface and are not as user-friendly as we would like. We have been told that graphical management utilities are under development.
SELECTION CRITERIA: The primary criteria for selecting the solution were scalability, storage economy, simplicity of administration, adaptability to meet changing requirements and ease of use.
DELIVERABLES: We have a highly compressed archive for storage of e-banking transaction log data, which can be queried directly.
VENDOR SUPPORT: The solution was implemented by TopEnd Technology, a leading systems integrator in South Korea and a SAND Technology partner. Throughout the implementation, TopEnd provided excellent consulting services and technical support, backed by very responsive product support from SAND Technology.
DOCUMENTATION: SAND Technology's documentation is thorough, well-organized and easy to understand.
SAND Searchable Archive
SAND Technology Inc.
Canada H3Z 3L5
Register or login for access to this item and much more
All Information Management content is archived after seven days.
Community members receive:
- All recent and archived articles
- Conference offers and updates
- A full menu of enewsletter options
- Web seminars, white papers, ebooks
Already have an account? Log In
Don't have an account? Register for Free Unlimited Access