BACKGROUND: InfoSpace is a worldwide leader in providing infrastructure services to wireless carriers, merchants and Web sites. Consumers view InfoSpace online promotions either on wireless devices or on Internet destination sites such as American Express. They then simply visit the online or physical storefront and make a purchase as they normally would, using their preregistered credit or offline debit card with no need to print coupons or present a copy of the offer. Consumers then receive promotions automatically on their card account. InfoSpace offers consumers, merchants and distribution partners an innovative, seamless method of enabling and measuring promotions for online, offline and hybrid merchants at national, regional and local levels. One of the many important values for all partners is InfoSpace's unique ability to close the loop in measuring the effectiveness of each promotion at multiple points in the promotion cycle. To enable this, the company has built a robust, high-availability, scalable and adaptable Internet solution.
PLATFORMS: CoSORT is installed on Sun UltraSPARC servers running Oracle7 and Solaris 7.
PROBLEM SOLVED: In order for InfoSpace to maintain a competitive advantage, we must be able to efficiently receive, sort and select large amounts of data coming from member banks that manage credit cards and the InfoSpace promotion transactions. The files we receive can contain millions of rows of raw transaction data. We have to sort and filter these massive files to find only those customers that have registered online for promotions with client merchants in order to provide reports and usage statistics to our clients. To solve the filtering issue, our data warehouse team looked at ways to increase the speed of selecting and sorting these huge data files outside of the database.
PRODUCT FUNCTIONALITY: CoSORT natively supports more than a hundred data and record types, and has several UIs including sortcl. The syntax of sortcl allows us to specify, join and remap differently formatted files from input to output, letting us reformat at will. In sortcl's input phase, record or file layouts are defined. The processing action can be sorting, merging, joining and/or reporting. In the output phase, selected records are further filtered and/or remapped to one or more output targets. Cross-calculations and multiple formats can be written to the same or different target(s).
STRENGTHS: By sorting and filtering data outside Oracle, files did not need to be custom loaded into the database, which eliminated a great deal of programming. CoSORT thus helped reduce or eliminate the development costs associated with an internal solution, and its join function is essential. CoSORT's ability to extract and transform files from multiple banks so efficiently is an enormous data warehousing advantage it has over other sort packages. CoSORT's ease of implementation also helped cut our initial setup time and costs.
WEAKNESSES: sortcl's initial join syntax (file field naming) restricted us to using left- right file names without extensions or absolute paths. A later version fixed this problem, however.
SELECTION CRITERIA: Our decision to purchase CoSORT was based on both performance and functionality. After evaluating many sort solutions on the market, we found that CoSORT was the fastest at sorting very large files critical to our daily operational time window and the only sort capable of performing an integrated (single pass) join to filter records. In our evaluation, we found that CoSORT was five times faster than the UNIX sort and two to three times faster than its major competitor. Such speed allows our entire network to run more efficiently.
DELIVERABLES: All CoSORT packages come with sortcl, sorti (an interactive sort/merge program), coroutine and subroutine API calls and drop-in replacements for the UNIX, MF COBOL, SAS and Natural sorts. The sortcl program can be called from the command line, in batch, from another program or even a new Java GUI launched remotely. Also included are tools for converting meta data sources such as COBOL FDs, Microsoft CSV and NCSA Web logs into sortcl data definition files. Several third-party applications also hook directly to CoSORT.
VENDOR SUPPORT: Any questions we have are answered promptly. Web support is also available.
DOCUMENTATION: Traditional UNIX manual pages accompany each deliverable, but the full Acroread manual is searchable and has more run-time examples and logic flow diagrams.
Antonio Casacuberta, senior director of software engineering at InfoSpace
Be the first to comment on this post using the section below.