The real-time data processing conveyor benefits business management
The articles “Real-time data processing is the way around the processable problem” and “Doing better real-time calculations in the data warehouse”, that appeared in Information Management, describe respectively real-time data processing as the way to eliminate big processable data problem and incremental functions as the technique of building real-time calculation of integrated data.
The combination of these approaches makes it possible to build a data processing system, which can work in real time for processing of source data and transforming it to the information required by the business. The advantages of such a data processing system, which works as a real-time data processing conveyor, are described in this article.
The functional structure of the data processing system, used by business management, consists of different functional steps, for example, data transferring; data transformation (change of the form of presentation); integration (verification and/or joining); and calculation (building the information, required by business).
Each of the steps is implemented as an operational node, for example, a program or a software package (job). Such operational nodes are combined to implement an IT process, which, after performing its operations, builds the required by business management information. The entire combined set of the nodes acts as a data processing conveyor.
The place of the data processing conveyor in the process of business management is shown in the diagram below:
The diagram showed that the return on investment depends on efficiency of the process of managing business, which, in turn, depends on the efficiency of the data processing conveyor. Obviously, the faster the data processed, the faster return on investment, which benefits the business. From this dependence, it follows that the business will get the greatest benefits when the runtime of the data conveyor tends to zero, which is a real time.
Definitely, when the business process does not happen so quickly, the effect of the real time of the data conveyor work looks small. But, in real-time operation of the data conveyor, firstly, it does not increase the length of the business process, and, secondly, it stimulates the acceleration of the process of managing business.
Using the real-time data processing conveyor allows the business management use in real time not only the final information, but also intermediate information, since all nodes of the real-time data processing conveyor work in real time.
For example, when the source Information processed initially by OLTP system, then – by data warehouse, after that – by data mart, and eventually, by OLAP system, different business users can use real-time information from each of these nodes (OLTP, data warehouse, data mart, and OLAP), and information from each node will be synchronized with each other. It supports the coordination of different parts (teams) of the business management and increases effectiveness of the business management work.