Starting in the late 1980s, many software companies emerged with a new document management technology as part of a document management system. These systems were originally developed to manage documents primarily for businesses to keep track of electronic files and other important information that needed to be stored centrally for legal or other purposes. The initial focus for document management systems was on accounting/financial and legal data, because having a traceable record of the information and who accessed it is an important part of data management in an electronic world. As these DMSs matured, software companies introduced new capabilities and additional functionality that went beyond document storage and retrieval. These new capabilities included security/access, auditing, workflow, distribution and collaboration.

Over the past five years, driven by these additional capabilities, uses for a DMS have seen some dramatic changes. While we continue to see important documents and files getting stored and accessed in traditional document management systems, we now see a much wider spectrum of information and data being managed by these systems. This new data includes information that is resident in spreadsheets, reporting data, supply chain data and other operational data that needs to be shared across the business. This new use of the traditional DMS – for the storing, distribution and exchange of information – is triggering a paradigm shift in document management technology.

The fundamental methodology of a “file” or document-based management system has been used over the past 20 years and is core to all document management systems. This means that managing and saving a file, retrieving a file, creating workflow and collaboration works only when the file is stored in a central place, with access granted to various people to retrieve and review or make changes to the file. Once changes are made, the file is saved in the DMS and is then available for review, or for changes to be made when the next person needs access to that particular document.

The change in document management technology is the move from a file or document-based management system to management of information resident within these documents at the data level. This is a fundamental shift in the traditional thinking and methods of how to deal with security, auditing, workflow, distribution and collaboration. The reason document management technology can no longer work strictly at the file or document level and must work at the data level is that users are pushing the limits of the existing technology/systems and asking the current methodology to handle something it was never built to do.

What does it mean when we say that today’s DMS works only at the file or document level? Today’s technology is fundamentally asymmetrical. A document or file is a static object. In the current world of document management systems, someone creates a document or file and stores it on a server. If allowed, others can view the document, and in some cases check out the document, make changes to it, and check the document back in. Much like a library book, when a document or file is checked out, it means that no one else can access that document or file until it is checked back in. This is called file or document locking. This file locking strategy has worked well when the requirement was that a limited number of people needed access to the file. It also worked well as a method of saving data for future review, or for an audit. However, when multiple people want to view or make changes to a file or document at the same time, it creates locking because traditional systems are based on an asymmetrical model.

In today’s constantly changing technology environment, users are demanding and driving change at a rapid pace. Software companies need to listen to their customers and either change or get out of the way. In the case of document management technology, we are in the beginning stages of a broad transformation. Users are pushing the limits of existing systems beyond an asymmetrical “check-in/check-out” model.

The overwhelming sentiment from users regarding current document management systems is that they can’t wait for a document to be checked back in so that the next person can review the document, make changes to it and get their work done in a timely manner. This problem is exacerbated when you not only have many people that need access to the document at the same time, but also when you have data resident within a document that has a time sensitivity regarding distribution of the information. An excellent example is a price list. Imagine a company has 10,000 products. The product pricing is changing all the time based on available inventory levels, customer demand, competition and promotions, etc. If the marketing department is using a DMS to manage and make changes to the price list for distribution to the sales team, using the “check-out, check-in” model is difficult at best. In many cases, a user will simply make a new copy of the price list from the DMS, losing the connection to the master data. Later, if a sales representative is looking for the most current pricing, he or she can’t easily see which version has the current pricing, and a sale could be lost. Worse, if these changes are not readily accessible to the sales team, the price quoted could be outdated, causing the company to lose money or not get the margin expected from the sale of that product.

The other major issue driving the need for change in document management systems goes way beyond issues related to check-in and check-out. It has to do with a shift that has occurred regarding the type of data being stored and managed by these systems. We are now seeing not only documents being stored and accessed, but important transactional data/information, using spreadsheets as the container for the data. Data resident in a spreadsheet is very different than data resident in a traditional word processing or presentation document. For the most part, the structure of a document and the data inside of it is static, for example, a monthly financial report. Data and formulas within a spreadsheet are typically dynamic and constantly changing. The other interesting thing about transactional data is the dependency on the accuracy of the data down to the cell level. A spreadsheet functions, for the most part, as a container for information/data where the data and results are driven from formulae based on preset business logic used as part of the process. If one part of the data changes, it can change other values within the spreadsheet or change the entire spreadsheet itself. Static data (documents) and the management of the data within a document can work in a traditional DMS; however, dynamic data (spreadsheets) do not function well, if at all, in a traditional document management environment.

To properly manage dynamic data when multiple people or systems are making changes to it, you must have a system with the ability to handle multiple people or systems working on data simultaneously, along with a mechanism for tracking changes down to the cell level. The only way to do this is to have a DMS that is much more sophisticated than a traditional document management application or system.

New database systems are starting to emerge that can handle management of dynamic data in a shared environment with multiple users. These systems include the traditional document management level features of security, auditing, workflow and distribution at the file level. These new systems also can enable a secure two-way exchange of data between users working in desktop spreadsheets and a server that allows people to work offline in an access-controlled set of data. When a person is ready to exchange data and formula changes with other users, they simply click a submit/refresh button in the spreadsheet template, and the data is exchanged much like the sync process with email systems. The server then provides the distribution of these changes. The next time the process participant(s) submit and refresh, these changes are brought into their local version of the shared range of data. The server also can provide an audit trail of all changes (down to the cell level) and an automatic, multilevel roll up of data. All of this is presented within the spreadsheet on the desktop, but managed within a new database environment designed specifically to work with tabular data.

Another big advantage of this new kind of system is the ability to integrate information from transactional systems of record into the process. As these processes become more sophisticated, the need to include transactional data from multiple systems is a huge requirement. Transactional data management and the value of the data relies heavily on having all of the data in the same place at the same time, allowing users to collaborate and make the appropriate decisions. For example, the inventory level of a particular item has changed, and the only place this data resides is in an existing inventory management system. Appropriate decisions cannot be made unless the participants have access to this data. If the data is not integrated into the process automatically, the participants are working with only part of the data and will not be able to make an informed, timely decision. Therefore, transactional data management systems must be connected to system of record-level data – at the data level – in order for the process participants to have up-to-date and meaningful data.

These new database-driven systems are providing much needed improvement in the area of process change management. Since the data is dynamic and ever-changing, the process and participants are changing as well. The need to have a flexible model that provides users with the ability to change the data structure, process participants and workflow only can be done in a flexible database-driven environment. The new database environments make it easy for business users to make changes, such as adding new columns or rows of data, and having the structural changes show up on process user’s desktop. These types of changes need to be made by the information worker and not the DBA.

As users push the limits of the capabilities of traditional document management systems, new technology is emerging. Management of static information in documents is a very different process than managing dynamic data in a transactional environment. Users need to stop “shoehorning” processes that are transactional and dynamic in nature into a static asymmetrical model where the systems do not support the requirements of the process and the users. The paradigm shift in document management is upon us.

Register or login for access to this item and much more

All Information Management content is archived after seven days.

Community members receive:
  • All recent and archived articles
  • Conference offers and updates
  • A full menu of enewsletter options
  • Web seminars, white papers, ebooks

Don't have an account? Register for Free Unlimited Access