Data Services and Virtualization

We are in the midst of a paradigm shift in how organizations view and address data usage and delivery of information. Technology has brought us a plethora of data - how companies define, secure and distribute this critical information is what sets them apart.

The term “data services” refers to turning data, and its corresponding metadata, into a service that is callable to the enterprise. Think of it as a service that could provide for CRUD (create, write, update and delete) for an individual piece of data (or even a unit of work – connect data elements).

Virtualization of data means that the consumer of the data does not know where the source (calling party in the case of a service) or location of the data is; it just requests it and gets it – much like a Web service is called not knowing where the address resolves to.

Place the two together and you end up with a very powerful combination information availability for the enterprise and the basic model for master data – the virtualization of key data within the enterprise so that it can be managed and governed.

Toward that point, the need to have data managed is the key to what we are discussing here. I have blogged previously about the need for data governance, with data services this is enabled and becomes easier to manage. This is because the source of the data is defined within the service and can be only be changed from this service point – the single point of entry into your data challenges.

Data virtualization best practices call for environment management, governance and policies, organization and skills, and change management (control of the changes to the data services).

The requirements for data services have come about primarily because of:

Data access:

• Need for speed
• Number of producers and consumers constantly changing
• Read only
• Logging
• Reporting

Data volumes:

• Logarithmic increase in data
• Machine generated data (events, RFID, GPS, camera’s, etc.)

Data types changing:

• 80 percent of new data growth is unstructured content
• Emails, images, audio, video, etc.

Dataset expiration:

• Some date/time in the future
• Based on check-in or heart-beat criteria
• Incorrect access attempts

In a Ventana Research survey of 102 executives, the 7 biggest development challenges had the top three as:

• Assuring high data quality, 47 percent
• Supporting highly complex conceptual data, 6 percent
• Supporting access to intra-day real-time data, 34 precent

With this survey in mind, plus the requirements mentioned above, you can see quickly that data services and virtualization are quickly becoming a “hot topic” in information management.

If you  are attending Enterprise Data World 2010 in San Francisco next week, you should attend the session hosted by Dr. Peter Aiken (President of DAMA International) and the author of “Data Services & Virtualization” on Sunday 3/14/2010 from 3:30PM to 6:45PM, where we will take the attendees through this topic extensively. Here’s a brief overview of the program:

This workshop will begin with a discussion of the history of data services, what are data services and how data virtualization is changing the way organizations are securing and delivering information to its constituents. We will examine the key drivers, definitions, requirements of and review successful as well as new strategies that show how to deploy and sustain a data services/virtualization plan in your organization. Areas to be covered include:

• The context for data services and virtualization - How did we get here?
  
• Definition of data services and virtualization
  
• Turning data into trusted information through delivery using data services
  
• Data availability and accessibility improved
  
• Best practices

Attendees will learn:

The background, definitions, components and key terminologies of data services, best practices and new trends in data services and data virtualization.

I will devote the next blog entry to responding to the comments that people have made and try to offer some idea’s for discussion.

Note that if you are on Linked-In, I have created a group entitled “IT Governance & Management” with the idea of ‘building a better, more informed world together’. The first task I have challenged the members (23 and growing) with is to build a definition for Wikipedia of what is an “elevator pitch” for IT governance and management”. Susan Early has already posted a first shot (and a good one at that) – join and pitch in.