Wouldn't it be nice if business intelligence could generally distinguish common cause/market risk from skill components when assessing business performance?
I’m developing a return on investment calculator for data warehousing appliances, using the Forrester Total Economic Impact methodology
2010 has already delivered a list of top acquisitions; drivers include maturity, customer acquisition and cross-selling, but the confusion now falls to buyers
IBM has indicated their focus on using Initiate as a lever into healthcare and government industries and clearly will need to rationalize their technology portfolio
The rumors of IBM’s acquisition of MDM vendor Initiate Systems were confirmed in an IBM press release. Herewith, Jill pays homage to a class act.
You wouldn’t know it following this week’s activity, but there are more than two MDM vendors in this ever-changing market. What does this massive disruption mean?
With CIA profilers reportedly moonlighting to monitor earnings calls for "deception detection," speech analytics vendors are likely licking their chops for what comes next
The year is underway and the optimism about using technology for business has a new decade mentality
Performance measurement is a key driver of business intelligence. I'd guess that most BI deployments today are at least in part supporting PM
Now, Informatica can advance the use of MDM within many areas of their business from data governance, cloud computing and business-to-business processes
I usually root for the little guy. So when I heard the rumor that there were multiple bidders for Siperian, I figured SAP would take it home and that it really was a three-vendor world
Some pros and cons of implementing a vendor provided analytical logical data model at the start of any business intelligence or data warehousing initiative
Look out IBM, Oracle and SAP — you’re about to lose a bit of your dominance in the master data management market to Informatica
Although I do not see a stampede to replace traditional BI applications with SaaS alternatives in the near future, BI SaaS does have a few legitimate use cases even today
Just like iTunes, granular Web data services are just beginning to sprout into a new transactional data economy
The market for information applications is heating up and the demand is coming from line-of-business and vertical industries
After writing two blogs on the delights of the agile language Ruby, I decided to eat my own dog food and challenge myself to convert a significant piece of my portfolio returns simulation program from the statistical package R to Ruby
Let’s do a deep dive and discuss the current business and market environment and impact to IBM efforts in collaboration
Jill advocates behavior changes, starting at the top. But not for you. No, you’re fine. Just for other people
Today's blog embellishes on Ruby's suitability for basic data management tasks that precede business intelligence and analytics
This industry activity validates an automated approach to discovering and matching data as part of organization's information management processes
We need to be more pragmatic about technology predictions and focus on what we need to do to improve in iterative time cycles
Deconstruct your daily tech briefing, find your self-interest and use last week’s HP/Microsoft news as a learning experience
The intelligence value of any individual tweet in isolation is negligible. Then again, what’s the intrinsic value of a single attribute-value pair in a relational database?
I came across an article with the headline “Mining the Cloud.” The cynic in me immediately issued a silent scoff: How is that different from “crawling the Web”?
You never know what’s coming at you next, which is why process agility is so important
Maybe IT should establish a functional team that’s responsible for data packaging and distribution, just like the movie industry
Price-performance is everything in data warehousing (DW), and it’s become the leading battleground for competitive differentiation
Parents want to see their kids with a happy and successful career, but these days there's not a lot of useful advice
Dusting off a favorite open source tool, the programming language Ruby
The more complex business processes are the more difficult it is to find the magic blend that distinguishes a well-tuned process from a miserable flop
Let’s see if 2010 is the year of the workforce and if Peopleclick can help organizations truly invest into the most valuable asset
Before we shift our focus completely to 2010, it’s worth a last look back to consider all that was learned in 2009
Silver Creek is the only data quality vendor at this point that we feel has sufficiently addressed the unique and complex challenges associated with product data quality
Your customers are spending more time — and venting more of their true feelings — in social networks, so it only makes sense to move customer-facing operations into this brave new world
Social networks have always been with us, of course, but now they’ve gained concrete reality in the online fabric of modern life
Not that it will change overnight, but an annual budgeting cycle for IT spending might not stand the test of time
The core problem with today’s CEP offerings is that many of them are power tools, not solutions suitable for the mass business market
In data warehousing, the most likely casualty of the Teens will be the very notion of a data warehouse
This searching-planning compromise lens now colors my assessment of business strategy and intelligence
Weta Digital’s Paul Gunn runs a state of the art facility but doesn’t play by all the new rules
I expect this new category of technology to be like business intelligence in the 90s, where the range of vendors and opportunity will with significant demand
My writing retreat is completed for 2009. I'm not sure whether I'm comforted or humbled
The unique element of Lombardi was their ability to help organizations gain visibility through their process models that can easily measure activities and business flows across the enterprise
Everyone’s talking about single version of the truth, but how often are our reports reviewed for accuracy?
Amazon’s 12/14 launch of Spot Instances for EC2 is something we might want to mark on our calendar as a minor historic date in computing
I recently downloaded the latest stock portfolio returns from the website of Dartmouth professor Ken French
It is usual (and customary), during the holiday season, to give thanks for the blessing we have. I thought a short IT list would be sufficient
Advancements in business applications for performance management do not come very often in the industry but Actuate has one that we should all pay notice
Here are some quick thoughts on the trends that will shape advanced analytics in the year to come
If you're going to tackle a new center of excellence or competency center, at some point you're going to have to throw out the analyst playbook
About five years ago, a Newsweek review of the then new iPod raised a concern that the shuffle feature might not be random
SAP made it clear that they would like to eliminate the three letter acronym soup of the past and focus on the line of business needs by vertical industry
Don't sweat data governance, make it a measure of political maturity to challenge your peers and look for tools to help your people processes
From an enterprise perspective, social networks are the “buzz” that can spell the difference between success and failure in a reputation-driven online economy
Sales opportunities are continuously updated but the impact of those changes to the sales forecast and overall financial health of the organization is not well understood
Because a truly randomized tournament is both impractical and unattractive, officials must grapple with quasi-experimental methods and designs that attempt to control bias, all within a limited budget
Yes, Virginia, there is a Santa Claus and we are rapidly finding that this is getting the information into the hands of those who can use it
The main challenge and opportunity for Plateau is to demonstrate the value of their integrated suite of applications and convince, not just HR and training professionals, but finance and operations management the value of adopting their approach
When it comes to bad data, a lot of the problem stems from companies letting their developers off the hook
A technology vendor named Splunk has come to market with a solution to index volumes of log files and provide search and display capabilities along with a new generation of interactivity and application capabilities
Beyond simple delivery of alerts, URLs or actual reports via email here are a few newer approaches to deliver business intelligence on a mobile device
I like the concept of discovery-driven planning a lot. It seems a reasonable compromise between pure planning and searching
It's probably happenstance that statistical juggernaut SAS has been in the news several times over the last week following my recent blog. But like me, the press hasn't exactly been favorable, adding credence to the claim that SAS is now facing serious threats on several competitive fronts
Business is all about placing bets and knowing if the odds are in your favor
Salesforce steps into the business and social collaboration market where most would not expect
Building a service-oriented architecture implies the understanding of the business and the use of patterns - because SOA is really about architecting for reuse
The point here is whether you choose to build or buy your MDM solution, know what you need, and what you don’t need. I know that sounds obvious. But obviously, it’s not
Informatica 9 continues to evolve PowerCenter and PowerExchange data integration and access technologies into a much more comprehensive data management platform going well beyond the scope of traditional, batch-oriented ETL that remains Informatica’s bread and butter
Gaining better consistency and quality of information is what IT needs to be addressing more than consolidating BI as the underlying ingredient as your organization's most valuable asset
With my inbox groaning under overload, two themes have emerged: some will watch the establishment evolve (Goldman Sachs has picked the winners already); and others will simply go for it
Business intelligence practitioners can analyze, critique and debunk the works of popular management gurus till the cows come home, but the demand for expert wisdom grows unabated
A key critical point to "what differentiates one business intelligence software as a service vendor from another" discussion is what really constitutes multitenant architecture. Here are some initial thoughts to stimulate the discussion
I picked these 20 next generation business intelligence trends, not because the market labels them as trendy, but because I have tons of evidence that the buy-side o f the BI market can use these technologies today
Start looking at advanced features to select a data visualization vendor/solution
Informatica has advanced their efforts with support for business and IT collaboration, pervasive data quality and SOA-based data services
It might have been software or an act of providence, but it appears that Windows 7 saved my job
Kronos introduced the next generation of workforce management that will dramatically improve the usability of its new releases next year
You know that you don't have an enterprise BI strategy if ...
Statistical and predictive analytics software is fast becoming a next big competitive landscape in the business intelligence market. Heightened competition will be a boon for statistical consumers of BI
The SAP enterprise performance management and governance, risk and compliance solution is a refreshing approach that organizations should review if they have not
I downloaded a 30 day trial of the World Programming System and started constructing simple scripts. It was like seeing an ex after eight years of separation
Information on demand depends on a growing base of content assets and services and the cut-to-chase script looks increasingly agnostic
IBM announced a new version of IBM Mashup Center that will simplify the assembly and publishing of information across a workforce and across the Internet for consumers, constituents and customers
SOA is not a technology but an architectural paradigm and it is on the move again
IBM largest challenge with their software investments will be to continue to simplify them and connect with those in line of business and ensure they can get smarter solutions in the right time frame at the right price
At IBM Information on Demand in Las Vegas, the conference started today to introduce the latest insight to the fastest growing components of their business with analytics
The business expects high quality data, but hasn’t taken much accountability in delivering it
I revisited MITOpenCourseWare a few months ago in search of a gentle computer programming course I could recommend to those interested in BI without a strong technology background
With all the press releases emitted from OpenWorld, Larry Ellison’s $10 million IBM challenge stands out
Teradata announced their advancement into cloud computing called Teradata Enterprise Analytics Cloud, providing a range of options for organizations to use the database technology
Since 2004, when the Red Sox rallied from three games down to beat the Yankees for the American League championship and swept the Cardinals in the World Series, the Red Sox have been New England's academic darlings, “champions” of the data-driven, predictive analytics approach to running a baseball team
In this year’s keynote Ellison briefly discussed Linux and their efforts to drive neutrality in operating systems though they will potentially own Sun Solaris and open source version of Solaris by their acquisition of Sun Microsystems
The growing glut of Web services will increasingly flow not through browsers, but through application programming interfaces
When it comes to planning and budgeting, the gap between what’s possible and what companies actually do is still wide
At Oracle OpenWorld the focus on Oracle Complex Event Processing was quietly demonstrated in educational sessions how this dedicated technology is helping organizations
Better than the massive defeats by the San Francisco 49ers and Oakland Raider football games in the bay area on Sunday was the opening keynote at Oracle Open World. Silicon Valley legends Larry Ellison and Scott McNealy opened up the evening
The demand for new college grads is still soft. This cycle reminds me a lot of 2000, when the high-flying technology consulting career choice crashed and burned, no doubt mirroring the same fate of the Internet boom
To become truly future-focused, organizations must build out their predictive muscles through deepening commitment to these and other advanced analytics technologies
The information function in the organization is a network of the people who really know what’s going on; “They’re not organized or rewarded, yet they’re running the damn place”
It is not often that there are technologies that make you rethink your current methods of how you work and the tasks that you do constantly
For every operational system, a company can save hundreds or even thousands of hours every week in development and processing time
At this year’s HR Tech in Chicago a broad range of new software products for human resources professional to address their people-related processes across a range of activities were available for HR and associated IT professionals
The October 2009 Harvard Business Review spotlights Risk Management and Performance Measurement, both significant consumers of BI
It is not uncommon to expect with the downturn in the economy, that CIOs would be challenged to do more with less. IT executives are focusing on ensuring that business is conducted efficiently to get more mileage out of their budgets
As data and projects become collaborative, a metadata editor sounds like someone you’d like to have, and maybe wouldn't have to hire
Be forewarned that as you read analyst reports on GRC that they are a small component of what you will need to address a range of needs in the enterprise and across business and IT
The other day, I came across an article in the Wall Street Journal noting that movie rental company Netflix had announced a winner of the $1 million contest to improve the accuracy of its film recommendation engine
It is up to SuccessFactors to see if they can make something of this new category and focus in business execution and what other vendors might join them
The six words above are misleading, I’m really trying to get your attention to tell you about something about business intelligence, so c’mon, click it
There is a need to define a standard quickly for data interchange with regards to the electronic health record and the electronic medical record
The majority of data issues within an operational system are data entry-related. The challenge in business analysis is to establish standard business processes to automate
Having IBM seriously promote an analytics product line will help all statistics/mining vendors and provide a much-desired commercial jolt for academically-focused R
I am, slowly but surely, beginning to believe there couldn’t be a better case for demand for business intelligence software as a service -- especially after findings from a project that I am currently conducting
Experts say search, information extraction and text analytics can close the structured/unstructured divide; can it really be that easy?
The growing volumes of data from the Internet and enterprise placing pressure to gain better insights on a more frequent basis is an issue that continues to be for business and IT
I received a very nice note from Eric Siegel, PhD., Conference Chair, Predictive Analytics World last week in response to my recent blog that mentioned an upcoming PAW
It's an age old question: Which came first: the data or the process? Okay, not an age old question, but an interesting one to ponder nonetheless
Blue-sky challenges and benefits of health care data management will touch individuals and economies
Synygy's new set of capabilities will help organization address the sales and revenue performance priorities and drive new levels of operational efficiency in sales
This relationship long predates this announcement, but it could be the start of a relationship of considerable strategic importance to both partners
I've assembled my BI-Searchers toolkit
While Forrester does not have a formal description for a head of business intelligence, if I map requirements for BI best practices, here’s what I come up with
Logically this acquisition expands Informatica beyond data integration into event integration
As a consultant I’m often asked about how roles and responsibilities should be delegated or identified within the IT organization to support the data warehousing
It has been some time since I advanced the definition and focus of operational performance management over eight years ago
There's a randomness to company performance that, as deftly chronicled by Leonard Mlodinow in The Drunkard's Walk, is a much more powerful force than we'd like to believe
I believe the massive stimulus package signed by President Obama in early 2009 was a catalyst to a situation that was already fraught with issues – the integration of information in health care
Now as an entire business needs to operate on the Internet the responsibility for business and IT executive’s leadership and involvement is essential for success
In any dynamic business environment, the last thing you want is to indulge in navel-gazing. If you’re not adept at responding to breaking events or anticipating the future, you will find yourself marginalized in the new economy
We need to understand the variance between the data as it exists and its acceptability, not its perfection
The rapid development of technology and expansion of the Web provide the infrastructure for promoting ever-more-rapid innovation. But what's behind this technology enablement are two themes
Here’s a question to ponder: Why do so many metadata initiatives deliver well below expectations, or outright fail?
The unique aspects of eThority are the usability and interactivity, which makes their analytics more intuitive and relevant than most business intelligence products
Consumer packaged goods companies spend enormous of time and money to optimize their penetration of brands within their relevant product categories
The scary part for decision-making is that the errors are both deductive - from population to sample - and inductive, from sample to population
It’s important to realize that mastering data isn’t really necessary if you only have a single system that contains one copy of data
The business intelligence implications of the rumored SAP/TIBCO merger are huge
The technology for information applications will continue to evolve and become one of the fastest growing software categories
Most manufacturing and services organizations have driven their supply chain processes to be lean and are now looking at the means to find new methods to gain incremental efficiency
I read what is certain to be my favorite BI book of 2009: 'The Drunkard's Walk, How Randomness Rules Our Lives'
Cisco Systems aims at operational insight that looks at the customer experience across multiple transaction platforms
One of the more common phrases in corporate life is: 'Close the loop.' This mantra refers to completing an improvement cycle by putting into play the insights gained by analytical processes
Much as I love the behavioral economics gospel espoused in Dan Ariely, I'm uncomfortable with the driving methodology of controlled classroom or laboratory experiments that use MIT business graduate students as subjects
Polycom is hoping to use product information management to shorten sales cycles in a complex, regulated global marketplace
IBM’s bold move has already sent shockwaves throughout the analytics market
Whoever says that business intelligence market is commoditizing is smoking something funny
IBM now can empower new classes of analytic solutions with SPSS that go well beyond the traditional business intelligence applications that focus only on historical data
A discussion about the direction of business intelligence in several fields, including economics, statistical learning and the R platform
This acquisition will focus on two specific areas: high availability to support data replication and migration efforts and noninvasive real-time data integration capabilities
As business process and infrastructure outsourcing continues to trickle up in organizations, internal process owners could emerge to manage core business challenges
What really impacts our business/technology careers falls into a scope greater than the sum of our expertise and skills; for those not born to a single working destiny, follow the macro along with the micro
I think the planner/searcher dichotomy is quite pertinent for business intelligence
When I previously referred to architecture, I implied an engineered approach to solving a complex, multifaceted issue. One issue has to do with making complex decisions about inaccessible information
Actuate must take some credit for their work to date and be aggressive in what they can help organizations deploy easily with their existing development and technology skills
I’ve noticed lately that data warehouse vendors are dusting off the arguments and pitches of days gone by
Bet you didn’t realize that IDS Scheer has ARIS solutions in the fast growing markets of BI, analytics and complex event processing
Many business and IT professionals will attest to the fact that project management is the key to successful data implementations, although few data analysts will agree
Just what is behavioral economics?
A weak job market and poorly identified roles are making life dodgy for IT information workers; don’t expect human resources to help, find the competency center and align yourself to it
IT needs to fully engage in IT performance management and document their strategies and scenarios to determine the best effective alignment for business
Master Data Management and Data Governance have common goals
The complexity of IT systems has caused IT to become reactive rather than proactive
Do your existing CRM investments work well and, if you replace them, will they just be a legacy investment?
As information career paths disintegrate and reform, the convergence of business and technology finds people creating their own jobs and the 'smartest' among us may be the most challenged
What good are economists anyway?
Different business audiences require varying intelligence needs and focus
The recent acquisition of Sun by Oracle has raised a lot of speculative discussion about the latter vendor’s strategic pursuits
At MDM Summit in Toronto, random pearls of wisdom from Ed Unrau at Canadian Tire Financial Services
I periodically revisit Bogle's wisdom not only to affirm my investment approach but also as a guide for BI
The market for BI might seem rosy for most in the industry. While some analyst firms predict significant growth, most of us who have been in the industry for decades know a different reality
A corporation loses money every time it delays getting information into the hands of decision-makers
BI can learn a lot from education and other not-for-profit programs for its mission to assess the performance of business
As I have dug into improving customer satisfaction, the question I ask myself is that although this seems like a laudable target, is it really?
The short answer is yes and it's surprising the titan of search hasn't done so already
At the Information Builders 2009 Summit in Nashville music and technology came together to present innovation and practical use of BI and information-enabling technologies
I was pretty distraught over a column I recently read in the Economist. The article, Light Work, challenges the legacy of the Hawthorne effect
With official release of free reporting suite, the BI vendor offers a clever reverse consolidation path for customers
Forrester has developed a maturity model for enterprise adoption of mashup-style, self-service development of business intelligence applications
The dirty little secret in most companies is that the BI reporting team has morphed into a de-facto enterprise reporting team
Let’s start with my basic opinion: IT is not the business, it is an enabler
Quasi-experimental studies attempt to control potential confounding variables by clever design techniques and statistical adjustment
RFID-powered vending machines that let customers mix their own water, juices and sodas hoped to provide data for new product introductions
Behind a fairly generic name is SPSS's new version of their enterprise feedback management technology, which in my opinion is one of the key supporting technologies providing customer feedback management
After a day behind closed doors with executives I got a deeper appreciate for the portfolio of technologies today and in the future
I must admit I'm obsessed with BI designs for business performance measurement
Examples of transaction and performance management based systems to manage Eco Footprint are showing up in Europe; vendors will be ready to deliver products stateside as regulations call for
Virtualization is a venerable old computing concept that has achieved new life in recent years
The IT community has sensed a need for consistent terms. I propose we start a process to keep us all sane and volunteer to head the IEEE, ANSI or other committee
I came across an intriguing blog entry that raised issues pertinent to business and business intelligence
The tyranny of technology moves slowly and leaves the best bits behind for a longer run than people appreciate
The more this space matures, the more evident it becomes that analytics is a perfect match for column-based database architectures
This is yet another proof – and I’ll never get tired of saying this – that BI market is as vibrant, exciting and far from commoditization as ever
I find the intersection of the social and quantitative sciences of particular interest – and pertinence for BI
Molson Coors has been testing SAP’s Business Objects Explorer release and they really like it
Corporate restructuring isn’t just a financial challenge. It includes realignment of marketing activities, sales and operational issues
The current ascendance of Bayesian analysis in the statistical world is, I believe, a boon for BI
A service-based architecture requires information abstraction using metadata, whereas Web services do not use common metadata
Have a closer look at how consolidation and product maturity are paying off now for businesses with stalled IT budgets; you may already be a winner
The ball’s in IBM’s rivals’ courts regarding whether, when and how they plan to add automated source discovery to their BI portfolios
I received an email from Mike Driscoll, co-chair of the Bay Area R Users Group, announcing the formation of three new groups – Los Angeles, New York and Ottawa
Just another data quality failure from the annals of direct marketing, or am I missing the point?
We in IT struggle with understanding the business – or is it the business that struggles with explaining themselves (and their needs or requirements) to IT
Corporate travel budgets have been slashed, but what are the long-term effects of less opportunity
There was no shortage of strong reaction at the MySQL conference to the acquisition of Sun. Open source purists were incensed
What’s really going on with advances in solid-state disk drives and how far will it go?
Oracle is acquiring longtime partner Sun Microsystems, putting the software powerhouse fully into the hardware business - and hitting the DW industry like an earthquake
We’ve been talking for several years about the concept of a data supply chain. But IT executives are only now starting to catch on to its importance
There are few truly zero cost alternatives to BI tools, but there are some
The Elements of Statistical Learning: Data Mining, Inference and Prediction, Second Edition, by Trevor Hastie, Robert Tibshirani and Jerome Friedman is now available
Confusion about services-based architectures has been created by a number of industry elements
We need to consider enterprise architecture not as a hindrance, but as an asset to the IT organization and to the business that we are enabling
The R user community had just been provided access to a latest learning algorithm hot off the development presses from three world-renowned practitioners – for free
We as a country should try to smooth over these economic dislocations so that they don’t completely wipe some places off the map
On my way back home from www.himss.org show in Chicago, I have a creepy feeling of déjà vu. Even worse, it feels like the movie Groundhog Day where the main character keeps waking up on the same day, same date, never able to get to tomorrow
Why business rarely ties outcomes to job success for managers, and, if the New York Yankees are baseball’s AIG, why don’t we hate them?
I recently read an article in the Microsoft Architect Journal on so-called service-oriented business intelligence or, as the article’s authors call it, SoBI
No sweat I thought. We’ve honed our expertise on open source BI in the cloud. We’ve performed this drill several times already. Alas, not so fast ...
Bill Inmon treats data federation as mutually exclusive from enterprise data warehousing, when in fact they are highly complementary approaches
With hype contained, developments in cloud computing are on display at conference
A colleague recently asked me for a good introductory text on the R statistical computing platform. Though there are a seemingly endless number of published books on R, I recommended a personal favorite
Has everyone forgotten database development fundamentals?
Cynics might call Semantic Web a technology looking for a solution. And they might have a point
Lessons from the NCAA tournament coverage; don't drop the ball on Web content management
Nudge is a concept derived from behavioral economics. It denotes a gentle “push” or incentive to coax decision-makers to choose a preferred option from a series of alternatives
When the going gets tough, the tough get lean, focused, and flexible. To help organizations survive the bad times and thrive in all climates, their information management initiatives must remain agile and adaptable
I always predicted that Open Source BI has to reach critical mass before it becomes a viable alternative for large enterprise BI platforms
I’ve subscribed to the Harvard Business Review for years. It seems there are either several articles pertinent for BI or none at all. The February 2009 edition was the former
Reflecting on a conference, same as it ever was and still relevant
A couple of nights ago, I found my handwritten notes from 1997, summarizing a literature search I was doing at that time while authoring my first book, the IDG Books title “Workflow Strategies.” Reading the prettier handwriting of a 12-years-younger Jim Kobielus, I was struck by how little has changed since then
I recently talked to a client who was fixated on a hub-and-spoke solution to support his company’s analytical applications. In the world of software and data, the one thing I’ve learned is that there are no absolutes. And there’s no such thing as a universal architecture
The following are nuggets of wisdom from RMS for planning/executing modeling studies, along with a statistical blogger’s commentary
You’ll know content vendors at a glance with this clever subway map; also, Skittles turns Web site over to those crazy social networkers
I had an amazing client experience. I searched long and hard for a client with flawless, 100 percent efficient and effective BI environment and applications. Imagine my utter and absolute amazement when I stumbled on one
As I mentioned in last week’s blogs, I was pleasantly surprised by version one of Predictive Analytics World, finding it quite useful on a number of levels. Today, I offer a few final observations on the conference
Why we're bad at benchmarking BI failures
Steve Miller's thoughts his trip to Predictive Analytics World
Nassim Nicholas Taleb was right. The world financial system was devastated by unpredicted catastrophes of grand proportion – a financial black swan
Some integration approaches developed into multibillion dollar standalone markets. But others, while valuable, haven't survived as well. EII is one example
No matter how carefully one words a report, there is always the potential for misunderstanding
Everyone thinks that SOA is an integration framework. In fact it’s a means of remotely accessing other systems and their related information without having to know the details
Goes around, comes around; back office meets its match in tech-savvy users
As with any other current buzzword, the world seems to be piling on and the meaning of operational BI seems to be is evolving (or eroding)
FREE email newsletters 
7 Comments
Thanks for an insightful column! Tension will always exist between the desire for high data quality, the need to handle the inevitable defects that occur when humans are involved, and the need to complete a business transaction / unit of work in a timely manner. The point of initial entry is of course the best place to apply the business rules; an intelligent interface might also include workflow, heuristics or meta-rules that help decide in near-real-time when it's better to let a messy record pass (when it's on the critical path to closing a deal, for example), thereby reducing latency.
Posted by: Dean G | December 9, 2009 4:14 PM
Report this Comment
One approach I have used in the past with relative success is to use a Rules Engine like I-Log to gain better control at the same time customisable validation to the operational system
Posted by: Muralidharan G | September 1, 2009 4:12 AM
Report this Comment
It is best to manage data quality at the source and tools are available that can help; especially when looking to improve address quality. However it is also critical to make improvements in the data capture processes at the same time. By reducing the number of data source for a specific set of data, you are then able to focus on a identifying and developing a single/consistent set of rules that help you govern the data at the source.
Posted by: Frank G | August 19, 2009 9:10 AM
Report this Comment
Great post, Jim! One caution/caveat: keep in mind that the business rules/checking along the way probably aren't perfect, and you don't want a hyper-focus on data quality to get in the way of completing the initial transaction. It's important to always include a user-driven "override" option to bypass the in-line data quality check. Who hasn't been to a web site where they type in some piece of information about themselves that is honest and accurate...but the validation on the form keeps kicking back, "Sorry, that's not valid. Please fix X." When "X" is already correct!
On the one hand, having an override capability provides a loophole to enable users to enter "bad" data into the system. But that assumes that users are maliciously striving to generate bad data. Most users *want* to create good data, and putting in quality checks to check for fat-fingering and mental lapses is something they appreciate. The overrides won't see rampant use in most cases. And, if implemented such that the data steward can see who set the overrides and when, the stewards have a much more targeted, useful set of data to work from in their root cause analysis.
Posted by: Tim Wilson | August 19, 2009 7:16 AM
Report this Comment
I agree with you, if software developers can look into customizable data quality rules for input items then this would be achieavable to a greater extent. Some validation rule framework standard can be developed to be used by the developers. This also will help in B2B applications
Posted by: Fakhrudin J | August 19, 2009 1:06 AM
Report this Comment
I agree with you, a process for data quality at input would be very beneficial to organisations. Implementing customizable validation rule sets by software developers would make the quality process quite an asset.
Posted by: Fakhrudin J | August 19, 2009 1:01 AM
Report this Comment
Hi Jim;
I think we're going to see alot more of this proactive information management...this is exactly how our brains work: compare the incoming data points to an existing store and either create equivalencies to existing and update or add as necessary.
It's leaner, cheaper, works in multiple languages and disciplines and reduces the amount of sludge we have to pack around.
John O'
Posted by: John O | August 18, 2009 4:33 PM
Report this Comment
Add Your Comments...