Related Links

Web Seminars

"If I want to track a bad guy, I would look at all the things that can be tracked. I would track his credit cards, I would track when he swipes a badge at a parking lot, I'd track his EZ-Pass and I'd want some analytics that transform the relevant parts of the data into dots of locations and times."

That's one simple use case for intel data. Compared to a domain like business intelligence, military intelligence is murderously complex, with extremely diversified needs that call for one data infrastructure to support them all. This is what the Dataspace proposes to be.

DSGS-A is where she met her future research partner Eick, a globetrotting visualization expert brought in to apply geospatial mapping technology to the Army project. Almost immediately, he bought into the Dataspace's potential for tapping into "one big soup of stuff" for intel, a theoretical capability to track any amount of data in one big horizontal swipe without the need for a unique project model or schema. It could be selective, like an observatory pointed at a galaxy in a constellation.

The idea grew while Stover worked as technical lead in a project to build a secure data vault to let war fighters, from the lowest to highest security clearance, authenticate their credentials and see all the data they are entitled to and no more. More important, it would filter across many domains or types of information.

From these bread crumbs, Suzi Stover spent the next year on an idea to crack open the space between all the disconnected silos and models we create to describe data. Her idea for a new Data Description Framework and ULS would tear down the walls, decouple data from the models and storage that confine it, and turn all that into an abstraction, a hands-off way to dip into all of the data, semantics, and interfaces we have without distorting or losing bits of it in the process.

In doing so, it would expand the amount of data by a factor of at least four or five and the computing and storage needed to manage it. "So there I was, preaching these ideas to the Army and they basically paid my salary as I looked for a place to apply this," she recalls. "There were some pilot projects, and then all of a sudden, cloud computing shows up."

A lucky coincidence had arrived with massively parallel processing and inexpensive infrastructure that could be built or rented to allow enormous computing and storage to be summoned at will. Because of the cloud, ULS wouldn't have to excavate anything comparable to the 17-mile tunnel beneath Switzerland used to smash atoms at CERN.

The top brass took a meeting and liked Stover's ideas. "There was a lot of excitement and I briefed the Army's G-2, the top intelligence officer, a brilliant three-star general who understood our data problem. I had 15 minutes, and suddenly our tiny low-profile project was understood."

For the next nine months, 10 developers built the core of a Dataspace system using cloud computing and storage to "ingest" structured and unstructured data of any format into a unified data store. The cauldron was boiling, and work was ramping on the top layers for unified storage of data models and the disparate interfaces, the "lenses" through which users could "see" the data.

Then suddenly, in February, the Army "decided to go in a different direction." Our conversation goes silent for a moment before she resumes. "It was beautiful work on a beautiful system but the government civilians decided to take a more conservative, if not legacy, standard of ETL, system integration and data integration."

Pressed on the point, she defends the work done in a flash of resolve, the only fleeting exception to her usual didactic cheeriness.

"Look, there is a tremendous pressure to go with your history, your team and what they understand. A lot of the swirl at DCGS-A was the result of contortions over the language being used, and I tried to prevent that from happening. We had a very rigorous and well-defined language that was co-opted for another agenda."

The subtext would be clear to anyone around the business of government. In the defense industry, where big contractors carve out massive multiyear budgets of appropriations from Congress, there is pressure to "refresh" spending cycles and update familiar infrastructure.

From another view, the machinery is simply so big that it exists mainly to perpetuate itself. Old Town Alexandria is a tourist destination known for cobblestones and quaint brick courtyards, but the residential mix is dominated by professional and technical services and vested administration jobs in government and the military. It sits just downriver from the Pentagon, and like the rest of the beltway, it's peppered with lobbyists and influence that keep an eye on the status quo.

A small consultancy or nonprofit will likely come into this setting under the wing of a larger established contractor that will use it for its own purposes, big idea or not. That's not exactly what happened to Yoakum-Stover and Eick, and they are quick to praise the talent and support of senior military leadership. There are pockets within agencies right now, they say, where people see the vision and are eager to make it happen.

In any event, the wheels had turned. She stayed on as a consultant until July before disengaging entirely, but that was not the end of the story for Stover and Eick. They used the interruption to press their own "refresh" button and took the work back in their own hands. Both are talented software engineers who love writing code and seem to relish the tea-spiked 7 a.m. sessions that run Monday through Thursday for 12 hours or more.

"We went back to brass tacks, redesigned the data architecture and the whole system. Software developers will tend to make decisions for expediency and reuse what they already know. Conceptually, the fundamental ideas are the same but the implementation changed profoundly because everything changes when you start writing code. You see things you don't see otherwise."