Until recently, applying public data at scale required special knowledge, expert tools, and a significant investment. That’s changing. Fast.
Today, rich public datasets on everything from demographics to real-time weather to transit patterns are available for little or no cost. And unlike in years past when government agencies made data available in non-machine-readable formats like PDF or even paper (the horror!), today you can almost always get CSVs. More and more agencies are going ever further, creating APIs and data streams you can subscribe to.
Simultaneously, the tools for working with this data are also becoming much more accessible. Cloud data warehouses can crunch huge volumes of data for pennies, require no infrastructure, and can be spun up in seconds. Cloud providers such as Amazon Web Services and Google Cloud Platform have even partnered with governments to preload the data, further reducing the barriers to entry.
So what does it look like when businesses start using this data? Well if you’re in real estate or retail, the census provides fine-grained data about the demographics of every neighborhood in the country. As enterprises plan expansions into new territories, understanding where people who are similar to their current customers live can be the difference between success and failure.
At a broader scale, as companies are deciding where to place new offices, understanding the prevailing wage, education rates, job market slack and housing costs of different cities can give them a leg up in selecting an ideal office location. Think of it like Amazon’s search for HQ2, but for even the smallest of business.
With weather data, companies can make better decisions about inventory allocation, discounting, and promotions. Cold front coming to the Northeast? Time to promote long johns to visitors from Massachusetts. Summer arriving early in Seattle? Get the swimsuits on the homepage, stat!
Whenever a snowstorm is headed to New York, stores inevitably run out of ice melt and snow shovels. And that makes sense--their storage space is very limited so they can’t keep much backstock. But today’s weather predictions are accurate days in advance. There’s no reason that savvy retailers couldn’t move inventory around, adding pallets of snow shovels to their deliveries in the days leading up to the storm. With today’s tools, this reallocation could easily be automated.
On the transport side, huge datasets that effectively map the movement of cities are now commonplace. And while many are proprietary--owned by ride sharing companies, map app makers, and delivery companies--there are plenty that are available to anyone who’s interested. Whether it’s detailed taxi trip data from New York or Chicago, or traffic flow studies from Seattle or Austin, many cities now make it easy to map how their inhabitants move (and how quickly).
If you’re a business planning delivery routes in those cities, taking traffic patterns into account could dramatically increase the number of deliveries a driver can make in a shift. You could go even further and incorporate this data into your pricing algorithm for deliveries. Want your delivery at 6pm in a neighborhood that’s usually choked with rush-hour traffic? You’ll pay more than if the delivery can come at 9pm.
So who’s going to benefit the most from the radical change in how these public datasets can be leveraged? Perhaps not who you’d expect. The reality is, if you’re a very small business, in one or two locations, these datasets aren’t going to tell you anything you don’t already know.
Ask an experienced cabbie how to best get home in rush-hour traffic and she won’t even have to consult a traffic map. If you’re managing a couple of stores and looking for a third location nearby, your ground-level observations about the neighborhood are going to be more detailed and more current than any dataset you can download.
On the other end of the spectrum, the largest enterprises are already effectively utilizing these datasets. With their enormous resources and scale, these global companies weren’t deterred by the difficulty of accessing the data. Walmart and Target don’t blindly choose the next location for their stores, they deploy teams of analysts who analyze traffic access and demographics.
The companies who are going to see the biggest benefit from this explosion in accessible public data are the vast group of mid-sized companies. These firms have the need and use cases to benefit, but haven’t had the resources to grapple with the expense and hassle of getting access.
The new world of public data--where the data is fast, accessible and cheap (or free)--is revolutionizing the way these companies learn, grow and compete. And for their customers, it should lead to better experiences, prompter service and fewer empty shelves.
Register or login for access to this item and much more
All Information Management content is archived after seven days.
Community members receive:
- All recent and archived articles
- Conference offers and updates
- A full menu of enewsletter options
- Web seminars, white papers, ebooks
Already have an account? Log In
Don't have an account? Register for Free Unlimited Access