Much has been written, and continues to be written, about the explosion in data, both in retail specifically, and in the world in general. I remember when a terabyte was considered a huge amount of data, but these days, it is a mere drop in the bucket. Dramatic advances in hardware, software and communications technologies have enabled the management of data sets of a size which only a few years ago were considered out of the realm of the possible.

But I am terribly afraid that the technology associated with building and managing databases has advanced much more quickly and effectively than the ability to analyze and understand these databases. That is to say, we are all amazed at how much data we can collect and manage, but we have not done the job we need to do to understand all of the new data we are collecting – or even to simply understand the data we already have.

There is an old saying that using and understanding data goes through a four-step process. First, you start with data – raw numbers. Then, that data is then reformatted into information – data assembled into tables which allow rational analysis. This information is then analyzed and sorted to create knowledge – interesting insights derived from the study of patterns and correlations within the information. And finally, this knowledge is parsed and understood, and turned into actionable changes to the status quo – what we think of as intelligence.

While we have recently become so good at the data and information parts of this process, I believe we’ve done a less-than-adequate job of developing knowledge and wisdom from this data. In fact, what I see more and more in the marketplace is that data users find the analysis and understanding aspects of their work to be very hard – so they conclude that they need more data, rather than performing a stronger analysis of the data they have already collected.

In a world where we want instant gratification, data analysis is genuinely tedious work. That reality, coupled with the on-going process of data sets growing larger and richer, has led many to say that the “current” data is not so useful, and will be supplanted shortly by better data. But I think that this is a major error. The reality is that there is now a huge amount of very beneficial and useful data. The issue is that we don’t feel comfortable or capable of analyzing all of it, so we wait for something easier to come along.

So what I am urging all of us to do -- from data vendors to retailers to manufacturers -- is to spend less time working on building the better mousetrap and more time making the current mousetraps catch mice more effectively. Invest more time and effort on your existing data, and the returns will be much greater.