Introduction: Data and Decisions - The Rising Cost of Useful Information

This morning after a late breakfast, I was strolling down Cannery Row in Monterey, California browsing the shops, and I happened into a modern art gallery. I am not a fan of modern art, I prefer classical styles if art must be visual, and I prefer dance and music to other types of art. In an effort to hit every shop I stopped in anyway. As I entered I found myself struck by a painting on the wall. There was something familiar about this painting, although I had never seen it before.

I kept browsing the gallery and came across another painting, similar and familiar, but I had never seen it before. This time I looked at the tag, and the artists name was Eyvind Earle. I walked back to other painting and sure enough, Eyvind Earle was the artist of that painting as well. After admiring both pieces of art, I continued through the gallery until I came across yet another painting that I really liked. Again, Eyvind Earle was the artist. These were all original paintings, and well outside my price range, but there was something very familiar about the art, a feeling of comfort and familiarity. I began to think, who is Evyind Earle?

I asked the gallery shop keeper who Evyind Earle is and it turned out, Evyind Earle is an artist who worked for Walt Disney, and he drew the background for several movies, including my youngest daughters favorite movie that I have seen 1,000 times (it seems) lately Sleeping Beauty. Was this the reason that of the hundreds of paintings and dozen or so gallery's I walked though this morning, I kept getting pulled back to paintings by Eyvind Earle? Is my unfortunate attachment, as a father of a 4-year-old princess lover, to the never-ending Sleeping Beauty DVD the subconscious reason why Eyvind Earle was 'familiar' to me?

The question I kept asking myself, as I attempted to drown out my thoughts at the Scheid Vineyards wine tasting store a few doors down was whether my subconscious knew the painting before I did, after all, in one of the early scenes of the Sleeping Beauty DVD we own, there is a short story about the background and making of the movie. I have seen that introduction before, and without seeing it again I'd place a $1000 right now Eyvind Earle's name is mentioned.

As I do before all long plane flights, I loaded several books onto my iPod for this trip just in case the stereotypical Manhattan mom is taking her kids on a trip. You know the type, the parents whose reliance on the nanny becomes obvious, because they have no connection or authority over their own young children, and forgot to pack the "essential tools of the traveling trade" that regular parents and grandparents get gold stars for having readily available.

(Tip for parents and grandparents, always bring Ring Suckers to help with the ear popping and lack of entertainment after 3 hours of flight, and you too can be a gold star parent or grandparent of kids 2-5 years old on a plane. I happily gave out 8 ring suckers yesterday to one such parental couple behind me on the plane, and between becoming the most popular person in my section of the flight and the iPod books, I enjoyed my proverbial 'nanny' gold star in peace on the 4+ hour flight from Chicago to San Jose.)

One of the books I added this time was Blink: The Power of Thinking Without Thinking by Malcolm Gladwell. Blink is a great book, and includes a very interesting analysis regarding Millennium Challenge 2002 btw, but covers two topics that I have been thinking about lately. The first is the subconscious reaction to information, and the other is rapid "thin-slicing" of data into useful information. Both subjects are beginning to get a lot of attention in the military lately, but should get more.

There is a steady drumbeat in military circles of the opinion that data is becoming more and more costly, and by that as we accumulate more, it is becoming less valuable because it is so expensive for the military to turn massive amounts of data into the useful chunks of information needed for an efficient decision process at all levels. I don't know if the problem indeed exists, but given the immaturity of the tools for data dissemination in the private sector, and known interoperability problems across military services with data, I imagine there is probably some truth to the issue. For 9/11 the data was there to prevent the attack, but dissemination of the data into useful information didn't exist.

But I have a feeling, and I note theory, that this is a generational problem and would resolve itself over time even if nothing was done proactively. My problem has been, I can't explain why I have this theory, what makes mass data dissemination generational and why do I believe future generations will have a better grasp on managing the problems? In Blink, I think I found part of the justification for my theory.

In describing the process of 'thin slicing' data in Blink, Malcolm Gladwell notes that thin slicing is skill, not art, and can be trained. Web 2.0 has created an internet data flow too massive for many to keep up, even organizing data via RSS and making information search capable doesn't resolve the issues with data dissemination, because dissemination of data requires a decision process that turns data into useful information. I believe that because Generation Y has grown up, evolved and trained in massive data flows they have developed natural skills for thin slicing data with available tools, be it cell phones, computer software, or simply how they how they follow streaming data. I liken it as similar to how code breakers in WWII were able to naturally, and with a high degree of success, follow specific units around Europe based on the communication habits of coders in the German Army.

I don't want to give the impression that thin slicing could ever replace research, we are talking about two very different types of information dissemination of large amounts of data, but I believe it will be the natural thin slicing skill of Generation Y that will demand, thus force the evolution of data dissemination tools to resolve some of the costing issues of data today.

This is just an primer on the topic, if I get a chance this week I want to open the discussion up and apply what I am trying to say to a specific example, which is the topic of Maritime Domain Awareness. For those who want to get ahead of the discussion, consider this July 2009 article in Proceedings by LT Mark Munson. There are a lot of great points made in this article I very much agree with. There are also some gaps in the argument, small but critical in my opinion. I think Lt Munson is both right and wrong at the same time, and I hope to tie in where thin slicing applies to my thoughts on that Maritime Domain Awareness article later this week, time permitting.