Data Science: Simplicity from Complexity

It's not what you look at that matters, it's what you see."

Henry David Thoreau

Journal entry, August 5, 1851

Thoreau insisted that we “simplify, simplify” and the phrase has become nearly a worn-out rag of a cliché. A pity, because its timelessness is its genius.

A hundred-seventy years after Walden was penned, we wonder: are his words anachronistic and unfulfilled or, rather, do we march to a different drum of simplicity?

The data scientist holds the key.

As Data Analytics leader, steeped in strategy or driving the operation, your discovery of customers' patterns, behaviors and needs is your advantage. The discipline of data analytics is not optimal without preparation and partnership, engagement of the business. But do not engage the business ad hoc. Build a culture of collaboration with your data scientists and enterprise leadership, speaking the same strategic language. Then the transformation from data complexity to simplicity is invaluable.

From 1850's "Walden-era" railroads to 20th century subways, and now with driverless automobiles taking us to airports, simplicity was elusive. Our transactions traverse cyberspace and microwave towers; unseen by the naked eye, they are instead felt in neurons of mental stress. Simplicity is perhaps an empty dream, a two-week per year respite, a quaint wish. But recognize Jean-Baptiste Alphonse Karr's famous words, “The more things change, the more they stay the same.” Tools and computations are faster and more sophisticated, but the desire for simplification is a romantic call in the wild. Digital designers, data decipherers, and executives all seek inspiration; to achieve it we face volumes of data that would never fit in the Library of Congress.

We need those volumes to etch a simplified but hidden story.

For a data scientist and the world of data analytics, the tags, identifiers, graphical points were once foundational for problem solving, association and forecasting. We weren’t grappling with just a “little” data, but at least it was manageable. In our current state, we no longer have a campfire size to analyze. We’ve got forests. The talent of data scientists lies in their ability to embrace the firm’s strategy, collaborate over the firm’s use cases and, like archaeologists, dig among the ruins of zettabytes.

The skills of the data analytics team are thoroughness and speed. But speed is borne of optimal collaboration, inside the firm.

We often think of speed in terms of the computational access and its offspring of dashboards. But the ability to electronically access data is not a competitive differentiator. Software query tools and applications are readily available. Commoditization of compute power is less expensive by the day.

The speed that differentiates firms is not achieved by just “getting data” but, rather, getting the 'right data for the right purpose.' And this happens through enterprise-wide partnerships.

Your data analytics function must be fluent in multiple arenas -- technical and strategic. It is not enough to hold ad hoc query/requirements sessions with your internal customers. Gathering of requirements should be expeditious, thanks to the documented use cases of the firm, the strategy of the enterprise and the understanding of the competitive landscape. Data scientists should be versed in these competencies ahead of time.

The strategic arm of the business and the operational leadership should have already socialized and standardized the flow of the firm, and the engagement of the consumer. Data scientists are expected to know this, as well as they know the distributed clustering practices of Hadoop. Do not sit and wait for dashboard requests, or customer activity requests before you create your portfolio of data intelligence. The firm's data analytics function should already have a base set of use cases, which can then be refined in terms of dates, regions, cause and effect...

Is the data scientist coached and engaged by the Marketing team?

Does that occur regularly?

Are the Sales leads collaborating with the Data Analytics leaders, discussing the prospecting challenges, what the potential clients are seeking? More importantly, is this happening regularly?

In negotiating a project with the data scientists, are internal customers using a baseline set of customer journeys? Not ad hoc but, rather, standardized and shared with all stakeholders in the corporation?

These are foundational expectations of the mature company, and the mature practice of data analytics. It is a practice, not a function.

Strategists and marketers should not be sitting empty-handed as they brainstorm with data scientists toward the answers they seek. The collaborating team should be leveraging artifacts that are proactive, available and constantly shared.

Your Data Analytics library should include a documented, illustrated, yet evolving set of customer journeys, competitor trends, and industry innovation. These are prerequisites to best-in-class data analytics capabilities.

The journey of big data is a trek of analytical sophistication. Its applicability is a quest for explanation under a flood of information. Whether you’re leading a team of 50 or you are a team of 1, the data scientist requires one foot in the technology realm and the other on the environmental landscape.

Data scientists are multi-faceted translators whose fluency is expected in four distinct skill sets.

(1) Fluency in the Technology

Computational analysis, usage of tools, and the integration capabilities of your analytical platforms should follow a sensible and scalable architectural model. The scalability is not simply in terms of carrying capacity and performance growth, but in terms of its evolving interconnectivity with external data sources, secure but robust APIs, and drag-and-drop UI’s.

Cultivating insights by way of diverse data, and culminating in the beauty of quantitative insight, pattern recognition, trend analysis, and evolution toward AI are staples in the tools. More than “looking at” data, the principal engineers must understand what they “see.”

(2) Fluency in Use Cases

The Data Analytics power-user is focused inwardly and outwardly. From customer analysis, feedback loops of discovery as well as internal, operational insights, data scientists should not be starting with a blank sheet of paper, awaiting a laundry list of client requirements every time a knock at the door happens.

The constituents inside the enterprise might seek insights on internal operations and improvement.

Executives throughout the value chain might seek corporate and competitive intelligence.

Sales professionals might seek customer behaviors. Therefore...

All analysis queries should be grounded in a baseline (but evolving) set of use cases, customer journeys, taxonomical documentation on impacting factors. The data scientist is partner in work flow. And the internal library will grow.

(3) Fluency in the Corporate Strategy

Henry David Thoreau spent two years in the woods, and he saw more than woods. What did he discern among the flora and fauna? He sought repose and serenity among the pastoral landscape. He embraced the seasonal trends and the factors that brought shifts in the environment.

You enable discovery through recognition of the customer landscape, by observing the trends on competition, innovation breakthroughs that are occurring. This is strategic analysis, partnerships with the corporate leaders and chief technology officers.

Consider their tried-and-true SWOT analysis, tool of virtually every strategist. Data scientists should be analyzing every quadrant of the framework, ascertaining the data points that can be investigated for relationships and guidance.

Without fluency in the corporate and IT strategy, the data scientist is misused and limited.

In fact, their value will diminish under the automation and the AI of insight-querying. The genius of the data scientist is more than just familiarity of a few strategic principles but, rather, actual immersion into the company strategy.

(4) Fluency in Consultative Guidance

Create a practice within your data analytics environment. Like any journey of discovery, there is beauty in data mining. Data diamonds are in the rough, but cutting them for maximum demand is an art. Engage your constituents and explain the framework of your data practice.

Educate your peers and partners on the informational classification of your data stores; I don't refer to the foreign and complicated code. I mean, what kinds of information do you manage and in what form is it available.

Determine what kind of data (structured and unstructured) each department manipulates in its daily activity, and what do those departments typically use in their 'day-in-the-life'?

Describe your own 'day in the life' as well, which will provoke new ideas and recommendations from all corners of the enterprise, technical or not.

The value you and they provide will be elevated when you can simplify the story of your data analytics practice.

Data scientists live in a world of complexity, a world that needs to be simplified. They hear Thoreau, but their job is monumental. They are asked to do the nigh impossible – to translate, then take exabytes and flip them into three charts!

Those charts will be focused on the corporate strategy,

which will spill into Marketing’s palm,

which will conjure artifacts for the Sales team,

which will improve scenarios for Operations,

which will also be imitated by competitors...

so the scientists shall need to recursively delve deeper, again and again and again.

The value of simplicity will rise eternally, as complexity increases throughout our ecosystem.

If your data scientists are transaction warriors who simply start digging when asked, you are expecting only one quarter of their accountability. They are business professionals who advance strategy through their statistical, mining genius and partnerships. They are foundational to innovation.

Thoreau’s call to order -- simplicity – is the mantra of your team. The technology will change, the querying code, the architectures, the pattern matching speed, the AI, the sampling. The greatness in the data analytics team lies in their holistic and strategic understanding, their adaptation and their heightened skill in simplifying an evermore complicated world.

As the leader in the Data Analytics center of excellence, understand the four cornerstones of your advantage:

• expertise in the technology;

• fluency in the corporate use cases;

• immersion into the company strategy;

• and mastery of consultative practice.

Like an archaeologist, facing a mountain of information, layered with unfathomable complexity under millennia of neglect, it’s not simply what you’re looking at, but what you can really see.

---

An afterword:

Over time, Henry David Thoreau’s cabin in the woods disappeared.

For decades after his death, the area thought to be the home site was erroneous.

The analysis of estate information was simply not detailed enough.

But on November 11, 1945, after months of research, analyzing notes of 19th century cabin visitors, deciphering clues from yellowed diaries, and following recursive treks throughout the ‘big data’ of the woodland, amateur archaeologist Roland Wells Robbins discovered the site -- nearly a century after 'Walden' was first published.