I have been asked to share my thoughts on Big Data and Information Architecture in an internal group of Chief Architects by Iasa Global.  As I gear up to deliver this session, I think about the critical need for an effective ecosystem where Big Data can enjoy a healthy existence – and the vital role that the Chief Architect has in the definition, implementation and sustenance of this ecosystem.  The ecosystem must be such that the Big Data is continuously collected, processed and analyzed to yield insight providing the enterprise a fair opportunity to take timely action.  So, what are the factors that the Chief Architect must take into consideration to architect the ecosystem of Big Data to yield actionable insight? Let us take a look.
The life cycle of Big Data and Information can be broadly categorized into the phases listed below.

  1. Collection. There should be a cohesive process in place for the continuous collection and assimilation of the data through business-driven integration across the enterprise. The fundamental need to collect such data has only been accentuated with the introduction of newer technologies like wearables and the Internet of Things (IoT), along with the nearly instantaneous sharing of thoughts through various social media channels. There must be a systemic mechanism in place to collect and integrate data that matters to evolve enterprise wide Systems of Intelligence.  Call to Action: Data elements must have an effective avenue for them to be channeled through the Systems of Intelligence.
  1. Cleansing. Data is growing in exponential volumes through various channels every second.  The accuracy and integrity of this data is vital to it serving a useful purpose.  Moreover, adversaries are always on the prowl to compromise critical data to fulfill their own objectives.  Application of inaccurate or stale data can be devastating especially when increasing levels of automation are injected across the business processes.  Call to Action: Big Data Ecosystem must ensure the continuous monitoring and cleansing of the data at the source, during its transit and in its persisted state.
  1. Analysis. Big data does not yield much value without analytics! Enterprises need to have the right skills in place (think: data scientists or even data artists) so they can leverage the right tools and techniques to analyze—with context—the historical data resident in legacy systems as well as the more recent, up-to-the minute data collected through wearables. Call to Action: Big Data Ecosystem must ensure that the right technologies and platforms are in place to analyze the data across the Systems of Record and the Systems of Engagement.
  1. Insight. Analysis of such data yields valuable insight. Perceptive insight is like telling a nice story about what is happening and what is likely to happen purely based on the raw data available to the enterprise.  Leave it to the Drones to tell a nice story around Big Data! It takes more than just pure data science to glean insight.  It takes Data Artists! Call to Action: Big Data Ecosystem must ensure that the right solutions are in place to provide continuous visibility to the data that matters to the Data Scientists within the enterprise.
  1. Action. All of these steps may be to no avail unless the enterprise takes timely action.  It behooves key leaders and stakeholders within the enterprise to react in a timely manner to the insight gathered through these channels.  Therefore, the ecosystem must ensure that there are self-governing mechanisms in place that drive the required checks and balances. Call to Action: Big Data Ecosystem must ensure that there are governance mechanisms in place to ensure that timely action has been taken on the insight gathered.

World leaders like Mother Teresa, Abraham Lincoln and Mahatma Gandhi have analyzed the data available to them through not-so-technical channels to achieve their goals and the objectives of the organizations they led. They did not have an Information Technology team that enabled them to take these powerful steps.  The Big Data ecosystem must also have the right overall mindset in place including the mindset of the people architecting it.  Hello Chief Architects !
Those are some of my seeding thoughts on what it takes to architect an effective ecosystem for Big Data and Information.  What say you?  What are some of the steps that Chief Architects can take when defining the Big Data and Information Architecture?
Let the discussions begin right now, right here on this blog with your comments!
Follow Nadhan on Twitter LinkedIn