The payoff for a data driven organization is clear; More effective decision making, better response to customer needs and the ability to stay ahead of competitors with innovative ideas and methods. Data driven organizations can execute more quickly, better understand the impact of decisions, and ensure that all elements of the organization have an opportunity to propose ideas that will improve the customer-product relationship.
A key component of the transition to becoming a data driven organization is the acceptance of a constant state of change, with a rigorous measurement component. These constant changes involve the automation of formerly manual tasks, requiring human approval, to a state where machines and algorithms make decisions and execute them on their own. This constant state of change is managed through common best practices (Figure 1) that assist an organization in ensuring high-quality decisions are made. These best practices are driven by data accessible to a wide portion of the organization to greater facilitate collaboration.
Figure 2 illustrates the common maturity levels an organization will progress through as they become a data driven organization. An organization will not seamlessly move from one level to another, but rather mature each portion of the organization at different rates, depending on the skill sets and outside influence.
Description of the data maturity levels:
- Data Access – This is the first level of data maturity, and characterizes organizations that are early in their data journey. These organizations often store information for reference but do not regularly use that information to drive decision making or work to integrate the data into third party systems for automation in use.
- Consolidation – This stage in maturity characterizes organizations that have taken initial steps to integrate their separate datasets and create more formalized applications for the presentation and updating of the information. Decisions are still manual and human driven.
- Reporting – As organizations improve their use of data, they begin to report on these integrated data sets. This Reporting phase is focused on providing visibility to common metrics and measurements so that staff have readily available information when making decisions.
- Alerting – As organizations mature beyond solely accessing integrated data for decision making, they develop capabilities to alert key decision makers when common metrics look like they are on a negative trajectory. This enables rapid response by experienced staffers to make the necessary adjustments and corrections.
- Engaging – The most mature organizations move beyond alerting and use data in all aspects of their business to measure, respond and improve. Organizations at this level of maturity have a culture that values data driven decisions and deep analysis of correlated effects and causation for events.
Being a data driven organization means the simplification of all processes for decision making. This ensures that decisions can be automated, made quickly, and effectively measured and recovered from if failures do occur. All processes should be broken down into discrete elements and automated where possible. There will inevitably be areas of the organization that will require manual input in various phases. But the key is to minimize those, and eliminate them as better data becomes available for decision making.
One way to prioritize this transformation of decision-making processes is measuring the cost of a decision. Decisions that have a low cost of failure or high rate of return should be automated first. Decisions that have a higher cost of failure, often including the impact on human life, should be the last to automate until the appropriate safeguards and checks are in place to eliminate the risk of negative consequences.
This process of re-engineering will be dependent on the staff who understand the processes today and thus will be tasked with the automation of those processes. Cultural habits will have to be developed to encourage staff to think in new ways, let go of job functions that were once manual, and encourage collaboration around integrated data sets to better facilitate decision making across departments.
This cultural change will require new skills, both technical and soft skills, to enable staff to be effective. Training should augment any outside hiring in the organization. This training should focus on enabling staff with new skills, both to make the transition to a data driven organization, but also to encourage staff to constantly improve their use of data.
Creating a data driven organization involves making a simultaneous commitment to maturing people, processes and technology. The technology enables the process that is defined and executed by the people. A data driven organization integrates data sets, provides access, and encourages automation of discrete process elements. These organizations focus on the measurement of all decisions so that constant improvements can be made and staff enabled to focus primarily on exceptions, in order to provide maximum value to the business.