Deriving business knowledge and value from data has never been more critical to organizations to stay competitive nor has it been more challenging due to the disruptive nature of big data. The big data phenomenon – the volume, variety, and velocity of data – has created new challenges for organizations to deliver business analytics.
One primary challenge is using existing project methodologies to deliver business analytics projects. Applying traditional project methodologies is problematic and has been identified as the largest contributing factor for business analytics project failure; organizations are treating business analytics projects like any other information technology (IT) project. New agile processes are needed to increase business analytics project success.
Best Practices in Business Analytics Project Delivery
Business analytics is still evolving; thus best practices are just starting to emerge. The following best practices have emerged from leading analysts and practitioners based on some of the failure points of business analytics projects. These best practices reinforce the failure of existing methodologies to address the challenge of big data and the need for leveraging agile practices.
Due Diligence in Defining Business Goals and Problem Statement
Business analytics projects have failed due to the lack of clear business goals or problem statements. The success and value of a business analytics project will be defined at the start of the project. By clearly defining the business goals to be addressed and formulating a problem statement, the scope of the project becomes clear.
A clear scope impacts the steps of data understanding and data preparation. If the investment is not made up-front to understand expectations, subsequent work on the business analytics project is impacted.
The problem statement should clearly identify the data needs of the project. Leveraging a collaborative team of stakeholders and delivery personnel can assist in addressing this.
Allow Time for Data Understanding, Acquisition, and Preparation
A key failure point in business analytics projects is allowing for the time needed to get the data required for the project. Often the time needed to work with the data is underestimated or not known at the start of the project.
The challenges that come with big data emerge in this area. Working with multiple technology platforms, multiple data structures, data sampling, data integration, and merging data into a final set for modeling takes time and planning.
Additionally, once a final data set is created, the project team should document the data lineage to ensure the data set logic is clear and the data set can be recreated via a data pipeline.
Identify Needed Toolsets and Skill Sets at Project Start
As outlined, the technology infrastructure in the business analytics industry can be varied and complex. Tools can be open-source where support is lacking adding to the complexity. Technical tools are needed for analytical modeling and data wrangling which are key for the success of the project.
The project team will need to have a diverse set of roles focusing on technology, analytical modeling, statistics, and using business intelligence systems. The project leader will need to have a strong technical and analytical skill set to lead the project.
Identifying the tools and skill sets at project start prevents technical and team barriers during the iterative stages. Leveraging small versatile teams can be of benefit here.
Allow Time for the Cycle of Modeling and Evaluation
The value of business analytics occurs when insight is attained. Insight refers to the capabilities provided by the model which typically falls in the categories of lowering risk, increasing revenue, increasing productivity, and supporting and shaping an organization’s strategy.
Modeling and evaluation tends to be experimental which results in answers but also more questions. Models also need to be built with different algorithms to be able to validate accuracy increasing the time needed for more experimentation Iterative software development cycles can be leveraged to support the cycle of modeling and evaluation.
These best practices highlight the need to consider adapting agile principles in business analytics delivery. The next section explores the prevailing methodology and agile application in business analytics.
Methodologies have value in that they help deliver projects effectively and efficiently, which, in turn, enables business value from the project investment. Business analytics projects often are initiated without clear objectives and outcomes, inviting constant scrutiny on whether business value occurs . Deriving value from information means that each business analytics projects need to deliver usable results, where each project is a transformational effort to create knowledge from data.
The scope of each business analytics projects covers all activities that need to occur in the information value chain, which includes converting data into information and information into knowledge. Business analytics projects have a high degree of complexity as converting information into knowledge requires the use of statistical and analytical models as well as incorporating the use of big data. The Cross-Industry Standard Practice for Data Mining (CRISP-DM) is the top methodology in use in business analytics
CRISP-DM is a data mining process model that conceptually describes the stages that are used to tackle data mining problems. CRISP-DM was originally created to align with data mining, but it has organically evolved into the primary approach used by data scientists.
CRISP-DM contains six stages which appear to be in sequence; however, the stages are not strictly sequential, and iterating through the stages is expected.
Business understanding focuses on determining the business requirements and objectives to create a problem definition. Business analytics initiatives start with a question of interest or problem to be addressed. The outcome of this stage is a problem definition. The problem definition may be diagrammed using decision modeling or other approach.
Once a problem statement is understood, data collection can proceed. Data collection involves obtaining data attributes required to address the problem statement.
Often data integration is required as data may come from various sources in various formats. Once data is usable, data is profiled and statistically analyzed to determine demographics, relationships, distribution, and quality levels. The outcome of this stage is initial insights into the data to support the problem statement.
Data preparation is necessary to create the data set that will be used in the analytical modeling stage. Data preparation includes the final integration of various attributes, cleansing of attributes, and deriving of new attributes.
Activities in this stage are often iterative as new data may be required to support the subsequent modeling and evaluation stages. The outcome of data preparation is a data set that is to be used in the first iteration of modeling.
Traditional project delivery methodologies are not robust enough to address the challenges of big data and or business analytics. Synthesizing ASD principles, CRISP-DM phases, with emerging best practices to create the business analytics, project delivery framework is a start in evolving business analytics projecst delivery.
Myself Nadeem KTK is a Business Intelligence and Analytics Consultant with a focus in the Pharmaceutical and Consumer Product industries. My experience includes data visualization in custom dashboards using Tableau, architect of AWS environments, and analyzing data using the open-source software platform R. My focus is on customer insight analysis, marketing optimization, and data visualization. I hold an MS in Business Analytics from Drexel University and a BA in Economics and Communication from Rutgers University.