The rise of AI in data analytics

“Artificial intelligence is not a substitute for human intelligence; it is a tool to amplify human creativity and ingenuity.” ~ Fei-Fei Li

Transforming Data to Insights

Data analytics in its most basic form is the process of obtaining raw data and applying analysis techniques to obtain meaningful information from this data. The explosion of big data, characterised by its volume (the amount of data generated), velocity (speed at which data is generated) and variety (the different types of data being generated) coupled with the advent of user-friendly data analytical tools for non-technical users, saw a significant demand for data analytics across various industries. As we navigate a data driven era, data analytics continues to evolve. The integration of artificial intelligence with data analytics revolutionises the way we analyse and approach data.

Traditional analytics focuses on what happened and why it happened. While these are fundamental questions that need to be answered, they are only the beginning. Data analysis is no longer just concerned with volume of data, but also how it can be used. To increase the value of the analytics performed, organisations must strive to not only obtain information from their analysis but to be able to use it to optimise decision making and strategic outcomes. The combination of predictive and prescriptive capabilities enables faster, more accurate and more relevant decisions in complex and fast-changing business environments. The Gartner Analytic Ascendancy Model best explains this:

 Source: Gartner

The Gartner Analytic Ascendancy Model is a framework developed by Gartner to guide organisations in using data analytics. The model depicts the progression of analytics capabilities from basic descriptive analytics to more advanced predictive and prescriptive analytics. It emphasises the increasing value and complexity as organisations move up the hierarchy of analytics.

Enhancing Data Analytics through Artificial Intelligence

Artificial intelligence (AI) simply defined, is the creation of computer programmes that can perform tasks typically requiring human intelligence. Such tasks include, solving problems, visual perception, speech recognition and even learning from experience. From smart devices and tailored content to Chat GPT and digital assistants, artificial intelligence has quietly infiltrated our lives without some of us even realising. In the current fast paced digital world, AI aims to improve human efficiency and productivity.

Basic data analysis was usually done by designated individuals within departments, creating data silos with results that often differed from each other. The expertise, time, and investment in performing effective data analysis often presented a barrier in its implementation. AI now allows organisations to drive innovation and add value without the substantial investments in data resources or departments. One of the prominent advantages AI offers, is the increased speed and efficiency in which complex analyses can be performed. AI algorithms can filter through large datasets faster than conventional means, providing detailed, accurate and relevant results as and when needed. Leveraging AI within data analytics can help elevate analyses and assist in answering, what will happen and how can we make it happen.

AI in data analytics relates to a broad range of tools and techniques. Machine learning, a subset of AI, involves training algorithms on historical data and applying it to current data to perform predictions and forecasts. Machine learning can be used to identify patterns and behaviours within the data that might not be easily identifiable through standard data analysis techniques. For example, learning customer behaviour can help focus marketing strategies specific to their lifestyle or even detect fraud where transactions are seen as outliers in its population.

Natural language processing (NLP) overcomes the limitation of analysing numerical data only. NLP can analyse large volumes of text data to extract sentiment and identify trends. By employing various NLP techniques, raw text data can be transformed into a structured format suitable for analysis. For example, NLP can be used to extract sentiment from customer reviews over a particular product or from employee surveys over working conditions. Generative AI can be used to prepare reports based on the analysis performed or even be used by others to interact with the results.

In addition to the various AI algorithms available, AI can also enhance the automation of analytics. Routine, repetitive and manual tasks can be automated to increase the level of efficiency and consistency achieved. Data cleansing, preparation and preliminary analysis can be performed directly by AI allowing the analysts to better focus their time to apply their professional scepticism and human insight to the results. Furthermore, automated procedures provide the benefit of more accurate results by minimising human input and thereby reducing potential errors.

The use of AI in data analytics does not replace the need for data analysts. However, it assists them in expediating analyses on a deeper level, so that more time can be spent on interpreting results, formulating strategies, and providing solutions. AI will not only streamline the process of analysing data but also be the driving force to optimise productivity and efficiency. By replacing manual and time-consuming tasks with AI, more resources can be expended on ensuring strategic objectives are aligned and realised.

Confronting AI challenges

Any discussion on AI cannot be complete without mentioning the risk and challenges that arise when using or relying on AI. To realise the potential value of AI, high quality data should be readily available and accessible. Without appropriate and sufficient data, even the most advanced algorithms would not be able to provide accurate results. Incorrect result can be devastating for organisations when used as the basis to make decisions. Furthermore, owing to the sensitive nature of data, data privacy and security should always remain a top priority with any data analysis tool.

Low code/no code developments and AI continue to dominate innovation within the data analytics space, enabling more users to perform data analysis. However, highly skilled professionals are still crucial in implementing AI analytics. These professionals are needed to ensure the right models, algorithms and techniques are selected and trained to achieve the required results. They are also required to ensure the results can be interpreted correctly into actionable insights that can be communicated to the relevant stakeholders. Human judgement remains indispensable in maximising the value that data analytics and AI can achieve.

AI is frequently thought of as operating as a black box. A black box is explained as a system or programme where the inner workings are not easily understood or made available, due to complexity or proprietary information. As AI advances, more complex algorithms are created, additional layers and parameters are included, which inherently complicates the inner workings. The black box nature of AI also raises ethical concerns as models trained on historical data might be subject to bias. Such bias and the lack of transparency further affects the trust and reliability of AI.

In conclusion

As AI advances, it reshapes the analytical landscape that was once understood. These disruptive technologies are changing the way we think and work with data. What once seemed far fetched is now a mere prompt away, accelerating both innovation and competition. While the future of AI is eagerly anticipated, the continuously advancing realm of data and analytics needs to be aligned to robust data governance and ethical frameworks to ensure trust, reliability and accuracy of results. While AI, like any other innovation, is not without risks or challenges, the rise of AI in data analytics must be embraced to deliver value in an evolving digital world. Ultimately, “Artificial intelligence is not a substitute for human intelligence; it is a tool to amplify human creativity and ingenuity.”

Salim Mohamed

Data Analyst, Johannesburg