The trap of disconnected data and outdated processes

Five monkeys were placed in a cage as part of an experiment. A ladder, with bananas at the top was placed in the centre of the cage. Every time a monkey tried to climb the ladder, all of the monkeys including the monkey climbing the ladder, were sprayed with cold water. Eventually, when any monkey attempted to climb the ladder, all the other monkeys forcibly prevented them from climbing the ladder so as to avoid the cold water.

The researcher then substituted one of the monkeys in the cage with a new monkey. The new monkey also attempted to climb the ladder, however, after several failed attempts, the new monkey learned the social norm. He never knew “why” he was not allowed to climb the ladder since he had never been sprayed with cold water, but learned that this behaviour would not be tolerated.
One by one, each of the monkeys in the cage was substituted with a new monkey until none of the original group remained. Every time a new monkey tried to climb the ladder, the rest of the group pulled him off, including those who had never been sprayed with the cold water. By the end of the experiment, the five monkeys in the cage had learned to follow the rule (not to go for the bananas), without any of them ever knowing the reason why. The rationale behind not letting their cage mates climb the ladder, would probably be ”that’s just how its always been done.”

This story, whether authentic or a fable, captures a pervasive theme in many organisational cultures: Do what had always been done, without questioning or revisiting the reason behind it, even though the original reason might no longer exist. This mindset leads to the infamous trap of disconnected data and outdated processes.

To achieve growth and remain competitive, innovation and collaboration is required within all spheres of the organisation. Outdated processes and disconnected data is one of the major barriers to innovation and collaboration. Resistance to change combined with legacy systems cause lower levels of employee satisfaction and increased operating expenses as a result of high staff turnover, more errors and unproductive time. Internal constraints and inability to remain competitive can be catastrophic in an already challenging economy.

One of the most apparent and detrimental impacts of disconnected data and outdated processes is financial loss. When data is fragmented throughout an organisation, stored in different formats and across multiple databases, its access and availability becomes limited. This lack of integration impedes management’s ability to obtain a comprehensive view of the organisation's data, hindering decision-making processes. Uninformed decisions lead to missed opportunities and reduced flexibility in adapting to consumer behaviour and/or market conditions.

Disconnected data significantly impacts the effectiveness of data analysis. Collating and harmonising data from various sources is a tedious and time-intensive process. These processes are often heavily dependent on manual procedures which means that these tasks may not be automated. This laborious data collection process has to be repeated each time new data is required, frustrating employees even further. Despite all this effort, the lack of standardisation from outdated processes produces inferior data quality for analysis. The delay and questionable integrity of data, once again leads to uninformed or delayed decisions, ultimately affecting the profitability or reputation of the entity. The effort and resources consumed within this process often outweigh the benefits of analysing this data, thus reverting to “what has always been done”.

Outdated processes and disconnected data erode the fabric of a data literacy culture in any organisation. Without the drive to innovate, businesses become obsolete. Legacy systems might not be compatible with recent applications and therefore increased manual processing may be required. This causes further inefficiencies, errors and decrease in overall employee satisfaction. Without direct access to data, departments become reliant on IT to provide information they require which often leads to miscommunication and widens the expectation gap. As a result, departments, usually under pressure to provide management reports, tend to use only the information they have, despite knowing their reports and insights will be limited. Outdated workflows can also result in redundant and duplicated tasks due to limited system capabilities.

Disconnected data does not only stem from outdated or manual processes. Over utilisation of technology can result in a substantial amount of data being scattered across an organisation, creating data silos. Traditional workflows typically lock data in silos which do not favour team collaboration. Data silos refer to data isolated from others within an organisation which in turn creates obstacles to information sharing and collaboration across departments. Decentralised data becomes difficult to manage, exposing it to various risks including data loss and theft. Legal and regulatory risks might also become evident if the data is not managed and governed as required by the relevant legal frameworks. Costs associated with storing and maintaining such data and applications also affect the profitability of the entity.

Investing in a digital solution suitable for the nature and complexity of an organisation improves  efficiency and productivity. Built in validation, input checks and approval reduces the risk of errors and override of controls. Less time is spent on reviews and corrections allowing for additional capacity to streamline and improve workflows. Complete and accurate data will be readily available to analyse and provide insightful feedback to stakeholders. Management should prioritise the development and maintenance of a data warehouse to ensure all organisational data is available, complete and reliable. Integrated processes and its data facilitates collaboration between departments to enhance analyses and achieve strategic objectives.

Despite the most advanced technology, few departments implement data analysis successfully. It is often noticed that costly investments are made in data analytical tools but not in the data analytic process. This means that while organisations are committed to invest in the latest technology, they fail to ensure users are adequately trained and overlook the importance of enabling users to access and comprehend available data. When employees lose confidence in the data analytic process, they tend to revert to their previous methods, finding them more reliable and easier to follow. This cycle perpetuates into all areas of the organisation which prevents process owners from updating or modifying already established processes.

To avoid the trap of disconnected data and outdated process, various strategies may be implemented. A key area is to continuously monitor and improve workflows. This is the best method to ensure processes remain current and effective in achieving their objectives. Over time, any process will become outdated, however, consistently monitoring and improving the process allows an organisation to identify weaknesses timeously and adjust or amend the process to mitigate any negative impacts. Automation of these processes where possible will ensure less time and resources are consumed in repetitive tasks which allows more resources to be invested in focussing on key matters within the organisation. Six sigma and other such methodologies can also assist to improve workflows.

No organisation can completely avoid the trap of disconnected data and outdated processes. Prolonged outdated processes or disconnected data leads to more costly and time consuming redesign of these processes and the development of new data infrastructure. However, early detection and awareness allows one to proactively adjust to reduce costs and minimize business interruptions. Data and process management should form a key part the organisations short and long term business strategy to ensure these processes evolve in line with the business and industry requirements.

In conclusion, disconnected data and reliance on outdated processes not only reduces operational effectiveness but also affects employee morale, their faith in data analytics and deters the necessary evolution of processes. This dilemma not only hinders the optimisation of existing processes but also poses a challenge to using the organisations data to its full advantage. Addressing these challenges requires strategic emphasis on fostering an environment conducive to embracing updated technologies and methodologies. Only through embracing change can organisations liberate themselves from the trap of disconnected data and outdated processes, leading the way for enhanced efficiency and innovation.

Salim Mohamed

Consultant: Data Analytics, Johannesburg