Stay informed with free updates
Simply sign up to the Technology sector myFT Digest — delivered directly to your inbox.
The writer is CEO of the Royal Academy of Engineering
The inquiry into the Post Office Horizon scandal has moved into a crucial phase, with former senior executives having given evidence on what they knew about the capabilities of their computer system. Much of the case has understandably centred on the appalling human consequences of this failure — but how can we ensure that it does not happen again?
The problems of designing and implementing complex IT systems were already well known in the engineering and computer science professions at the time of Horizon’s development. Twenty years ago, the Royal Academy of Engineering and BCS, The Chartered Institute for IT, issued a stark warning for those commissioning such systems. Data at that time suggested that only 16 per cent of the UK’s IT projects were considered successful and that billions of pounds were being wasted on new systems each year.
Our report identified a critical shortage of systems architects with the skills to translate a business plan into a deliverable technical blueprint. Since then, the pace of change has accelerated thanks to increased computing power and other developments including AI. As a result, the complexity of software supply chains has increased and lines of accountability are often obscured.
The risks posed by this are compounded by more frequent cyber attacks that exploit software defects. There is also widespread use of software components that have not been developed to professional standards and come with neither guarantees nor acceptance of liability by originators.
Failure to tackle these problems could be a barrier to progress as the UK aspires to drive the adoption of artificial intelligence. Risk assessment and management are important if we are to ensure that AI-enabled systems are robust and resilient, and that failures are handled appropriately.
Engineering ethics is a core part of that professionalism, as shown by a 2023 survey. One-third of engineers and technicians reported that the work they undertook made them feel ethically compromised.
Other areas are in urgent need of attention too. There remains a legal presumption that the computer is always right. The Post Office could rely on the common law position that the courts were entitled to assume that the IT system was operating correctly. It was for the Post Office staff to prove that the outputs and logs from the computer system were flawed or not accurate. Yet how could non-IT specialists be expected to prove this when even some experienced IT professionals would find it a challenge to do so?
Organisations relying on evidence generated from computer systems to support prosecutions should be required to prove that the underlying computer system is reliable; we hope this will be a clear recommendation from the current inquiry as this is crucial to prevent recurrence of such injustices.
The engineering profession has a vital part to play in enabling the progress needed. This includes supporting the development of new technical standards that can be embedded in regulation and procurement contracts to guard against system failure.
Education, a culture that supports ethical behaviour and continuous improvement, and greater diversity within the profession, can all contribute to improved outcomes. Employees must also have access to reporting mechanisms for failures and near misses, including robust whistleblowing procedures and support when needed.
Modern life depends on computer systems that are becoming ever more complex and interconnected. While many examples of good practice in commissioning IT systems exist, it is vital that competency and ethical practice become the norm for those procuring and supplying complex software systems if we are to avoid another Horizon.
Read the full article here