7 Principles to Create Trustworthy Artificial Intelligence SAP Systems
Artificial Intelligence (AI) has been a topic of immense importance for Businesses for over the past few years, as it offers many possibilities for optimisation leading to competitive business advantages over other players in the market. The Merriam Webster dictionary defines AI as “a branch of computer science dealing with the simulation of intelligent behaviour in computers and the capability of a machine to imitate intelligent human behaviour”. According to Wikipedia, it is also sometimes referred to as “Machine Intelligence” meaning intelligence demonstrated by machines in contrast to natural intelligence displayed by humans. It also says “Colloquially, the term “artificial intelligence” is often used to describe machines (or computers) that mimic “cognitive” functions that humans associate with the human mind, such as “learning” and “problem solving””.
According to SAP, “Machine Learning” is a type of AI that allows computer programs to adjust when exposed to new data, making it “learn” without being explicitly programmed. Because of the superior computational power, the machines have, they can crunch voluminous data to find patterns, trends, make projections of future outcomes based on historical trends, arrive at fact-based decisions and so on. The intention is to use the computational power of machines for AI to do the heavy thinking for us.
Allowing AI to mimic human “cognitive” functions has raised many concerns about its trustworthiness and considerations of ethical aspects, when it makes recommendations or proposes certain outcomes and decisions. This blog article is an attempt to explain SAP’s approach to build trustworthy and ethical AI and the principles involved behind it.
There are many academics and knowledge groups working on this topic, across the globe. There is a European Union High level Expert Group, which has released a set of guidelines and seven key principles for a trustworthy AI. SAP has adopted them, and we will review these seven principles, detailed below:
- Human agency and oversight
- Technical robustness and safety
- Privacy and data governance
- Transparency
- Diversity, non-discrimination and fairness
- Societal and environmental well-being
- Accountability
Human Agency and Oversight
This principle states that human beings are in the centre of an AI system, as opposed to be being treated as an object that is manipulated in the AI system. Human oversight is about finding the right approach to make humans in command and/or in control of what AI, IT and the human system is doing in a situation. Three types of approaches are possible in a situation. The first approach is to make the human approve or disapprove every single recommendation by the system. The second approach is to have the human in the loop but deal with only exceptions and not every transaction. The third approach is to have humans in command. This implies humans are involved in the design, specification, engineering of the solution and also have them involved in periodic assessment & evaluation of the outputs & quality that AI system delivers. Depending on the situation, a suitable approach has to be taken. One can see an adoption of this principle in SAP Intelligent Robotic Process Automation solution, where a “BOT” can work as a “Digital Assistant” or as a “Digital Worker”. Refer to our blog post SAP Robotic Process Automation for more info on this solution.
Technical Robustness and Safety
This principle involves AI systems behaving reliably as intended and preventing unacceptable harm to humans. AI systems must also have resilience to attacks such as hacking. AI system designs must consider the possibility of misuse to unintended applications and prevent/mitigate them. AI systems should have safeguards that enable a fall-back plan in case of problems. For example, switch from statistical-based to rule-based procedures or ask for human operators before continuing an action. AI system results must be reliable and reproduceable. Reproduceable means AI systems exhibit the same behaviour when tests are repeated under the same conditions. Using Cloud Robotics, a newly developing area where safety is of high importance, SAP has provided a robust and safe Warehouse Management solution. Refer this interesting blog post Cloud Robotics for SAP-Extended Warehouse Management for more information.
Privacy and Data Governance
AI systems must be designed to guarantee data privacy and protection throughout its lifecycle. Adequate data governance must exist to ensure quality and integrity of the data that is used, such that AI systems can access and are capable of processing data in a manner that protects privacy. It must be guaranteed that digital records collected about humans will not be used by AI systems to unlawfully or unfairly discriminate against humans. Processes and data sets used to train AI systems must be tested and documented at each step such as training, testing and deployment. There are regulatory requirements such as GDPR in Europe which the AI system design and behaviour must comply with. In SAP Intelligent RPA, this principle is adopted as explained in the exhibit-A below.
Exhibit-A (Credits: SAP SE)
Transparency
Transparency is an important aspect, as it creates trust. Transparency comes from three different aspects. The first being traceability, secondly is “explainability” and thirdly is communication. Data sets and processes leading to AI system results must be clearly “traceable”. Decisions made by AI systems must be “explainable” by humans based on data sets and processes used by AI. AI systems must “communicate” as such and not represent themselves as human beings. Transparency is quite complex in real life processes. To understand this, consider the example of an AI system in an autonomous car, where it takes decisions on detecting a pedestrian crossing a street and taking necessary action to avoid people. The data sets involved; continuous recalculation of trajectory, makes it complex to build a trustworthy AI system. As an example (a time recording use case), in SAP Conversational AI, one can observe how transparency is maintained, so that the user is very clear in understanding how AI is working for him/her. Refer to this blog post Talk to the hand ’cause it’s listening to record your time for more details.
Diversity, Non-discrimination and Fairness
AI systems are trained using huge volumes of data. It is important that diverse sets of data, including every possible scenario, type of people, etc… is included in training data sets. Care should be taken to avoid prejudice in collecting data. AI products and services must be designed to cater to people regardless of their age, gender, abilities and characteristics. The design should consider Universal Design Principles to address the widest possible set of humans adhering to relevant accessibility standards. One of the ways to ensure fairness is to include people from diverse backgrounds, cultures and disciplines during the AI system lifecycle. In SAP, the award-winning Fiori-3 design is a user interface, enshrining this principle. For more info on Fiori-3, refer to this blog post SAP Fiori 3.0 – Curtain Raiser – Sneak Preview we published earlier.
Societal and Environmental Wellbeing
Apart from the impact of AI systems on individual human beings, the impact on society and the surrounding environment in which it operates must also be considered. AI systems interacting with democratic processes such as in an electoral context or aiding political decision making, can impact significantly the human society. AI systems designed to solve environmental issues must be evaluated carefully. Hence AI systems for all these situations must be given careful consideration and evaluation before implementation. Overall, the AI system impact on society and environmental wellbeing, of current as well as future generations of human beings, must be considered. An example is the Australian National Disability Insurance scheme, which is providing personalised treatment plans for Australians living with disabilities, using SAP digital core and machine learning. For more details refer to this Digital Social Services: Seeing Through Homelessness Sludge blog post.
Accountability
This is interlinked with many of the principles explained above. The principle of accountability requires mechanisms to be put in place to ensure responsibility and accountability of AI systems and their outcomes, both during development and after deployment. It must be possible to independently audit AI systems’ algorithms, data and processes. Any negative or unexpected outcomes of AI systems must be critically evaluated during the development/test phase and also during operation. Proper assessments must be made, considering the proportionate level of risk the AI systems pose. Adequate redressal mechanisms to impacted humans must be provided to create trust in AI systems. In SAP Intelligent RPA, where machine language processing is used, the architect must design the Bot and its interactions considering the accountability aspects of results.
Summary:
AI has got immense potential to transform many processes and services, in our day-to-day life, into automated tasks handled by computing power. On the other hand, uncontrolled usage can cause immense damage to our society. Creating a “Trustworthy AI system” is the key to harness this power. Striking an appropriate balance by adhering to good principles, SAP has provided solutions such as SAP Leonardo and SAP Intelligent RPA. By adopting these principles, SAP consultants and architects alike, can create “Trustworthy solutions” for customers.
Author: Ravi Srinivasan , SAP Alumni
Get in touch with Eursap – Europe’s Specialist SAP Recruitment Agency