Artificial intelligence as a decision-making aid has improved to the point where its efficacy and trust may be starting to exceed that of a human decision maker.
This intersection of trust is a relatively new phenomenon, albeit one that mirrors the pace of adoption of artificial intelligence in the enterprise.
Only two years ago, it was the other way around.
A 2017 PwC survey found 76 percent of CEOs were concerned at the possibility of unintended bias creeping into an AI algorithm or decision-making model. Clearly, humans were still to be favoured.
Just a year later, in a mid-2018 study published by the Harvard Business Review, researchers tested whether a 15-minute conversation between humans, or an algorithm, was the better method for measuring the trustworthiness of a new colleague.
While both methods were considered reliable, the algorithm was seen as “a more rational and less intuitive approach in evaluating an individual’s trustworthiness”.
And when “asked to indicate which assessment method they preferred to use, most participants opted to use AI (61 percent) rather than the judgements of the human (39 percent),” the study authors reported.
In short, the efficacy of AI is now forcing us to make tougher calls on who is better placed to make a particular call: an algorithm or us?
Getting uncomfortable
There will undoubtedly be some challenges in relieving humans of more of their decision making.
AI is likely to cause some people discomfort because it will directly challenge their conclusions and logic.
Your brain has no audit trail. Its assumptions and biases aren’t clear. And, at some point in the not-too-distant future, this kind of opacity in the decision-making process will no longer be acceptable.
Consider hiring decisions. These are often still based purely on the judgement of one or more interviewers. It’s a very subjective process. Ultimately, when a decision is made, candidates may never really have full transparency on what the decisive factors were in whether they were successful or not.
Contrast that with how an AI algorithm can make recommendations. You can be very explicit in defining the attributes that will lead to a particular outcome. If you build or train the model yourself, and you understand how it works, then you have full transparency over the decision criteria, which should give you trust and confidence in what the recommendation is.
You may or may not like the outcome, but at least you understand it and can have confidence in it, as opposed to an opaque “black box” process driven entirely by a human brain.
Isn’t AI also a black box?
The challenge here is that transparency over the algorithm is not a given. It is still the exception rather than the norm and will continue to chill deployments while it remains so.
Transparency of the reasons used to reach a decision is arguably just as important as the conclusion or outcome itself.
People want to be able to understand how and why a conclusion was reached. That directly influences how much trust they are willing to put in the conclusion (whether reached by AI, a human or both).
Transparency is a core principle in BlackLine’s AI investments. We see it is a critical success factor, but this isn’t necessarily a commonly held view. Levels of transparency in algorithms still vary between industries and solutions, and this will need to change if AI is to achieve its potential.
Greater levels of AI and automation are being introduced to accounting and finance, and those algorithms now recommend paths and decisions to varying degrees of probability and accuracy.
But an accountant is not going to feel comfortable certifying books or numbers based on an AI recommendation unless they understand why. They want to understand the attributes and variables used to come up with a particular recommendation and how they correlate.
Think about approving a loan. If you’ve got an algorithm that looks over the chances of default history for a particular loan, based on factors like credit, age, marital status and so on, that’s one thing. But if, for example, the algorithm also factors in (and draws inferences from) an applicant’s postcode, it may violate laws or regulations, such as the US Fair Lending laws, which prohibit “redlining” applicants by zip code. This is why understanding how a recommendation was reached is just as important as the recommendation itself.
Until there is greater transparency over the algorithms, trust in AI may be determined by the perceived reputational cost of backing an AI-based decision.
If AI provides an answer with an 80 percent probability that it is right the first time, does that really tell you, as a human overseer or user, what you need to know or do, and what the cost of making that decision could be? In the case of an autonomous driving car, where the costs of a bad decision can be fatal, clearly not.
The cost of an error is likely to determine how much trust and level of accuracy is required. Until such time, people may be unwilling to put their name to AI-led or AI-recommended decisions if it is not completely transparent how they were reached.
Pete Hirsch is the chief technology officer at finance and accounting software leader BlackLine.
