Apple co-founder says Goldman's Apple Card algorithm discriminates

Jennifer Bailey, vice president of Apple Pay, speaks about the Apple Card at the Steve Jobs Theater in Cupertino. Photo / AP

Apple and Goldman Sachs, two of the most recognisable companies in tech and finance, are caught up in a growing debate over whether lenders unintentionally discriminate when they use complex models to determine how Americans borrow money.

On Saturday, Bloomberg reported that a Wall Street regulator had opened a probe into Goldman's credit card practices after a viral tweet from a tech entrepreneur alleged that the Apple Card's algorithms discriminated against his wife.

Now another high-profile user of the Apple Card - Apple co-founder Steve Wozniak - is calling for the government to get involved, citing excessive corporate reliance on mysterious technology.

"These sorts of unfairnesses bother me and go against the principle of truth. We don't have transparency on how these companies set these things up and operate," Wozniak said in an interview on Sunday. "Our government isn't strong enough on the issues of regulation. Consumers can only be represented by the government because the big corporations only represent themselves."

Wozniak said that he can borrow 10 times as much as his wife on their Apple Cards even though they share bank and other credit card accounts, and that other lenders treat them equally.

"Algos obviously have flaws," Wozniak said. "A huge number of people would say, 'We love our technology but we are no longer in control.' I think that's the case."

Lenders have promoted the models because they're supposed to level the playing field among different borrowers by removing human error and focusing only on data.

Apple Card only offers individual accounts, and it is possible for two family members to receive significantly different credit decisions, a Goldman spokesman said. "In all cases, we have not and will not make decisions based on factors like gender," he said.

The investigation was launched in response to a series of Twitter posts from David Heinemeier Hansson that railed against the Apple Card for giving him 20 times the credit limit that his wife got. The tweets, many of which contain profanity, immediately gained traction online - and a response on Twitter from Wozniak.

U.S. entrepreneur and co-founder of Apple Inc. Steve Wozniak. Photo / AP

Hansson didn't disclose any specific income-related information for the couple but said that they filed joint tax returns and that his wife has a better credit score than he does. Wozniak said he and his wife also file joint returns and share credit card and bank accounts.

"The department will be conducting an investigation to determine whether New York law was violated and ensure all consumers are treated equally regardless of sex," said a spokesman for Linda Lacewell, the superintendent of the NY DFS. "Any algorithm that intentionally or not results in discriminatory treatment of women or any other protected class of people violates New York law."

It's the second such action in recent weeks from the regulator, which opened a probe against health-care giant UnitedHealth after a study found an algorithm favoured white patients over black patients.

"New technologies cannot leave certain consumers behind or entrench discrimination," Lacewell said in a statement on Sunday. She also solicited complaints from aggrieved consumers on Twitter.

Traditional lenders are increasing their use of machines to decide who gets how much credit as part of a strategy to reduce costs and boost loan applications. Meanwhile, technology companies are moving in on the financial services industry's turf, with businesses such as Amazon, Apple, Facebook and Google threatening banks' lucrative business lines by offering loans and payment options.

The algorithms have drawn scrutiny in Congress. In June, the House Financial Services Committee heard about examples of algorithmic decision-making where researchers have found instances of bias targeting specific groups even when there was no intent to discriminate.

Some lawmakers are demanding a federal response. Sen. Elizabeth Warren, D-Mass., who is running for the Democratic nomination to challenge President Donald Trump in the 2020 election, told federal regulators in June that the government "will have to take action to ensure that anti-discrimination laws keep up with innovation."

Goldman's growing ambitions for Main Street are bringing increased scrutiny and a new set of challenges it hasn't faced previously. The Apple Card is a joint venture between Apple and the New York-based bank, which is responsible for all the credit decisions on the card. It was rolled out this year - the tech giant markets it as "created by Apple, not a bank" - and executives at both firms hailed it as the most successful launch ever.

Hansson said Goldman isn't treating inadvertent bias seriously.

"As soon as this became a PR issue, they immediately bumped up her credit limit without asking for any additional documentation," he said of his wife in an interview Saturday. "My belief isn't there was some nefarious person wanting to discriminate. But that doesn't matter. How do you know there isn't an issue with the machine-learning algo when no one can explain how this decision was made?"