Mustafa Ciftci / Anadolu Agency / Getty Images / WIRED
Six days a week during the pandemic, Shahid* got in his car and drove around the streets of Birmingham for Uber. Over the last five years, he had built up his profile: a customer rating of 4.96 and a Gold driver status, gained by accepting as many rides as he could. But one morning in early August, he found he couldn’t log in to work.
Uber’s algorithm, which governs who gets rides and how much they get paid, had detected a “continued pattern of improper use”, and the company had barred him from the system. He could not appeal the decision, he claims.
He says that Uber also refused to tell him exactly what he had done wrong. “We are not able to go into great details but the examples of improper use include using your rider and driver account at the same time, creating duplicate accounts, accepting trips without the intention of completing them, claiming false fees or charges, the installation, and use of software which has the intention or effect of manipulating the Driver App and trip details,” the message sent to him stated.
“Due to the above, we made the decision to terminate our partnership. As a result, you will no longer be able to provide independent services via the Uber App.”
According to Uber’s own terms and conditions , Shahid was being called a fraudster. He denies any wrongdoing. But there was no way to appeal the decision, he claims. In an instant he had lost 70 per cent of his income and has since relied solely on ride-hailing platform Ola to make a living.
He isn’t the only one in this predicament. In July 2019, a year before Shahid was disqualified, Ali* took out his phone to start his shift in London and found he couldn’t log in. He received a message from Uber, saying that his account had been flagged for violation of Uber’s terms and policies. “Upon review of your recent trips, we noticed a number of irregular trips associated with fraudulent activities,” the message read.
“Your account has been flagged for inappropriate use and has therefore been deactivated. Unfortunately, the deactivation is permanent.”
The father of four claims that the app is mistaken and, like Shahid, denies any wrongdoing. He was working as an Uber driver full time, and says he invested in a car worth £29,000 on finance a couple of months earlier — Uber had told him that his high rating on the app would allow him to drive an executive car, he says. After he was dismissed, he was stuck paying off a car he couldn’t use. “I must have emailed and called Uber fifty, sixty times in the last year and a half. I was going through hardship. I pleaded with them, I begged them and still I had no response,” Ali says.
Shahid and Ali are part of a group of four Uber drivers who have filed a case against the ride-hailing company this week, accusing it of “robo-firing” them by algorithm. The drivers — three in the UK and one in Portugal — claim that they were not involved in fraudulent activity and that Uber has not shared any of its evidence against them to justify dismissing them from its platform. Uber has not reported any of the four drivers to the police for fraudulent behaviour.
The legal claim, brought in the Netherlands where Uber’s data is held, argues the courts should overrule the algorithm that caused drivers to be dismissed, and force Uber to share the information that it gathered about them and used to ban them in the first place. None of the four drivers are named in the case and those that spoke to WIRED agreed to do so on condition of anonymity.
The lawsuit is the first of its kind that has been filed under the General Data Protection Regulation’s (GDPR) Article 22. GDPR, introduced by the European Union in 2018, protects user data and allows European citizens to demand information if they feel if they feel an algorithm – and an algorithm alone – independently made a decision about them.
If they are successful, a number of other legal challenges will likely follow. The App Drivers & Couriers Union (ADCU), which is bringing the legal challenge on behalf of the four drivers, says that since 2018 it has seen well over 1,000 individual cases where drivers have allegedly been wrongly accused of fraudulent activity and immediately had their accounts terminated with no right of appeal.
The ACDU representative for the drivers, James Farrar, argues that Uber can’t prove there was “meaningful” intervention by human beings in their bans because they were sent automated replies to their requests.
This argument is at the heart of the claim against Uber: according to the Information Commissioner, AI systems that only support or enhance human decision-making are not subject to these conditions. However, a decision will not fall outside the scope of Article 22 just because a human has “rubber-stamped” it: human input needs to be meaningful.
Both the ICO and the European Data Protection Board have previously stated that human reviewers must be actively involved in checking the system’s recommendation and should not “routinely” apply the automated recommendation to an individual; and that human involvement must be active and not just a token gesture. They should have actual influence on the decision, including the “authority and competence” to go against the recommendation, the guidance states.
Yet this complaint doesn’t factor in the new information fed into Uber’s algorithm. Farrar claims that automatic dismissals have increased since Uber introduced real-time ID verification technology last year to help resolve its licence issue with Transport for London (TfL). He says that TfL is “complicit” in this dismissal process because it pushed Uber to do real-time verification, which, in turn, caused Uber to “rush flawed technology and worsen the existing problem”. TfL declined to comment.
The real-time ID technology regularly asks drivers to take photos of themselves for verification. Those selfies are then matched against the account holder’s profile picture, which in turn has been checked against the driver’s license. If a driver fails the ID check, their account will be temporarily suspended pending further checks, and could then be suspended indefinitely.
This measure failed to satisfy the capital’s transport authority, which decided to strip the company of its London licence after it discovered that over 14,000 trips were taken with 43 drivers who had faked their identities on the app. The 41 drivers involved had their licences revoked by TfL (two were unlicensed). Uber has since won an appeal against TfL, gaining a temporary 18-month licence to operate in the capital. The September 28 judgement stated that the communication between Uber and TfL had improved, and that the court was satisfied with the improvements the company had made to its platform to safeguard customers.
But that communication has not necessarily extended to drivers, who have long complained about the opaque nature of Uber’s algorithm. It is this algorithm that underpins decisions on who gets jobs and how much they are remunerated, and rates driver profiles on matters such as inappropriate behaviour and late arrivals.
In January 2020, Uber implemented a loyalty scheme for drivers in the UK, offering free education courses, roadside assistance, gym memberships and discounted car maintenance in exchange for them accumulating “points” per ride. Drivers are encouraged to accept rides to reach each status: Blue (the basic level), followed by Gold (500 points in London, 600 elsewhere), Platinum (900 in London, 1,200 elsewhere) and Diamond (1,300 points in London, 1,800 elsewhere). Drivers say they don’t know how the algorithm uses their information on this tier system to determine who gets rides.
It’s also not clear what happens when drivers don’t want to participate. The ADCU points to Uber’s community guidelines, which define fraud to include declining work offered and strategically logging out to await higher surge pricing. The union has accused Uber of undercutting its obligation to driver’s worker rights by concealing performance related dismissals as fraud related dismissals.
During the pandemic, those that found themselves banned from the app have struggled to make up for lost income. “At the moment I work for Bolt, Ola and FreeNow,” says Ravi*, the third UK-based driver bringing the case against Uber. “The pandemic has been terrible. I get £27 for working from 5am and finishing at around 2pm. I used to earn much more, £150 to £200 a day.”
London-based drivers who are deactivated are given 14 days to explain to TfL why they should keep their licences. But they don’t know what they have been accused of, or how to prove their innocence. The two drivers involved in the GDPR claim launched this week say they were contacted by TfL during the summer, one year and two years after they were dismissed from the platform respectively.
“It’s weak regulatory oversight – asking a driver about something they did years ago is not really smart enforcement,” says Farrar. “While TfL ignores the working conditions of drivers, they’re actually worsening the compliance problem.”
*Names have been changed as drivers agreed to speak on condition of anonymity.
Natasha Bernal is WIRED’s business editor. She tweets from @TashaBernal
More great stories from WIRED
🇸🇪 Not every country treated the pandemic the same – did Sweden’s Covid-19 experiment work?
💬 This AI Telegram bot has been abusing thousands of women
🧥 Apple’s new phones have arrived: Should you get the iPhone 12 or iPhone 12 Pro?
🔊 Listen to The WIRED Podcast, the week in science, technology and culture, delivered every Friday
👉 Follow WIRED on Twitter, Instagram, Facebook and LinkedIn
Get WIRED Daily, your no-nonsense briefing on all the biggest stories in technology, business and science. In your inbox every weekday at 12pm UK time.
Thank You. You have successfully subscribed to our newsletter. You will hear from us shortly.
Sorry, you have entered an invalid email. Please refresh and try again.