Where algorithmic management meets binding labor protections

What this risk is

AI systems used to manage, monitor, evaluate, and control workers — determining their assignments, pay, performance ratings, and employment status — in ways that remove human managerial discretion and create working conditions characterized by surveillance, unpredictability, and absence of recourse.

Scale: Algorithmic management affects an estimated 70+ million gig economy workers globally (Uber, Deliveroo, Amazon, DoorDash, Instacart) plus a growing proportion of conventional employees subject to AI performance monitoring.

How it occurs · Mechanisms

Opacity

Workers cannot understand why they received a specific assignment, rating, or deactivation decision. The algorithm’s reasoning is not disclosed. There is no one to ask.

No Appeal

Automated decisions with significant employment consequences happen without human review. Formal appeal processes exist on paper but route to automated systems that confirm the original decision.

Surveillance Intensity

AI management systems monitor workers continuously — GPS tracking, keystroke monitoring, camera monitoring, call recording, facial recognition for identity verification. Workers are the most monitored employees in history.

Instability by Design

Algorithmic management systems often deliberately create income instability to maintain worker supply. Workers cannot predict earnings, making financial planning impossible.

Ratchet Effect

AI performance monitoring continuously tightens standards. What was above average performance becomes baseline, which becomes minimum acceptable. There is no floor.

Real-world incidents

Amazon Warehouse Monitoring

Amazon’s productivity tracking AI monitors picker and packer performance to the second, automatically generating productivity warnings and termination recommendations without human manager involvement. Workers report being fired by algorithm.

Uber Driver Deactivation

Uber’s AI deactivation system terminates driver accounts based on rating patterns, cancellation rates, and other metrics. Multiple documented cases of erroneous deactivation based on algorithmic errors, with limited human review.

Teleperformance and Call Center AI

Call center workers monitored by AI systems that track micro-expressions, voice tone, and speech patterns. Workers reported significant psychological stress from continuous AI surveillance and real-time performance feedback.

Deliveroo and the “Frank” Algorithm

Deliveroo’s order assignment algorithm (internally known as Frank) assigns orders based on rider performance metrics. Riders report that the algorithm penalizes them for accepting rest periods or working in weather conditions that reduce speed, creating safety incentives that prioritize algorithmic performance over rider welfare.

Mitigations · Governance

• Human review for consequential decisions — Deactivation, termination, significant rating changes require human review
• Explainability for workers — Workers have right to understand basis for AI decisions affecting their employment
• Appeal mechanisms — Genuine human review of contested algorithmic decisions
• Monitoring proportionality — Worker monitoring is proportionate to legitimate business need; workers are informed
• Collective consultation — Worker representatives consulted before deployment of AI management systems

—