On-Time Pickup Test

Waymo: Define an on-time pickup, and use it to compare a new routing algorithm with the current one.

Assignment

For one week in May 2025, Waymo tested a new routing algorithm in one city: each ride request was randomly handled by either the current router or the new one. The router decides which vehicle goes to the rider and shows the rider an estimated pickup time in the app. The rider operations team wants to know whether the new router improves on-time pickups, and has shared every ride request from the test.

There is no agreed definition of an on-time pickup yet, so choosing one is part of the task.

Deliverables

• Your definition of an on-time pickup: what the pickup is compared with, which ride requests are counted, and why.
• The on-time rate for each router, and the difference between them.
• A short closing report for the rider operations team: did the new router improve on-time pickups, and is there anything they should double-check before relying on the numbers?

Data Description

waymo_pickup_test.csv has one row per ride request:

• ride_id
• router - current or new
• requested_at - when the rider requested the ride
• promised_eta_min - the estimated time to pickup, in minutes, shown to the rider when they requested the ride
• vehicle_arrived_at - when the vehicle arrived at the pickup point
• rider_boarded_at - when the rider got into the vehicle
• cancelled_at - when the ride was cancelled by the rider

Practicalities

You can solve this in Python, SQL, or a mix of both — StrataScratch notebooks support both languages in the same notebook.

Assume this data comes directly from the dispatch system and hasn't been cleaned, so review it carefully before relying on the numbers. When something looks off, work out what it is and decide how to handle it; when a definition or a final decision is genuinely ambiguous, choose a reasonable option and explain why. There's often more than one valid way to compute a result, so explain the approach you take and the reasoning behind your choices.

What's assessed: how well your definition fits the question, whether you apply it correctly to the data, and how clearly you justify your choices and conclusion. There is no single correct number — a definition different from the one in our solution is fine if it is well argued and applied consistently.