Software Release Safety

Zoox: Define a safety metric for test drives, and use it to compare two software releases.

Assignment

In July 2025, Zoox test vehicles drove with two versions of the driving software: the current release, 6.2.0, and a new release, 6.3.0. The safety team has to decide whether 6.3.0 is at least as safe as 6.2.0 before it goes to more vehicles, and has shared the test drives and the events recorded during them.

There is no agreed safety metric for comparing releases yet, so choosing one is part of the task.

Note: a disengagement is when the safety operator in the vehicle takes over control from the software. A hard brake is a sudden, strong braking by the software. A contact is any physical contact with another object, however minor.

Deliverables

• Your safety metric: which events it counts, what it is measured against (for example, distance or time), and why.
• The value of your metric for each release, and what it says about 6.3.0 compared with 6.2.0.
• A short closing report for the safety team: is 6.3.0 at least as safe as 6.2.0, and is there anything they should double-check before relying on the numbers?

Data Description

zoox_test_drives.csv has one row per test drive:

• drive_id
• software_release - 6.2.0 or 6.3.0
• drive_date
• road_type - city or highway
• miles - distance driven
• hours - time driven

zoox_drive_events.csv has one row per event recorded during a drive:

• event_id
• drive_id
• event_type - disengagement, hard_brake, or contact

Practicalities

You can solve this in Python, SQL, or a mix of both — StrataScratch notebooks support both languages in the same notebook.

Assume this data comes directly from the vehicles' logging system and hasn't been cleaned, so review it carefully before relying on the numbers. When something looks off, work out what it is and decide how to handle it; when a definition or a final decision is genuinely ambiguous, choose a reasonable option and explain why. There's often more than one valid way to compute a result, so explain the approach you take and the reasoning behind your choices.

What's assessed: how well your definition fits the question, whether you apply it correctly to the data, and how clearly you justify your choices and conclusion. There is no single correct number — a definition different from the one in our solution is fine if it is well argued and applied consistently.