Marketplace Product Case Study

How do you measure effectiveness of giving extra pay to delivery drivers during peak hours to meet the demand from consumers?

Here is a potential problem within delivery companies like Uber, Lyft and Instacart. How can DoorDash measure effectiveness of giving an incentive to drivers during peak demand? How can we go about answering this question if a business partner approaches a Data Scientist with this problem?

Clarify the Problem Statement

Is this a feature that is already available in DoorDash? If not, are we planning to launch and need to brainstorm if this feature makes sense and how do we go about implementing this?

If it is already launched, how long has this incentive been in place? If it available for all Dashers across all geographic locations?

Current pay for Dashers consist of base pay, commission and tip. Is this incentive part of commission?

Finally, what is a peak demand? Is it from 6-8pm every day or during special events like sports or incelment weather or only available on weekends?

Is this extra pay passed onto customers by higher fees? Typically, DoorDash incurs incentive pay and does not pass to customers.

Goal

Next we can clarify the goal of this incentive and how it ties to the mission. During peak demand, there may not be enough Dashers to make all deliveries and deliver meals and goods on time. When there aren’t enough Dashers to deliver, customers may receive cold food, longer delivery times, late deliberies or may have to pick up ordersthemselves. This in turn will negatively affect merchants who will experience a delayed deliveries and decline in reorder rate.

With that in mind, we can ask the business partner or the interviewer what is the goal of this incentive? Is it to attract new Dashers, increase engagement of current Dashers through bonus incentive or improve revenue for DoorDash? We can see that this incentive is closely tied to DoorDash’s mission of empowering local economies, making sure that a good customer experience improve local businesses to grow and for dashers to earn additional income.

Metrics

Assume this incentive is not yet launched how can we measure if there is an opportunity to introduce an incentive? Here are some metrics and data to compare peak to non-peak hour trends.

Hours required for delivery - We want to keep delivery durations low and maximize Dasher earnings. We may estimate 1000 hours of delivery demand during peak vs 600 hours required during non-peak where without mobilization efforts, DoorDash is undersupplied by extra 400 hours.

GMV (Gross Merchandise Volume) - If there is a large difference in total value of transactions/deliveries made during peak vs non-peak, then it makes sense to introduce an extra pay during peak hours.

Average delivery time could be higher during peak hours.

Order completion rate and driver acceptance rate would be higher in non-peak hours.

Once we get an idea of peak vs non-peak behavior, here are some metrics to mesure the success of this incentive.

Supply side metrics
Driver acceptance rate with or without an incentive. Higher acceptance during peak hours with an incentive indicate that surge pricing is working.

Driver availability or total number of available drivers should increase with the incentive.

We expect less deliveries are completed during a time of high demand without encoraging more drivers to log in or sign up to dash. With mobilization efforts, delivery completion rate should increase.

Marketplace Efficiency

Average delivery time should go down with the incentive compare to without the incentive during peak hours. Average time to completion could be calculated with total delivery hours divided by number of deliveries. Delivery time should go down as more dashers sign up.

% of late deliveries - Out hypothesis is that more dashers sign up or log in to Dash and are able to delivery on time or within estimated delivery threshold.

Additionally gather customer feedback via delivery rating from before and after introducing extra pay during a peak demand. Ideally, customer retention improves with introduction of this extra pay.

Monetization

Total GMV should significantly increase as DoorDash is able to make more deliveries with increasing Dasher fleet. The company makes more revenue as more dollar volume of transactions through the marketplace.

Time to Contribution Margin - If DoorDash spends $10 on Dasher at time 0, how long does it take to generate $10 in revenue.

Counter Metrics

Restaurants are swamped with high volume of on-premises dining during peak hours on a weekend. Food preparation time calculated as difference between order preparation start time and order ready to be picked up is high when there is a large demand.

Contribution to net earnings - If additional revenue from this surge pricing incentive does not outweight extra cost incurred from giving an incentive to dashers, perhaps this incentive is not financially sound.