Analytics

Step 1: ask clarifying questions

“Are we helping the environment?” could mean several different things depending on who’s asking and why:

• Are commuters actually replacing car trips with bike trips, or just logging bike trips they’d have taken anyway? → verified car-replacement trip rate
• How much CO2 is being avoided in absolute terms? → net CO2 avoided, computed from trips.co2_kg against a driving-baseline estimate for the same distance
• Is the impact sustained, or do commuters bike for a few weeks and then go back to driving? → downstream retention of car-replacement behavior
• Is air quality in the city actually improving? → a much harder causal claim, best checked cautiously against air_quality_readings trends, with the caveat that citywide air quality has many other drivers besides one commute program

Before writing a single query, ask what decision the city partner is trying to make — are they deciding whether to renew program funding, reporting to their own city council, or considering expanding to a new district? The answer changes which metric actually matters.

Step 2: propose concrete, measurable definitions

Once you know the decision behind the question, translate it into a specific, unambiguous metric definition — for example: “verified car-replacement bike trips per month, defined as bike trips in trips from commuters whose primary_mode was car at signup, on a route consistent with their prior commute” for the trip-based metric, or “estimated net CO2 avoided” for the emissions-based one.

Step 3: validate the definition with the stakeholder before building

Confirm the proposed metric actually answers their real question before investing in a dashboard or analysis — a common failure mode is building a technically correct CO2-avoided figure that doesn’t actually address what the city council cares about (e.g., they wanted district-level numbers, not citywide), because the translation step was skipped or rushed.

Step 4: avoid overbuilding

Resist the urge to report ten different environmental-impact metrics just because they’re all measurable from trips and air_quality_readings. Pick one primary metric that most directly answers the underlying question, with at most one or two supporting metrics for context — this keeps the eventual answer clear and actionable rather than a wall of numbers the city partner has to interpret themselves.

Why this matters generally

Most valuable analytics work starts with this translation step — vague language like “are we helping” into a precise, testable metric — and most analytics mistakes trace back to skipping it and guessing at what the stakeholder meant.

Why the numbers can legitimately differ

“Active bike commuter” is not a single, universally agreed-upon definition — it’s a design choice, and different teams often make different reasonable-sounding choices independently:

• What counts as a qualifying event? Any trips row with mode = 'bike'? Only trips that look like a car-replacement (matched against the commuter’s prior primary_mode)? A team focused on program impact might require the latter, while a team focused on app engagement might count any logged bike trip.
• What’s the time window? “Active this month” could mean a rolling 30-day window from trip_date or a fixed calendar month — these produce different numbers, especially near month boundaries.
• Is it deduplicated correctly? Counting distinct commuter_id values vs. counting raw trip rows in trips can produce very different totals if the underlying query isn’t using COUNT(DISTINCT commuter_id).
• Which commuters/cities are included? Test accounts used for QA, commuters not enrolled via employer_subsidy_enrolled, or one city’s pilot cohort — differing inclusion criteria alone can explain a large gap.

The fix: converge on one documented definition

Rather than trying to referee which team is “right” in the abstract, bring both teams together to agree on a single canonical definition, write it down explicitly (e.g., “an active bike commuter is a non-test commuter_id who logs at least one bike trip in trips within a trailing 30-day window”), and have every dashboard reference that same definition going forward.

Making it durable

A metric dictionary (or a semantic-layer metric definition that all city and employer dashboards pull from) prevents this problem from recurring — instead of every team writing their own SQL against trips and commuters with their own implicit assumptions about “active,” they all reference the same governed metric logic, so a change to the definition happens in exactly one place and propagates everywhere consistently.

The underlying lesson for leadership: this almost never means someone’s data pipeline is broken — it means two reasonable people made two different (undocumented) assumptions, and the fix is process (a shared definition), not a bug hunt.

Criteria for a good North Star metric

• Reflects real value delivered, not just company convenience. It should move when commuters are genuinely shifting away from driving, not just when the company does something that boosts a number in isolation, like a marketing push that increases app downloads without changing anyone’s commute.
• Leading, not purely lagging. Total subsidy dollars paid out is a real outcome but a poor North Star because it can be juiced short-term (raising subsidy amounts, enrolling commuters who were already biking) without reflecting whether the program is actually shifting anyone off driving.
• Actionable by the teams around it. Product, city-partnerships, and data teams should all be able to point to specific work that plausibly moves the metric — a North Star that’s too abstract, like “sustainability impact,” doesn’t help anyone prioritize.
• Simple enough to rally around. One number the whole company can hold in their head, not a dashboard of twenty KPIs pulled from trips, commuters, and air_quality_readings.

Why “verified car-replacement bike trips per week” fits

Counting a trips row with mode = 'bike' is not enough on its own — it needs to represent a trip that plausibly replaced a car trip a commuter would otherwise have taken (e.g., a trip logged by a commuter whose primary_mode was car at signup, on a route and time pattern consistent with their prior driving commute). That distinction is what makes the metric reflect genuine modal shift rather than just bike-app usage by people who were already biking anyway.

What to avoid

Picking total app downloads or total registered commuters as the North Star is a common mistake — it’s an easy number to report, but it doesn’t tell teams what to build the way a car-replacement metric does, and it can be inflated by marketing spend or a low-friction signup flow without any real commuting behavior changing.

Making it stick

A North Star metric works best paired with a small “metric tree” of supporting input metrics that feed into it — subsidy enrollment rate, week-over-week bike-trip retention, GPS/trip-logging data quality — so different teams (product, city-partnerships, data engineering) have a clear line of sight from their specific work to the company-level number, rather than everyone chasing one abstract “impact” target with no idea how their work connects to it.

Matching chart type to the question being asked

• Comparison across categories (average cost_usd or co2_kg per trip by mode — car, bike, bus, walk — right now): a bar chart, ideally sorted by value, lets the eye quickly rank and compare the four quantities. A pie chart forces the viewer to compare wedge angles/areas, which humans are demonstrably worse at judging precisely than bar length.
• Trend over time (total CO2 avoided by the commute program over two years, computed from trips.co2_kg against a driving-baseline estimate): a line chart shows direction, rate of change, and inflection points clearly — for example, a visible jump after a city added a new employer partner.
• Part-to-whole over time (how mode share has shifted across car, bike, bus, walk over two years): a stacked bar chart rather than multiple pie charts, since comparing pie charts to each other is even harder than reading one pie chart alone.
• Relationship between two variables (e.g., distance_km vs. cost_usd per trip, to see how cost scales with distance by mode): a scatter plot.
• Distribution of a metric (e.g., trip duration_minutes for bike commutes): a histogram.

Rules of thumb worth internalizing

• Avoid pie charts beyond 3-4 categories, and generally prefer bar charts for any precise comparison task — with exactly four modes, a bar chart is already the safer default.
• Avoid 3D charts and dual y-axes — putting cost_usd and co2_kg on two different axes on the same chart can visually exaggerate or hide how closely cost and emissions actually track each other by mode.
• Sort categorical bar charts by value, not alphabetically — sorting car/bike/bus/walk by cost or emissions makes the “biking is both cheaper and cleaner” takeaway instantly visible.
• Match granularity to the audience’s attention span. A city official with five minutes needs one clear chart with an obvious takeaway, not a dense multi-panel dashboard of every city and every month.

The underlying principle: chart choice should follow directly from the type of comparison the audience needs to make — composition, trend, relationship, or distribution — rather than from habit or whichever chart type looks the most polished.

What makes a metric “vanity”

A vanity metric typically:

• Only increases, never decreases — cumulative totals like total app downloads, total registered commuters, or total trips ever logged can’t go down, so they always look good regardless of what’s actually happening in the commute program right now.
• Doesn’t change your decisions. Knowing “2 million downloads” doesn’t tell a product manager whether to invest in better GPS trip-logging or a stronger subsidy incentive.
• Is easy to game or inflate without genuine impact — a marketing push or a low-friction signup flow can pump up total downloads without any corresponding increase in commuters actually replacing car trips.

What makes a metric “actionable”

• Moves in both directions based on real program changes, so it can actually signal whether something worked or didn’t — 90-day active car-replacement bike commuters can rise after a subsidy increase and fall after a city partner’s funding lapses.
• Ties to a decision. “90-day active car-replacement commuters dropped 5% after the subsidy renewal delay” tells you something concrete to investigate or fix.
• Comparable over time and across cities, so you can track trends and diagnose where a change is coming from in trips and commuters.

Applying it here

“2 million app downloads” tells the CEO nothing about whether the commute program is healthy today — it’s a running total that includes everyone who ever installed the app, including commuters who never enrolled in a subsidy or stopped biking years ago. “8% logged a verified car-replacement bike trip in the last 90 days” is a much more honest, decision-relevant number, because it reflects the actual current impact of the program, and it will move if GreenMile genuinely gets better or worse at converting downloads into real modal shift.

The internal conversation

It’s fine — even expected — for external marketing to lead with a big cumulative download number; that’s a legitimate (if slightly theatrical) way to signal scale to prospective city and employer clients. The risk is when the internal team starts believing the download number reflects program health and stops tracking the actionable one. The job here is to keep both numbers visible where they belong: total downloads on the homepage if leadership wants it there, and 90-day active car-replacement commuters squarely on the internal dashboards that drive product and city-partnership decisions.

Separating “what the data shows” from “what we should do”

The core move is to keep these two things distinct in the conversation: the analysis answers “how much CO2 did the subsidy program actually avoid,” and that’s not up for negotiation based on what the city official hoped to announce — but “what we do next” (expand the program, adjust the subsidy, wait for more data before a public claim) is a separate decision the official can and should weigh in on, informed by the honest result.

How to frame the pushback constructively

• Lead with the finding, plainly stated, before getting into methodology: “based on verified car-replacement trips this quarter, the program avoided roughly X kg of CO2 — not the larger number that assumes every logged bike trip replaced a car trip.” Don’t bury the answer in caveats before saying it.
• Show your work. Walk through exactly how CO2 avoided is estimated — which trips rows count as verified car-replacements, the baseline emissions assumption from vehicle_cost_reference and air_quality_readings, the time window — so the official can see the conclusion follows directly from a defensible method, not from a hidden agenda.
• Invite legitimate alternative hypotheses, without cherry-picking. If the official has a real reason to believe impact is understated in a specific district or commuter segment, that’s worth checking — but pre-register that as an honest hypothesis to test, not a search for any cut of the data that happens to look better, which is p-hacking regardless of how well-intentioned.
• Offer next steps rather than just a verdict. “Here’s the number the data supports today — do we want to expand the program to increase real impact, or revisit the assumptions together before any public number goes out?” turns a disappointing result into a forward-looking conversation.

Knowing when to escalate

If pressure continues specifically to reshape or inflate the CO2-avoided figure for a public announcement — rather than to explore legitimate alternative analyses — that’s a point to loop in a manager or preserve the original analysis in writing. Analytical credibility is the actual asset here; one instance of bending a number to please a city partner undermines trust in every future report GreenMile delivers to that city and others.

Building an intake process

The root problem with Slack-driven requests from city partners is that there’s no consistent triage — whichever account manager DMs loudest or most recently gets prioritized, regardless of actual business impact. A lightweight intake system (a ticket queue, a shared request form, or a structured Slack workflow) forces every request through the same triage criteria and creates visibility into total demand across all city and employer accounts, which is useful ammunition when advocating for more analyst headcount.

A simple triage framework

• Drop-everything tier: actively blocking a city council vote on subsidy funding happening this week, a live data issue affecting a client-facing report, or a request directly tied to an imminent press event or public announcement.
• This-week tier: important but not on fire — batch these (e.g., a city wants an updated mode-share breakdown for an internal review next month) and address them on a set cadence rather than interrupting focused roadmap work as they arrive.
• Self-serve/backlog tier: recurring or exploratory requests that don’t need a dedicated analyst — redirect the city partner to the existing client dashboard, or add to a backlog to be scoped into future roadmap work if there’s a pattern of repeat asks, like every city eventually wanting the same custom emissions-by-district cut.

Protecting roadmap time deliberately

Some teams formalize this with an explicit time allocation (e.g., “20% of analyst time reserved for city-partner ad hoc requests, 80% for roadmap”), which makes the tradeoff visible to leadership rather than an invisible tax on roadmap progress that only the analytics team feels.

Reducing repeat requests structurally

If the same type of ad hoc question keeps coming up across multiple cities — say, “how much CO2 did our specific district avoid this quarter” — that’s a signal to invest in a self-serve filter on the client dashboard so future instances of that question don’t need a dedicated analyst at all, turning a recurring ad hoc cost into a one-time roadmap investment.

Saying no gracefully

When something genuinely has to wait, the goal is to say so with a clear reason and a concrete alternative — “I can’t get to a custom cut until next sprint, but the client dashboard’s city filter should answer most of what you need in the meantime” — rather than either an unexplained no or reflexively saying yes and quietly falling behind on committed roadmap work.

Why calendar-time trends aren’t the right chart

A simple chart of “total bike trips logged over calendar time” conflates growth and retention — total bike trips can rise even while individual retention is getting worse, simply because new subsidy signups are outpacing the drop-off of earlier participants who stopped biking. That chart can’t answer “are participants sticking with biking,” only “has total volume changed,” which is a different (and less diagnostic) question.

The cohort retention curve

Group commuters by signup month (their “cohort,” from commuters.signup_date), then for each cohort plot the percentage who logged at least one bike trip in trips at day 1, day 7, day 30, day 90, etc. — measured relative to each commuter’s own signup date, not a fixed calendar date. Overlaying multiple cohorts (one line per signup month) on the same x-axis (days since signup) lets you directly compare whether a more recent cohort is retaining better or worse than an older one at the exact same point in their program lifecycle, isolating retention from overall program growth.

Reading the chart

• All cohorts start at 100% at day 0 by definition — every commuter who signs up for the subsidy is counted as active on day zero.
• A newer cohort’s curve sitting above an older cohort’s curve at the same day-since-signup is genuine evidence that retention is improving — for example, after GreenMile started sending a reminder push notification at day 5.
• A widening gap between curves over time suggests something specific changed (a subsidy amount increase, a new employer partner bringing in less-committed participants, a seasonal effect) worth investigating further.

Practical choices to get right

• Pick an “active bike commuter” definition consistent with realistic commuting cadence — a weekly window (at least one bike trip in trips per 7-day period) makes more sense than daily, since almost nobody bike-commutes every single day.
• Consider smoothing or curve-fitting (e.g., fitting a power-law decay) to project where long-term retention is likely to stabilize, which is often more useful to leadership than raw noisy data points at longer horizons with smaller cohort remainders.
• For the single slide, pick 3-4 representative signup cohorts (not all of them) to keep the chart legible, and lead with the headline takeaway in a sentence above the chart — e.g., “the March cohort is retaining 15 points better at day 30 than the cohort that joined before we added the reminder notification.”

Diagnosing the problem

A 30-chart dashboard covering every mode, every city, and every metric in trips and air_quality_readings usually signals it was built by exporting “everything that could plausibly be interesting” rather than by starting from a specific decision the city transportation department actually needs to make. The result is cognitively overwhelming — no one on the client side knows where to look first, and most of the charts are irrelevant to any given viewer’s actual job, whether that’s a budget request or an internal program review.

Principles for a dashboard people actually open

• Start from the decision, not the data. Interview the city team: are they deciding whether to expand the subsidy program to a new district, reporting mode-share progress to a city council, or justifying the program’s budget line? Build around that, not around every column available in trips and vehicle_cost_reference.
• Lead with one primary metric, top-left. For most city clients that’s something like “verified car-replacement bike trips this month” or “CO2 avoided (kg)” — immediately visible without scrolling or clicking.
• Limit scope to a handful of metrics with the ability to drill down into trips by city or mode for detail, rather than surfacing every possible cut up front. A good rule of thumb: if you can’t explain why each chart is on the dashboard in one sentence tied to a decision, cut it.
• Avoid vanity charts — cumulative total app downloads or total trips logged look impressive but don’t tell a city official anything about whether the program is working.
• Iterate with actual users, not just at launch — watch someone on the city team use it, or ask directly what they ignore versus what they actually check before a budget meeting.

How to know if the rebuild worked

Track dashboard usage itself (views, unique viewers, how often the city team returns) as a metric of the dashboard’s own health — if usage stays low even after a redesign, that’s a signal either the client doesn’t actually have a recurring decision this data should inform, or the dashboard still isn’t answering the right question, and it’s worth going back to a direct conversation with the city stakeholder rather than iterating blindly on chart design.

The underlying lesson

A dashboard is a communication tool built for a specific audience and a specific recurring decision — treating it as a comprehensive data catalog of everything in trips, commuters, and air_quality_readings instead is the most common reason expensively-built client dashboards go unused.

A structured investigation checklist

1. Rule out a data or instrumentation problem first. Before concluding anything real happened, check: did the app’s GPS trip-logging break on a recent release? Did an upstream data sync from trips fail or run late (leaving the dashboard showing an incomplete day)? Did a deploy change how bike trips get flagged vs. discarded? A “drop” that’s actually a logging or pipeline issue is extremely common and should be eliminated before spending time on a behavioral investigation.

2. Segment the drop. Break the metric down by:

• City — did one city in trips drop specifically, or is it broad-based across all of GreenMile’s markets?
• Employer partner — is it isolated to commuters at one employer (maybe their subsidy budget ran out, so employer_subsidy_enrolled flags lapsed), or spread across employers?
• Device/app version — is it isolated to one app platform, pointing to a broken trip-logging release?
• Day of week / weather — did the drop concentrate on specific days consistent with a storm or heat wave in one city, or was it spread evenly across the week?

3. Check external factors. Did a partner city pause its subsidy funding? Is there a seasonality effect (a holiday week, school break changing commute patterns)? Did a competitor bike-share program launch in one market?

4. Walk the trip-logging funnel step by step. If app opens are flat but logged bike trips in trips dropped, the issue is likely in the trip-logging or GPS-confirmation flow itself rather than in actual commuting behavior.

5. Communicate with an appropriate confidence level. Report what you’ve ruled out, what you’ve found, and your confidence in the explanation — it’s fine to say “still investigating, but ruled out a GPS logging bug and a city-specific subsidy lapse” rather than waiting for full certainty before saying anything.

Why this order matters

Starting with data-quality checks avoids the embarrassing (and common) outcome of building an elaborate “commuters are losing interest in biking” narrative for a drop that turns out to be a broken app release — always eliminate the boring explanation before reaching for the interesting one.

Why silence is the wrong instinct under pressure

The tempting shortcut under a tight deadline is to quietly work around the bug and ship the quarterly report to the city partner as if nothing happened, hoping it’s a minor issue. This is risky for two reasons: you might be wrong about how minor it is until you actually quantify it, and if the city’s own analysts discover the discrepancy later (which happens more often than people expect, since these reports get scrutinized by city council staff), it damages trust in GreenMile’s numbers — and in you specifically — far more than raising it proactively would have.

The right sequence

1. Quantify the impact first, even roughly. Re-run the CO2-avoided and cost-savings totals with the affected trips.distance_km values corrected or excluded — does it change the headline number by 2% or by 25%? This determines how urgent and how visible the issue actually is; not every data quality issue changes the bottom-line story the city cares about.
2. Flag it immediately to your manager or the city account owner, framed factually: what’s wrong with the GPS logging, how big the impact is on this quarter’s report, and what your options are — rather than either hiding it or catastrophizing before you’ve quantified it.
3. Decide together on the path forward: delay the report if the impact is large and a fix is feasible in time; ship with a clear caveat/footnote if the direction (bike trips are still saving CO2 and cost) doesn’t change even though the magnitude does; or push the deadline for just the affected city’s section while shipping the other cities’ numbers on time.

Documenting for the future

Once resolved, write down the root cause (e.g., the mobile app’s GPS logging double-recorded distance during brief connectivity drops) and the fix so the same issue doesn’t quietly resurface next quarter, and so anyone asking “why did last quarter’s CO2-avoided number get revised” has a clear answer.

The underlying principle

Deadline pressure is exactly the condition under which analytical integrity gets tested — the professional move is transparency about what you found and when, even if it means an uncomfortable conversation with a city partner the day before a deadline, rather than presenting emissions numbers you already know are inflated and hoping nobody notices.

Why single-touch attribution always causes fights

First-touch and last-touch attribution models are simple to compute but structurally guarantee conflict whenever a commuter interacts with multiple channels before enrolling — by design, one channel gets 100% of the credit and every other channel gets zero, regardless of how much each one actually contributed. If the paid social ad got the commuter’s attention and the employer HR email closed the enrollment, both “first touch = paid social” and “last touch = employer email” are defensible framings, and arguing about which one is “correct” is somewhat beside the point — neither one is measuring true incremental contribution.

Multi-touch models as an incremental improvement

Splitting credit across touchpoints — linear (equal credit to every touch), time-decay (more credit to touches closer to enrollment), or U-shaped (extra weight on first and last touch, some to the middle) — reduces the all-or-nothing conflict and is more defensible than single-touch models, but it’s still fundamentally rule-based: the weighting scheme reflects an assumption about how influence works, not a measured causal effect. A commuter seeing both a social ad and an employer email before enrolling is correlation, not proof that both caused the enrollment.

The gold standard: incrementality testing

To actually measure whether a channel is causing enrollments rather than just co-occurring with them, run a holdout experiment — randomly withhold a channel (pause paid social ads, or hold back the employer HR email) for a randomized subset of cities or employer partners, and compare enrollment rates in commuters (via employer_subsidy_enrolled) against the group that still sees the channel. The difference is the channel’s true incremental lift, independent of any attribution modeling assumptions.

The practical middle ground

Full incrementality testing for every channel and every employer partner is expensive and slow, so GreenMile can use data-driven/algorithmic attribution (weighting credit based on statistical patterns across many commuter journeys, rather than a fixed rule) as a pragmatic middle ground, while reserving incrementality tests for the highest-spend channels or the biggest employer accounts — and being transparent with both teams that attribution numbers are directional estimates, not ground truth, which helps defuse the “my channel deserves the credit” framing in the first place.

Why “usage vs. non-usage” comparisons are misleading

The natural first instinct is to compare car-replacement trip volume between commuters who used the instant e-bike signup and those who didn’t. But usage is self-selected — commuters who click through the new flow might already be more motivated to switch away from driving, or already lean toward biking per commuters.primary_mode, meaning any observed difference could reflect who chose to use the feature, not what the feature caused. This is the same self-selection confound that shows up in any “commuters who used X have better outcome Y” observational comparison.

The correct comparison: randomized exposure, not usage

If the instant signup flow was rolled out via an A/B test (some commuters have access, some don’t, assigned randomly), the valid comparison is: total verified car-replacement trips in trips for the test group (has access to the new flow, whether or not they use it) versus the control group (only has the old carpool-subsidy enrollment path). This comparison is protected by randomization — both groups should be statistically identical in every other way, so any difference in total car-replacement trips is attributable to the feature’s presence, not to who self-selected into using it.

• If total car-replacement trips in the test group are meaningfully higher than control, the instant e-bike signup is adding incremental impact — some commuters are switching off driving who wouldn’t have otherwise.
• If total car-replacement trips are roughly flat between test and control, despite high e-bike subsidy adoption within the test group, that’s strong evidence of cannibalization: commuters who would have enrolled in the carpool subsidy (and reduced driving) anyway are simply switching which subsidy they claim, with no net increase in trips shifted away from car.

Digging further

• Look at substitution patterns directly: among test-group enrollees, did commuters who newly enrolled in the e-bike subsidy show a corresponding drop in carpool-subsidy enrollment roughly matching the volume? A clean one-for-one substitution is a further confirming signal.
• Segment by employer or city: cannibalization might be concentrated among employers that already had a mature carpool program, while genuinely adding incrementality among employers or commuters with no prior subsidy participation at all.

Beyond features: the same logic applies to channels

This exact reasoning — usage/exposure comparisons are confounded by self-selection, randomized holdouts reveal true incrementality — is the identical logic behind measuring cannibalization or incrementality between marketing channels driving subsidy enrollment, not just between two program features.

Bottom-line-up-front

City officials with limited time need the answer before the reasoning: “after accounting for commute distance, city, and prior primary mode, the e-bike subsidy was associated with a real drop in car trips among enrolled commuters — smaller than we initially hoped, but concentrated among commuters who previously drove under 8 km each way.” Everything else in the conversation should support that one sentence, not precede it.

One visual, chosen for clarity not completeness

Rather than showing five coefficient estimates in a regression table, pick the single chart that most directly conveys the finding — e.g., a bar chart of average weekly car trips per commuter, before and after subsidy enrollment, for just the short-commute segment where the effect is concentrated. It’s fine (often better) to leave four other control variables out of the exec-facing visual entirely, even though they were essential to the underlying analysis.

Translating statistical language into plain language

• “Controlling for commute distance, city, and prior mode” becomes “even after accounting for differences in how far people commute and where they live” — same meaning, no jargon.
• “Statistically significant” becomes “we’re confident this isn’t just noise” (and only mention it if the official is likely to ask “are you sure”).
• Avoid “coefficient,” “p-value,” and “regression” entirely in the main narrative — these terms don’t add clarity for this audience and can make the finding seem less trustworthy, not more, if they land as jargon rather than substance.

Being ready to go deeper

Keep the full methodology — the regression specification, the control variables, robustness checks against GPS data-quality issues — ready to walk through if someone asks a pointed follow-up (“did you control for which commutes were already short enough to bike?”). The five-minute version isn’t a dumbed-down version of the truth; it’s the same finding with the supporting detail available on demand rather than front-loaded.

Using an analogy when it helps

For genuinely counterintuitive findings, a short analogy (“this is like only comparing commuters who live the same distance from work, so it’s a fair fight between drivers and bikers”) can do more to build intuition in thirty seconds than any amount of technical explanation — use it sparingly, and only when it actually clarifies rather than oversimplifies to the point of being misleading.

Reframe the outcome before presenting the number

A null result from a well-designed experiment is a successful experiment, not a failed one — the alternative (raising the subsidy citywide without ever testing it, based purely on conviction that more money moves more people to bike) is the actually risky path. Opening the conversation with “the test worked exactly as designed, and here’s what it told us” sets a very different tone than opening with “unfortunately, the bigger subsidy didn’t move the numbers.”

Establish methodology credibility upfront

Before presenting the null result itself, briefly re-establish that the experiment was well-powered and correctly designed (sample size of enrolled commuters, six-week duration, verified car-replacement trips chosen as the primary metric in advance) — this preempts the natural defensive reaction of “maybe the analysis is wrong” and keeps the conversation focused on what the (credible) result means, rather than relitigating the methodology under pressure.

Don’t over-hedge

There’s a temptation to soften a null result with so many caveats (“well, it’s not exactly significant, but there might be something there if we squint at one city…”) that it reads as inconclusive when the evidence is actually reasonably clear. If the confidence interval genuinely rules out a meaningful increase in car-replacement trips, say that plainly — false uncertainty is its own form of miscommunication and can lead to relitigating a settled question repeatedly.

Offer clear next steps, not just a verdict

A null result should come with a recommendation: hold subsidy spend at the current level and reallocate the budget difference elsewhere, iterate on a specific hypothesis for why a bigger subsidy didn’t move behavior (e.g., the real barrier is bike-lane safety, not cost, so test that instead), or, if the confidence interval was wide rather than tight, propose a larger or longer follow-up test before fully deciding. Leaving the VP with “here’s what I recommend and why” turns a potentially uncomfortable meeting into a forward-looking decision conversation.

The bigger picture for building trust

How a null result lands often determines whether a VP trusts future null results — or starts pressuring for a “yes” every time. Handling this one with credibility and a clear decision framing is what makes GreenMile’s experimentation program’s future negative results taken seriously instead of second-guessed or quietly buried.