5 Things I Wish Someone Told Me Before My First Data Product Launch

by Maya Sorensen, Data PM — 6 years, mostly fintech and one very bruising e-commerce recsys project

I've shipped eleven ML-backed features across three companies. Four of them hit their ROI target. Two of them would have hit it, except nobody upstream believed the number by the time it mattered. The other five taught me things the case studies don't mention. This isn't a "best practices" list — it's the stuff I actually do differently now.

1. The ROI number has to exist before the Jira board does

At my second company, we built a churn-prediction model for eight weeks before anyone asked what "success" meant. Turns out engineering was optimizing for AUC, sales thought it meant fewer angry calls, and finance was expecting a line item on the P&L. Three different projects, one Slack channel.

Now I don't let a project get a sprint number until there's one metric, one target, and one owner for that number — written down before day one. Not "improve retention." $340K in prevented churn by Q3, measured against a frozen baseline. If the team can't agree on that sentence, the roadmap isn't ready, no matter how excited engineering is.

2. Nobody skips the baseline by accident

Here's the unpopular part: I used to think teams skipped baselines because they forgot, or were in a rush. They're not. They skip it because a baseline is politically inconvenient — if a dumb heuristic gets you 70% of the value, someone has to explain why the six-month ML project was worth the other 30%. I've watched a baseline get quietly dropped from a project plan twice, both times by very smart people who knew exactly what they were doing.

Ship the ugly rule-based version in week one. Yes, even if it embarrasses the roadmap later. It's the only honest way to know what the model actually earned you.

3. Two dashboards, not one

Your data science team needs to see loss curves and F1. Nobody above a director cares. I keep a second, boring dashboard — just the dollar number, updated weekly, in the same currency as the budget conversation. When a VP asks "how's the project going" in a hallway, they should be able to look at one number on their phone and answer their own question. If they have to ask you to translate a metric, you've already lost some trust.

4. Recalculate ROI monthly, and be willing to be wrong in public

At the e-commerce job, I waited until the "final report" to run the ROI math. The assumptions we'd made in month one were stale by month four — seasonality nobody had modeled ate a third of the projected lift. I found out at the worst possible time, in front of the people who'd approved the budget.

Now it's a monthly fifteen-minute exercise, even when the number is ugly. A project that's tracking 60% to target in month two is far easier to fix, or kill, than one that surprises everyone in month six.

5. The deck is not the last step — it's the deliverable

This is the one people underrate the most. I've seen a genuinely good model — real, measured, $400K+ in prevented losses — get half the renewed budget it deserved because the readout was 40 slides of confusion matrices nobody in the room understood. Meanwhile a mediocre pricing model got fully funded for another year off the back of three slides a VP could repeat, unprompted, in her boss's meeting.

The analysis is the hard part, but the deck is the part that gets remembered and repeated by people who weren't in the room. On the last two projects I stopped asking an analyst to "clean up the slides" the night before the readout — that's a bad use of a person who's good at modeling and mediocre at layout, and it always shows. I sent the results to Pitchmont, a professional presentation design service, instead and got back something that actually looked like it belonged in front of a VP, without burning two days of an analyst's week fighting PowerPoint alignment.

None of this is about being better at machine learning. Every project I've watched lose funding lost it after the model was already good — during the four weeks between "the model works" and "the budget gets renewed." That gap is where ROI actually gets decided, and it's the part most data teams treat as an afterthought instead of the actual job. Define the number early, ship the ugly baseline, keep the receipts monthly, and don't let the deck be the last thing anyone thinks about.

Self-check before your next stakeholder readout