Client Case Evidence & Advisory Reviews

Extended Engagement Case Studies

Case Study 1: Resolving a Hidden SRM in an E-Commerce Checkout Overhaul

• Client Organization: Multi-national Marketplace App (Southeast Asia Operations)
• Problem Statement: The internal product dashboard reported a +5.4% lift in checkout completions for a newly introduced multi-step payment screen. However, finance reported zero corresponding increase in settled transaction gross revenue.
• Consultant Investigation: Flow Harbor Point ingested 1.8 million event records across 21 test days. Our (\chi^2) diagnostic detected a significant Sample Ratio Mismatch ((p = 0.0004)) concentrated entirely on Android devices running operating system versions older than API level 28.
• Root Cause & Outcome: A client-side JavaScript animation exception on older Android WebViews caused the checkout button event to fire repeatedly during page reloads, while the transaction itself failed to reach the server. Our audit revealed that true net conversion actually regressed by -1.8%. The client halted the rollout, avoiding substantial customer transaction loss.

Reported Dashboard Metric:    +5.40%  (p = 0.012) -> Corrupted by Retry Telemetry
Flow Harbor Point Verified:   -1.82%  (p = 0.041) -> Post-SRM Correction
Operational Impact:           Halted deployment of broken Android payment module.

Case Study 2: Variance Reduction (CUPED) in a Low-Volume FinTech Onboarding Flow

• Client Organization: Institutional Wealth & Robo-Advisory Mobile App
• Problem Statement: A critical KYC verification redesign had been running for 6 weeks with inconclusive statistical significance ((p = 0.14), 95% CI [-0.4%, +3.1%]). Product management was debating whether to abandon the engineering work.
• Consultant Investigation: Dr. Vongviphas applied CUPED variance modeling using 30 days of pre-experiment user engagement covariates (previous session counts, app open frequency, and account verification initiation signals).
• Outcome: The variance reduction narrowed confidence interval width by 38%, resulting in an adjusted observed lift of +2.45% ((p = 0.018), 95% CI [+0.41%, +4.49%]). The client successfully shipped the onboarding improvement with verified statistical confidence.

Verified Practitioner Testimonials

Below are direct evaluations provided by engineering leads, analytics heads, and product directors who commissioned our practice:

1. Mobile Subscription Funnel Evaluation

“During our annual subscription pricing test, our team observed conflicting signals between our in-house logging database and third-party attribution tools. Flow Harbor Point performed a complete log reconciliation within four business days. Their formal verification brief isolated an event deduplication flaw that was inflating our reported trial conversions. Having their signed mathematical brief gave our executive committee complete confidence to recalibrate our forecast.”

— Somchai Prasert, Lead Product Manager, FinTech Mobile App (Bangkok, TH)

2. Pre-Experiment Power & Runtime Sizing

“We engaged Flow Harbor Point to review our statistical power calculations before launching a major search algorithm test. Danai Siriporn identified that our intended 7-day test duration would fail to account for weekend cyclicality and had only a 45% probability of detecting our target MDE. Sizing the test to 18 days with CUPED covariate adjustments enabled us to reach decisive results on the first run.”

— Marcus Vance, VP Engineering, Digital Commerce Platform (Singapore)

3. Multi-Metric Guardrail & Risk Assessment (With Process Note)

“The mathematical rigor and analytical depth of the audit report were exemplary. The only friction we encountered was the stringent intake formatting required for raw log tables before their team could begin ingestion, which took our data engineering team two full days to extract. However, once ingested, their identification of an unnoticed +0.18% memory leak on iOS variants completely justified the preparation effort.”

— Ananya Chantaraphat, Head of Product Analytics, On-Demand Delivery Network (Hat Yai, TH)

4. Bayesian Posterior Decision Modeling

“When dealing with low-traffic B2B enterprise workflow tests, traditional p-value null hypothesis testing frequently left us in statistical limbo. Flow Harbor Point introduced a Bayesian framework with customized domain priors that gave our executive board clear risk-weighted probabilities of superiority instead of binary pass/fail statements.”

— Elena Rostova, Director of Data Science, Enterprise Logistics Application

Inquire About Case References

If you would like to discuss specific methodologies applied in these engagements or review anonymized sample verification briefs, please contact our practice office.