Senior Data Scientist – Experimentation & Causal Inference

Headcount: 1
Level: P6
Location: US R&D Center (collaborating with global teams across APAC)

About the Role

We are seeking a Senior Data Scientist – Experimentation & Causal Inference to serve as the statistical authority behind our rapidly expanding experimentation platform. This role is responsible for ensuring that every experiment we run is trustworthy, sensitive, and correctly interpreted.

You will own the methodology layer of experimentation—not the infrastructure. While our platform engineering team builds and operates the experimentation platform, you will design, validate, and continuously improve the statistical frameworks that power it.

You will be the expert the organization relies on when asking:

"Can we trust this result?"

More importantly, you will build the methodologies and automated validation frameworks that answer that question before it needs to be asked.

Team Overview

You will join a multidisciplinary team of data scientists and platform engineers within the Big Data organization, reporting directly to the Head of Big Data.

The experimentation platform engineering team will implement your statistical specifications into production systems, allowing you to focus on methodology, experimentation quality, and causal inference.

What You'll Do

1. Own Experiment Quality & Guardrail Frameworks

Design and maintain automated anomaly detection systems for live experiments, including:
- Sample Ratio Mismatch (SRM) detection
- Traffic allocation validation
- Experiment integrity monitoring
Define alert thresholds and circuit-breaking criteria to identify or stop compromised experiments before they influence business decisions.
Develop and validate guardrail metrics that balance:
- Sensitivity
- Directionality
- Interpretability
Establish experimentation standards and best practices across the organization.

2. Improve Experiment Sensitivity Through Variance Reduction

Design and implement variance reduction methodologies such as:
- CUPED
- Covariate adjustment techniques
- Pre-experiment normalization methods
Reduce metric noise caused by historical user behavior and external factors.
Enable faster and more reliable detection of treatment effects, particularly in:
- Low-traffic environments
- High-priority product launches
- Time-sensitive business decisions
Drive measurable improvements in experimentation efficiency, with a target of reducing average experiment duration from 14 days to 9 days and significantly increasing experimentation throughput.

3. Lead Continuous A/A Testing & Data Quality Monitoring

Design and operate continuous A/A testing programs as a permanent health check for the experimentation ecosystem.
Validate the integrity of the end-to-end measurement pipeline, including:
- Client-side instrumentation
- Event collection systems
- Message queues
- Real-time data warehouses
Monitor:
- Baseline metric stability
- Pipeline reliability
- Instrumentation quality
Define what constitutes a statistically healthy experimentation environment and provide specifications for engineering teams to operationalize monitoring.

4. Partner on Experiment Design & Causal Inference

Collaborate with Product, Engineering, Growth, and Leadership teams on:
- Experimental design
- Sample size calculations
- Power analysis
- Metric selection
- Duration estimation
- Result interpretation
Provide guidance on causal inference methodologies when randomized experimentation is not feasible, including:
- Difference-in-Differences (DiD)
- Synthetic Control
- Regression Discontinuity Design (RDD)
Ensure business decisions are supported by rigorous statistical reasoning.

5. Enable Scalable Self-Service Experimentation

Translate statistical methodologies into:
- Technical specifications
- Validation scripts
- Decision frameworks
- Best-practice guidelines
Empower experiment owners to self-serve while maintaining scientific rigor and consistency.

Required Qualifications

Education

Master's or PhD in:
- Statistics
- Biostatistics
- Economics (Econometrics)
- Computer Science
- Mathematics
- Or a related quantitative discipline

Experience

3+ years of industry experience designing and analyzing online controlled experiments at scale within a technology company.
Demonstrated experience working with large-scale digital experimentation programs involving meaningful user traffic.
Experience beyond survey-based or clinical trial experimentation is strongly preferred.

Statistical Expertise

Strong foundation in:

Hypothesis testing
Power analysis
Multiple testing correction
Sequential testing
Experimental design

Hands-on experience with at least two of the following:

CUPED or other variance reduction techniques
Sample Ratio Mismatch (SRM) detection
Continuous experimentation quality monitoring
Experiment health diagnostics

Technical Skills

Advanced Python proficiency, including experience with:
- SciPy
- Statsmodels
- NumPy
- Equivalent statistical libraries
Strong SQL skills, including:
- Window functions
- Common Table Expressions (CTEs)
- Data validation and quality analysis
Experience building simulations and statistical validation frameworks.

Communication & Collaboration

Ability to explain complex statistical concepts to technical and non-technical stakeholders.
Strong capability to translate business questions into rigorous experimental methodologies.
Comfortable writing clear specifications that engineers can implement in production systems.

AI-Native Workflow

Extensive hands-on experience using AI-assisted development tools such as:
- Claude Code
- OpenClaw
- Cursor
- Similar AI coding assistants
Proven ability to leverage AI tools to:
- Accelerate analysis
- Automate repetitive workflows
- Improve productivity
- Build self-service tooling

Language Requirements

Fluent in both English and Chinese.
Ability to collaborate effectively with engineering, product, and data teams across APAC and North America.
Comfortable serving as a communication bridge between the US R&D Center and regional teams.

Working Hours Flexibility

This role requires occasional cross-time-zone collaboration with teams across:
- Singapore
- Dubai
- Other APAC locations (UTC+4 to UTC+8)
Candidates should be comfortable joining important meetings during early mornings or evenings when necessary.

Preferred Qualifications

Experience with Bayesian experimentation frameworks.
Familiarity with multi-armed bandit methodologies.
Understanding of streaming data ecosystems and their impact on experimentation validity, including:
- Kafka
- Flink
- Real-time analytics architectures
Publications or conference presentations related to:
- Experimentation
- Causal inference
- Statistical methodology
- Machine learning evaluation
Experience in:
- Fintech
- Cryptocurrency
- Trading platforms
- Other high-scale consumer technology environments

Why Join Us?

Define Experimentation Standards at Massive Scale

You will establish the statistical foundations for experimentation across a platform serving 80M+ users globally. Your methodologies will directly influence product strategy and business decisions.

Immediate and Measurable Impact

With more than 120 active experiments and ambitious growth plans, your work will be deployed quickly and deliver visible business outcomes.

Science-Driven Decision Making

We run experiments to discover the right answers—not to validate predetermined decisions. You will have direct visibility with leadership when experimental findings challenge conventional thinking.

High-Caliber, Low-Bureaucracy Environment

Work alongside a focused team of data scientists and engineers where responsibilities are clearly defined:

You own the science.
Engineers own the platform.
Great ideas move quickly into production.

This is a rare opportunity to shape the experimentation culture, methodology, and decision-making framework of a global-scale technology company.

Senior Data Scientist- Experimentation & Causal Inference