Senior Data Scientist
Senior Data Scientist
lmarena
LMArena is seeking a Senior Data Scientist to analyze large datasets that inform AI model evaluations. The role involves collaborating with ML researchers to design experiments and improve AI evaluation systems. The company values craftsmanship, curiosity, and impact, and is composed of experts from prestigious institutions.
Qualification
- 6+ years of experience in data science, ML analytics, or applied research, preferably in AI, ML, or large-scale data environments.
- Strong proficiency in Python, with deep experience in Pandas, NumPy, and distributed frameworks like Spark.
- Expertise in statistical modeling, causal inference, and experimental design.
- Experience reasoning about data distributions, sample quality, and the effects of data distribution shifts.
- Strong communication skills.
Responsibility
- Explore and analyze large, complex datasets to uncover patterns, biases, and causal relationships in model behavior and system performance.
- Formulate hypotheses about data quality, evaluation outcomes, and model performance, then design experiments to validate or refute them.
- Build reproducible analysis pipelines using Python, Pandas, NumPy, and Spark to process and interrogate large-scale data.
- Partner with ML researchers and engineers to design metrics and analyses that evaluate how models perform across domains, prompts, and tasks.
- Develop causal reasoning frameworks and statistical methods that help explain why models behave as they do, not just how well they perform.
- Communicate insights clearly to technical and non-technical partners, informing both research direction and infrastructure improvements.
