Applied Research Scientist, Legal Domain

About the Role

As Applied Research Scientist, you will define how Arcline measures, evaluates, and improves its AI systems. You will own the full research loop — from problem framing and dataset design to running experiments, building evaluation infrastructure, and translating findings into product decisions. This is a role for someone who cares equally about rigorous methodology and shipping work that matters.

You will report directly to the CTO and work closely with AI and product engineers from day one. As the team grows, you will have the opportunity to build and lead a research function.

What You'll Do

Evaluation and Benchmarking

• Design and own evaluation frameworks for Arcline's AI systems — from document drafting quality to retrieval relevance and agent reliability
• Build the benchmarking infrastructure that lets the team run rigorous, reproducible experiments and track model performance over time
• Define what "good" looks like for legal AI outputs, working with lawyers to develop ground truth datasets and human evaluation protocols

Research and Experimentation

• Identify the highest-leverage research problems across agentic search, generation, and structured extraction in the legal domain
• Design and run structured experiments — agentic search architectures, prompting strategies, fine-tuning approaches — and translate results into clear, actionable recommendations
• Stay close to the frontier of LLM and agentic search research and bring relevant advances into Arcline's systems

Research Direction and Cross-functional Impact

• Set the research agenda: decide what problems are worth solving, in what order, and why
• Partner with lawyers and operations to surface failure modes and edge cases that quantitative metrics alone won't catch
• Translate findings into concrete product improvements, working directly with engineers to move from experiment to production
• Over time, define Arcline's research roadmap and build a research team around it

What We're Looking For

• 3+ years of experience in applied research or ML, with a track record of shipping work that measurably improved real systems
• Deep familiarity with LLM evaluation: designing metrics, building eval pipelines, measuring outputs that resist easy quantification
• Strong experience with agentic search systems — how retrieval, tool use, and generation interact at inference time, and how to evaluate them rigorously
• Solid statistical foundation: you know how to design experiments, interpret results, and avoid common pitfalls
• Experience in the legal domain — you understand how legal documents are structured, how legal reasoning works, and why that makes AI evaluation uniquely hard
• Strong communication skills — you can present ambiguous findings clearly to engineers, lawyers, and founders alike
• Comfortable setting direction and operating with high autonomy in an early-stage environment

Bonus Points

• Experience building eval infrastructure from scratch: annotation tooling, dataset versioning, automated regression testing for agentic pipelines
• Familiarity with fine-tuning or RLHF workflows for domain-specific models
• Background in legaltech, compliance, or regulated B2B SaaS
• Experience hiring or mentoring junior researchers
• Contributions to open-source evaluation frameworks or published work on NLP/LLM benchmarking

Why Arcline

• Direct access to both founders from day one — shape the research direction from scratch
• Set the standard for what rigorous AI evaluation looks like at an AI-native legal company
• High degree of trust and autonomy
• Hot market: we're building the defining AI-native firm in a $1T+ industry
• Clear path to building and leading a research function as the team grows
• Backed by Y Combinator — be part of the best startup network in the world, with regular opportunities to visit and work with the team in San Francisco

About the Interview

• Intro call with a founder
• Take-home assignment
• Final conversation with both founders