harvey Data Scientist Interview: Questions, Experience & Prep (2026)

Harvey is a fast-growing legal AI company that builds generative AI tools used by major law firms and corporate legal teams worldwide. Their Data Scientist roles sit at the intersection of NLP, large language model (LLM) evaluation, and legal domain reasoning.

As of July 2026, knok jobradar tracked 937 Data Scientist openings across India. Harvey alone accounted for 367 of them, making it one of the most active hirers in this space right now. Bangalore leads with 166 openings across the broader market, followed by Delhi (46), Hyderabad (27), Pune (18), Mumbai (17), and Chennai (8).

Salary ranges for Data Scientists across India, based on knok jobradar data:

These figures cover the broader India market, not Harvey specifically. For Harvey-level numbers, Glassdoor or levels.fyi are better sources.

The Harvey Data Scientist interview process typically runs 3-5 rounds. Candidates report seeing a recruiter screen, a take-home or live coding exercise, an ML and statistics deep-dive, a system design or product case study, and a final values or leadership discussion. The full process commonly takes 2-4 weeks end to end.

These questions are drawn from publicly reported candidate experiences and the nature of Harvey's core products. Exact questions vary by team and level.

1. How would you evaluate the output quality of an LLM on a legal document summarization task?
2. Walk us through how you would fine-tune a language model for a domain-specific legal task with limited labeled data.
3. How do you handle class imbalance in a legal document classification problem?
4. Explain the difference between RLHF and direct preference optimization (DPO). When would you choose one over the other?
5. You have a retrieval-augmented generation (RAG) system returning inaccurate answers. How do you diagnose and fix it?
6. How would you design an A/B test to measure whether a new model version improves user satisfaction for a legal research tool?
7. A law firm says the contract review model is flagging too many false positives. How do you investigate and resolve this?
8. How do you balance model performance with latency constraints when deploying ML models in a production legal tool?
9. Describe a time you worked with unstructured text data. What preprocessing steps did you take and why?
10. How would you build a named entity recognition system to extract key clauses from contracts?
11. What metrics would you use to evaluate a legal citation recommendation model, and why?
12. How do you stay current with LLM research, and how would you decide whether a new technique is worth implementing at a product company?

Q: How would you evaluate the output quality of an LLM on a legal document summarization task?

*Situation:* At my previous role, we deployed an LLM-based summariser for financial reports. Business stakeholders were rejecting summaries without being able to explain what was wrong, and the team had no structured evaluation framework.

*Task:* I was asked to build an evaluation pipeline that could catch quality issues before summaries reached users.

*Action:* I defined a multi-dimensional rubric covering factual accuracy (no hallucinated clauses), completeness (key entities and dates preserved), fluency, and length compliance. I combined automated metrics such as ROUGE and BERTScore with a human annotation layer where domain experts rated a sample of outputs weekly. I also built adversarial test cases using documents with subtle contradictions to stress-test the model.

*Result:* We caught a hallucination issue affecting a specific document template before it went live. Human annotation scores improved consistently after we used the rubric to create targeted fine-tuning data. For Harvey, I would extend this by adding legal-specific checks: citation accuracy, clause completeness, and jurisdiction correctness.

---

Q: You have a RAG system returning inaccurate answers. How do you diagnose and fix it?

*Situation:* At a previous company, our internal RAG-based Q and A tool started returning answers that were factually off, especially for edge-case queries that combined two policy topics.

*Task:* I needed to identify whether the issue was in retrieval, generation, or the underlying data itself.

*Action:* I split the pipeline into three diagnostic layers. First, I checked retrieval quality by logging the top-k chunks returned per query and manually reviewing them for relevance. Second, I tested whether the generator was correctly grounding its answers in the retrieved context by running the same context through a smaller, more predictable model. Third, I audited the vector store for stale or duplicate documents. The root cause turned out to be a retrieval issue: our chunking strategy was splitting policy sections at paragraph boundaries that broke context continuity.

*Result:* Switching to a sentence-window chunking strategy with overlap reduced answer errors by a meaningful margin on our internal test set. I would apply the same layered diagnostic approach at Harvey, adapted to the structure of legal documents.

---

Q: A law firm says the contract review model is flagging too many false positives. How do you investigate and resolve this?

*Situation:* This mirrors a situation I faced when our entity extraction model was over-triggering on boilerplate clauses in vendor agreements.

*Task:* I had to reduce false positives without significantly increasing false negatives, because missing a real issue had higher business cost than surfacing an irrelevant flag.

*Action:* I pulled a sample of false-positive cases and clustered them by clause type. I found that a specific set of standard indemnity clauses were consistently misclassified. I retrained the model with additional negative examples from this cluster, adjusted the confidence threshold for that clause category specifically, and added a rule-based post-filter for known boilerplate patterns.

*Result:* False positives on that clause type dropped noticeably in internal testing. I presented precision-recall tradeoff curves to the business team so they could make an informed decision on the final threshold, rather than me picking it unilaterally.

STAR for behavioral questions. Structure every 'tell me about a time' answer as Situation (one sentence of context), Task (what you were responsible for), Action (what you specifically did, using 'I' not 'we'), and Result (measurable or observable outcome). Harvey interviewers care about individual contribution, not team effort.

The 'diagnose before you fix' pattern for technical cases. When given a broken system or a product complaint, interviewers want to see that you decompose the problem before jumping to a solution. State your hypotheses, explain how you would test each one cheaply, and then describe the fix. This shows engineering discipline and signals you will not waste compute on the wrong problem.

The tradeoff narration pattern. Harvey builds products that law firms pay for, so every technical decision has a business consequence. When answering design questions, name the tradeoff explicitly: 'a higher recall threshold catches more risky clauses but increases the review burden on the lawyer.' Show you understand who is downstream of your model.

For LLM and NLP questions. Name the evaluation metric and explain why it fits the task. For generation tasks, mention both automated metrics and human evaluation. For retrieval tasks, mention precision at k and mean reciprocal rank. Interviewers at LLM-native companies expect fluency with these terms, not vague references to 'accuracy'.

Harvey is an AI-first company selling to a risk-averse, detail-oriented customer: lawyers. This shapes what interviewers look for in Data Scientist candidates.

Domain awareness, without pretending to be a lawyer. You do not need a law degree, but you need to understand why hallucinations are catastrophic in a legal product, why precision matters more than recall in some clause-detection tasks, and why explainability is not optional when a partner at a firm is reviewing your model output.

Genuine depth in LLMs and NLP. Candidates report that surface-level answers about transformers are not enough. Expect to discuss fine-tuning strategies, evaluation design, RAG architecture, and prompt engineering trade-offs with real specificity.

Product thinking alongside technical skill. Harvey interviewers typically want to see that you connect model decisions to user outcomes. A Data Scientist who can say 'this threshold change will reduce lawyer review time by removing low-confidence flags' is more compelling than one who optimises F1 in isolation.

Clear, structured communication. Lawyers are your end users and sometimes your interview evaluators. Being able to explain a model decision in plain language is a practical skill at Harvey, not a nice-to-have.

Week 1: Core ML and statistics review. Refresh classification metrics (precision, recall, F1, AUC), probability fundamentals, and Bayesian thinking. Practise explaining bias-variance tradeoff and regularisation in plain language, since Harvey interviewers may push back to test your depth.

Week 2: NLP and LLM depth. Review transformer architecture, attention mechanisms, and common fine-tuning approaches such as LoRA and instruction tuning. Read up on RAG pipelines: chunking strategies, embedding models, and re-ranking. Study evaluation frameworks for generative models covering BLEU, ROUGE, BERTScore, and human annotation design.

Week 3: Harvey-specific context. Read Harvey's publicly available blog posts and product announcements. Understand what their core products do: legal research, contract review, and document drafting assistance. Think about the failure modes that matter most in these products and how you would measure and mitigate them.

Week 4: Case practice and mock interviews. Practise 3-4 end-to-end case answers using the 'diagnose before fix' structure. Do at least two mock technical interviews with a peer. Record yourself answering behavioral questions and check that you are using 'I' not 'we' throughout.

Ongoing: knok checks 150+ job sites nightly, applies to jobs matching your resume, and messages HR for you, so your applications stay active while you focus on prep.

Treating LLM questions as general ML questions. Harvey is an LLM-native company. Answering a question about model evaluation with 'I would use accuracy and F1' without discussing hallucination, groundedness, or human evaluation signals that you have not thought about the specific challenges of generative AI.

Ignoring the legal product context. Candidates who give purely technical answers without acknowledging the stakes for legal professionals come across as a poor fit. A Data Scientist at Harvey is not just building models; they are building tools lawyers stake their professional reputation on.

Saying 'we' throughout behavioral answers. Interviewers cannot evaluate your individual contribution if every sentence uses 'we'. Use 'I' and be specific about your decisions, even when the work was collaborative.

Jumping to solutions before diagnosing. In case and system-design questions, interviewers watch for whether you ask clarifying questions and state assumptions before recommending a fix. Jumping straight to 'I would retrain the model' without diagnosing the root cause is a red flag.

Underselling evaluation design. Many candidates focus on model architecture and say little about how they would know if the model is working. At a product company like Harvey, evaluation is as important as the model itself. Be ready to design an end-to-end eval pipeline, not just name a metric.