Cresta Data Scientist Interview: Questions & Prep (2026)

Cresta is a real-time AI coaching platform for contact centres, built on large language models and conversation intelligence. The company helps customer service and sales teams improve agent performance by surfacing insights during live calls. With 111 open roles tracked on knok as of mid-2026, Cresta is actively building out its data science function across NLP, model evaluation, and production ML systems.

Candidates report a process that typically spans three to five rounds: an initial recruiter screen, a technical phone interview, a take-home or live coding assessment, and one or two final rounds with senior data scientists or engineering leads. Timelines vary, but many candidates hear back within two to three weeks of the final round. The focus throughout is on applied ML, product thinking, and your ability to communicate findings to non-technical stakeholders.

These are the questions Cresta interviewers most commonly ask Data Scientist candidates, based on what candidates typically report.

1. Walk me through how you would build a model to classify the intent behind a customer service conversation in real time.
2. Cresta surfaces suggestions to agents during live calls. How would you measure whether those suggestions are actually helping?
3. You are given a dataset of call transcripts labelled with agent performance scores. How would you approach feature engineering?
4. How would you handle class imbalance in a dataset where most calls are routine but escalations are rare?
5. Explain the difference between precision and recall. In the context of flagging coaching opportunities for agents, which matters more and why?
6. How would you design an A/B test to evaluate a new NLP model in a live contact centre environment where call volume varies by hour?
7. Describe a time you had to retrain or update a model in production. What triggered the decision and how did you manage the rollout?
8. How would you detect and address data drift in a model that processes live conversation transcripts?
9. Cresta works with large language models. How would you fine-tune or prompt-engineer an LLM for a specific customer vertical without overfitting to that vertical's quirks?
10. You notice that model accuracy is high on the test set but agents are ignoring the suggestions in practice. How do you diagnose and fix this?
11. How do you explain a complex model recommendation to a customer success manager who has no ML background?
12. Walk me through your approach to validating ground truth labels when annotation is done by human reviewers with varying levels of expertise.

Q: How would you measure whether Cresta's real-time suggestions are helping agents?

*Situation:* At my previous company, we built a recommendation engine that surfaced product suggestions to sales reps during calls, and we faced the exact challenge of proving business value.

*Task:* My task was to design a measurement framework that separated the effect of the recommendations from natural agent skill variation.

*Action:* I set up a randomised holdout experiment where a subset of agents received suggestions and the rest did not. I tracked four metrics: suggestion acceptance rate, average handle time, customer satisfaction score collected post-call, and first-call resolution rate. I also built a dashboard so team leads could see the data in near real time rather than waiting for weekly reports.

*Result:* Within the experiment window we had statistically significant evidence that agents using suggestions resolved calls faster. The experiment design was later adopted as the standard evaluation template for all new features.

Q: How do you handle class imbalance in a high-stakes classification task?

*Situation:* I was building an escalation-prediction model for a client where only a small share of calls ended in escalation, so a naive model simply predicted 'not escalated' every time and looked accurate on paper.

*Task:* I needed a model that could reliably catch true escalations without burying agents in false alarms.

*Action:* I tried three approaches in parallel: oversampling the minority class with SMOTE, adjusting class weights in the loss function, and threshold tuning on the probability output. I evaluated each using F1 on the minority class and precision-recall curves rather than overall accuracy. I also ran a cost-benefit analysis with the operations team to decide where to set the threshold, because a missed escalation was far more costly than a false positive.

*Result:* The class-weight adjustment combined with threshold tuning gave the best balance. Recall on true escalations improved substantially while the false positive rate stayed within the range the operations team could manage. The model went to production and reduced escalation incidents over the following quarter.

Q: Describe a time you had to explain a complex model to a non-technical audience.

*Situation:* I had built a transformer-based intent classifier and needed to present findings to a customer success team that had no ML background.

*Task:* My goal was to get their buy-in to integrate the model output into their workflow, so I needed them to trust it without drowning them in technical detail.

*Action:* I dropped all discussion of model architecture and instead built a simple visual showing real examples: a call transcript on the left, the model's predicted intent in the middle, and what the agent should ideally do next on the right. I also prepared a one-page 'confidence guide' explaining that the model flags low-confidence predictions separately so agents know when to rely on their own judgement.

*Result:* The team agreed to a pilot without any pushback on the technology. They later told me the real-example format was what made it click for them. The pilot ran successfully and the model was rolled out to the full team.

STAR for behavioural questions. Structure every story as Situation, Task, Action, Result. Keep the Situation and Task brief (two to three sentences combined) and spend most of your time on Action and Result. Cresta interviewers care about how you think and what you shipped, not the backstory.

The 'so what' test for technical questions. After every technical explanation, add one sentence connecting your answer to business impact. For example, after explaining data drift detection, say what happens to agent performance if drift goes unaddressed. This shows product awareness, which Cresta values highly.

Structured problem-solving for open-ended ML design questions. Follow a consistent order: clarify the business goal, define success metrics, describe the data you would need, outline your modelling approach, then discuss evaluation and deployment. Saying 'let me structure my answer' before starting signals confidence rather than hesitation.

Handling gaps in knowledge. If you have not worked with a specific technique, say so briefly, then describe how you would learn or what adjacent experience you would draw on. Cresta teams work with cutting-edge LLM tooling, so intellectual honesty about knowledge gaps is better received than bluffing.

Cresta sits at the intersection of NLP, real-time inference, and enterprise software, so interviewers look for a few specific signals beyond raw technical skill.

Product curiosity. Can you connect a modelling decision to the experience of an agent on a live call? Candidates who ask clarifying questions about the end user tend to stand out.

Production mindset. Cresta's models run in real time during customer calls. Interviewers want to see that you think about latency, monitoring, and rollback, not just offline accuracy.

Communication clarity. Data scientists at Cresta regularly present findings to customer success managers and enterprise clients. The ability to simplify without dumbing down is a direct job requirement.

Collaboration signals. Candidates report that interviewers ask questions like 'how did you handle disagreement with an engineer on model design?' Look for opportunities to show you can give and receive technical feedback without ego.

Comfort with ambiguity. Contact centre data is messy: noisy audio, inconsistent labelling, varied call volumes by industry. Interviewers often present underspecified problems on purpose to see if you ask the right clarifying questions.

Week 1: Foundations and company context.

Read Cresta's published research and product blog to understand their core NLP and LLM use cases. Practice explaining transformer architectures, attention mechanisms, and fine-tuning in plain language. Review your own past projects and identify two or three that involved real-time systems, NLP, or production ML deployments.

Week 2: Applied ML and statistics.

Practice end-to-end ML design questions out loud. Cover evaluation metrics for imbalanced datasets, A/B testing in production environments with unequal traffic, and detecting data drift. Refresh your knowledge of common NLP tasks: intent detection, named entity recognition, and summarisation.

Week 3: Mock interviews and behavioural prep.

Prepare STAR stories for at least five scenarios: a model you shipped, a failed experiment, a disagreement with a stakeholder, a time you worked under ambiguity, and a time you improved a process. Do at least two timed mock interviews with a peer. Record yourself if possible so you can hear how clearly you explain technical concepts.

The week before. Research the specific team you are interviewing with if possible. Review Glassdoor and LinkedIn posts from current Cresta data scientists for any publicly reported process details. Prepare three to four thoughtful questions to ask at the end of each round.

Skipping the business context. Candidates who answer 'how would you improve model accuracy' without first asking 'accuracy of what, for which user, and what is the cost of a wrong prediction' come across as purely academic. Always anchor technical answers to a concrete outcome.

Over-engineering the solution. Cresta moves fast and values pragmatism. Jumping straight to a complex ensemble or a large fine-tuned model before justifying the complexity can signal poor judgement. Start simple, then explain when and why you would increase complexity.

Ignoring deployment concerns. Discussing a model without mentioning how it would be monitored, retrained, or rolled back is a common gap. Even if you are not asked, briefly noting 'and in production I would watch for X' shows maturity.

Vague STAR answers. Saying 'we improved the model and results got better' is not a result. Candidates should quantify outcomes wherever possible, or explain why quantification was not possible in that context.

Not asking questions. Cresta interviewers typically leave time for your questions. Candidates who ask nothing signal low interest. Prepare two to three specific questions about the team's current technical challenges or how they measure success for the data science function.

On the application side, knok checks 150+ job sites nightly, applies to roles matching your resume, and messages HR on your behalf, so you can focus your energy on interview prep rather than job hunting.