Refinery89 Data Scientist Interview: Questions, Experience & Prep (2026)

Refinery89 is an ad-tech company that helps digital publishers grow revenue through smarter audience data, contextual targeting, and programmatic monetisation. Their Data Scientist team works at the intersection of publisher analytics, bid optimisation, and ML-driven audience segmentation. As of July 2026, Refinery89 has 10 open roles on knok's radar, pointing to active team growth.

The interview process typically spans several rounds. Candidates report a screening call (often with HR or a hiring manager), followed by a technical assessment covering SQL, Python, and machine learning concepts. Later rounds typically include a take-home project or live case study focused on ad-revenue or audience modelling, with a closing conversation with senior leadership. Round structure and order can vary, so treat this as a general guide based on what candidates have shared publicly.

The role sits firmly in ad-tech, so familiarity with programmatic advertising concepts such as real-time bidding (RTB), CPM, fill rate, and publisher yield management will set you apart from candidates with only generic data science experience.

The questions below are compiled from publicly reported interview experiences and job description patterns for Refinery89 Data Scientist roles.

1. How would you build a model to predict a publisher's expected CPM given historical ad performance data?
2. Walk us through how you would design an A/B test to compare two ad placement strategies. How do you determine sample size and significance?
3. Describe a time you worked with large-scale event-level data (clicks, impressions, page views). What pipeline did you build and what did you find?
4. How do you handle class imbalance in a click-through rate model where clicks are a very small fraction of total impressions?
5. What is the difference between contextual targeting and behavioural targeting? How would data science improve each?
6. Describe a situation where a model performed well in testing but poorly in production. What went wrong and how did you recover?
7. How would you segment a publisher's audience so an advertiser reaches the most relevant users? What features would you use?
8. Write a SQL query to find the top 5 publishers by total ad revenue over the past month, broken down by ad format (display, video, native).
9. How do you approach feature engineering for time-series ad data, for example predicting next-day fill rates?
10. Refinery89 works with real-time bidding data. How would you design a system to detect anomalies in bid prices or impression volumes?
11. How would you explain a model's output to a non-technical publisher partner who just wants to know: 'Will my revenue go up?'
12. What metrics would you use to evaluate an audience segmentation model, and how would you present the results to a product manager?

Q: How do you handle class imbalance in a click-through rate prediction model?

*Situation:* At my previous role, I built a click-through rate model on display ad impression data. Click rates for display ads are very low, a fact publicly reported across industry surveys, so the training set was heavily imbalanced.
*Task:* The model needed to rank ad creatives by likelihood of click to help the team prioritise spend, but early runs simply predicted 'no click' for everything.
*Action:* I compared two approaches: SMOTE (Synthetic Minority Oversampling) on the training set, and adjusting class weights directly in an XGBoost model. I also switched the evaluation metric from accuracy to AUC-PR, which is more informative for imbalanced targets, and used stratified cross-validation folds to prevent data leakage.
*Result:* The class-weight approach outperformed SMOTE on held-out data and was simpler to maintain in production. Ranking quality improved and the product team reported more relevant creative suggestions in the following sprint.

---

Q: Describe a time a model failed in production after passing in testing.

*Situation:* I deployed a publisher revenue forecast model that achieved strong results on historical validation data. Within two weeks of going live, predictions began drifting significantly.
*Task:* I needed to diagnose the root cause quickly because publisher partners were using these forecasts to plan ad inventory.
*Action:* I added feature distribution monitoring and discovered that an upstream data pipeline had quietly changed its timestamp format, causing a key time-based feature to default to zero for all new records. I fixed the pipeline, added automated schema validation checks, and set up alerts for feature drift using a lightweight monitoring script.
*Result:* The model recovered to its previous accuracy within a few days. The monitoring setup caught two more upstream changes in the following quarter before they could affect outputs.

---

Q: How would you segment a publisher's audience to help an advertiser target the right users?

*Situation:* A publisher partner wanted to help a retail advertiser reach users most likely to be interested in home appliances, but had no existing audience taxonomy.
*Task:* I needed to build audience segments from raw page-view and session data with no labelled categories to start from.
*Action:* I extracted features from content metadata (article topics, keywords), user behaviour (time on page, scroll depth, visit frequency), and device signals. I applied k-means clustering and validated clusters using silhouette scores and a qualitative review with the editorial team. I then mapped clusters to IAB audience categories so they were recognisable to the advertiser's buying team.
*Result:* The campaign targeting improved measurably, and the publisher reused the same taxonomy for several subsequent campaigns, reducing setup time for new advertiser briefs.

For technical modelling questions: Start with problem framing (what is the target variable, what data is available, what does 'good' look like). Then walk through data exploration, feature engineering, model choice, evaluation metric selection, and how you would deploy and monitor. Interviewers at ad-tech companies care as much about your thinking process as the final model choice.

For SQL questions: Restate the question in plain English before writing code. Name your CTEs clearly, comment on the logic, and mention edge cases such as nulls, duplicate rows, or timezone issues in time-based filters. Ad-tech SQL often involves large aggregations, so mention partitioning or indexing if relevant.

For business or product questions: Use a simple structure: define the metric you are optimising (revenue, engagement, fill rate), list the levers you would pull, and explain how you would measure success. Anchor your answer in Refinery89's context (publishers, advertisers, programmatic pipelines) rather than giving a generic response.

For behavioural questions: Use the STAR structure (Situation, Task, Action, Result). Keep the Situation brief (one to two sentences), spend most of the answer on Action (what you personally did), and close with a concrete Result. If you do not have a direct example, say so and describe what you would do instead.

For anomaly detection or monitoring questions: Describe your method choice (statistical process control, isolation forest, threshold-based rules), how you would set thresholds to avoid too many false positives, and how alerts would feed into a remediation process.

Domain fluency in ad tech. Interviewers typically want to see that you understand how programmatic advertising works: the relationship between publishers, SSPs, DSPs, and advertisers; how CPM, fill rate, and yield connect; and why these metrics matter to a business. You do not need to be an ad-tech veteran, but vague answers that ignore the domain signal that you have not prepared.

End-to-end ownership. Candidates who can speak to the full lifecycle, from data ingestion and feature engineering through model deployment and production monitoring, stand out. Refinery89 currently has 10 open roles (as of July 2026), suggesting a focused team where members typically wear multiple hats.

Clear communication. Publisher partners are not data scientists. Interviewers look for candidates who can translate a model output into a business recommendation that a non-technical stakeholder can act on.

Rigour with pragmatism. They want to see that you know the correct approach (proper cross-validation, unbiased evaluation, careful feature selection) but also that you can ship solutions in a real-world environment where data is messy and deadlines are real.

Curiosity about the data. Candidates report that interviewers respond well to answers where you proactively mention things you would investigate further, edge cases you would check, or follow-up experiments you would run.

Step 1: Build your ad-tech vocabulary. Spend time reading about programmatic advertising basics: RTB, CPM, CPC, fill rate, viewability, and the difference between SSPs and DSPs. You do not need deep engineering knowledge, but you should be able to use these terms correctly in context.

Step 2: Refresh core data science skills. Revisit SQL window functions and aggregations, Python libraries such as pandas and scikit-learn, model evaluation metrics (AUC-ROC, AUC-PR, RMSE), and A/B testing fundamentals including power calculations and common pitfalls like early peeking.

Step 3: Prepare your STAR stories. Pick three to four strong projects from your experience. For each one, write out the Situation, Task, Action, and Result in advance. Make sure at least one story covers a model that failed or required iteration, as this theme comes up frequently in Refinery89 interviews.

Step 4: Practise case-style questions out loud. Take one of the questions from the list above (such as building a CPM prediction model) and time yourself walking through it. Ad-tech case questions reward structured thinking more than memorised answers.

Step 5: Prepare thoughtful questions for your interviewer. Ask about the data infrastructure, how the data science team interacts with publisher partners, what a successful first few months looks like, and what the biggest unsolved problems are. These signal genuine interest and help you assess fit.

While you prepare, knok checks 150+ job sites nightly, applies to jobs matching your resume, and messages HR for you, so you can focus on interview prep rather than chasing applications.

Ignoring the ad-tech context. Answering as if Refinery89 were a generic analytics company misses the point. Generic answers about predicting customer churn, when the question is about publisher revenue, signal a lack of preparation.

Skipping evaluation metrics. Many candidates describe model building but forget to say which metric they would optimise and why. In ad-tech, choosing accuracy over AUC-PR for an imbalanced click dataset is a red flag.

Overcomplicating the solution. Jumping straight to deep learning or complex ensembles without first justifying why simpler models would not work suggests poor engineering judgement. Start simple and escalate with a clear reason.

Weak production thinking. Saying 'I would train the model and deploy it' without mentioning monitoring, retraining triggers, or data drift handling is a common gap, especially for senior candidates.

Vague STAR answers. Saying 'we improved model performance' without any indication of what improved or how it was measured is unconvincing. Even if you cannot share exact numbers, describe the direction and mechanism of improvement.

Not asking questions. Candidates who ask nothing at the end of a round tend to read as less engaged. Prepare at least two genuine questions per round.