The interview room your candidates use.

The real room, running live in your browser. Follow the steps of a sample Audition, or explore on your own. Code runs, the AI assistant answers and nothing you do is recorded.

Plan without AI. Read the brief on the left and write the plan on the right. The AI assistant is off in Plan, and the Plan locks when the candidate continues.

The interview room

Forward-Deployed AI Engineer · Sample interview

110:00

Phase 1 of 3: Plan (no AI)

Read the Business Problem first. It's the brief for everything you'll do. In Build you can work in Python, SQL, JavaScript, TypeScript, JSX or TSX.

This phase is your own thinking, so the AI assistant is off here. In the plan editor, write:

  • What the real question is, in your own words
  • Three or more assumptions your approach depends on
  • What you'd check first, in priority order
  • Two ranked questions you'd ask if you could
  • The risks you can already see

There is no minimum time or word count. Write as much or as little as you need, then continue whenever you're ready. A brief or template-only Plan is allowed and will simply remain part of your work evidence.

When you begin Build, your plan locks and you cannot come back to it.

Business Problem

Can the assistant change orders before they ship?

The situation

You are a forward deployed engineer at an AI company. Wrenbury, an online store for home goods and clothing in the US, has added your company's AI assistant to its help chat.

The most common request is changing an order before it ships: a new delivery address, canceling one item, or swapping a size. Today the support team does each one by hand. Ines Dalgaard, Head of Customer Support, wants the assistant to do them. She worries about packages going to the wrong place, changes made after packing starts, and customers being told things that are not true. Ines decides if the assistant goes live on its own. You recommend.

The model is a helper, not the judge. It sometimes misreads an order number, skips your tools, follows instructions hidden in a message, or promises something the store never offered. Your code decides what happens and what the customer sees.

What the business needs from you

Build two tools the model calls:

  • get_order(order_id, email) returns found and the order's items (sku, product, size). Never the address, the email or the total.
  • change_order(order_id, email, change, sku, new_value) applies the rules below and returns decision and refund. change is address, cancel_item or swap_item. For an address, new_value has street, city, zip and country. For a swap, it is the new sku.

The rules. The first one that applies decides:

  1. need_details: no order matches the number and email (ignore case), the sku is not in the order (for a cancel or a swap), or change is not one of the three.
  2. too_late: the chat is later than packing_starts_at minus your packing cutoff.
  3. handoff: the order total is over $500.00.
  4. not_allowed: a new address outside the US, or outside the order's city if your address scope is same_city.
  5. Swaps: handoff if your swaps setting is people. not_allowed if the new sku is out of stock or has a different price.
  6. changed. A canceled item refunds its price_paid. Every other refund is 0.

Your three settings. Pick one option for each. Any allowed choice can earn full marks.

SettingOptions
packing_cutoff_minutes30 or 60
address_scopesame_country or same_city
swapsassistant or people

Your code writes every customer message, from the decision, never the model's words. Confirm only what changed, give a refund with two decimals, and never promise a delivery date.

A chat is corrected when your code had to step in: the model never called change_order, its arguments matched no order or item, or its reply said something the result did not (a change that did not happen, a different refund or a delivery day).

The go-live call: alone if the corrected share is at most 0.20. Otherwise with_review: people approve each change.

Calling the model. Use from buildinterview import model, stand_in (examples in guide/). Try model on a few chats, then run every chat on the free stand_in. Send each chat as its own message and include its chat_id.

Partway through, a stakeholder will send one update. Decide what it changes.

The data

data/DATA_DICTIONARY.md explains every column. Times are store time. Money is in US dollars.

  • chats: one row per chat on October 12, 2026. This demo has 12; an Audition has about 150.
  • orders: one row per order, with packing_starts_at.
  • order_items: one row per item, with the price paid.
  • stock: the sizes a customer can swap to, with their price and in_stock.

What to hand in

Your main file writes:

  1. outputs/settings.csv (field,value): your three settings.
  2. outputs/change_results.csv, one row per chat: chat_id, order_id (empty if none matched), decision, refund, corrected (true or false) and message.
  3. outputs/summary.csv (field,value): chats, changed, too_late, not_allowed, handoff, need_details, refunds_total, corrected, corrected_share (a fraction like 0.25) and go_live.
  4. outputs/review.html for Ines's team, with the data inside: every chat and its decision, the chats that need a person, and each corrected chat with what the model tried. Keep it simple; it is not judged on design.

Write 0, not an empty cell. No commas or double quotes in a message.

Also write docs/stakeholder_writeup.md for Ines (up to 300 words: your call, the model's mistakes, your settings, what to watch) and docs/technical_writeup.md for the next engineer (up to 400 words: how it works and how you tested it).

How it is checked

  • Decisions, counts and corrected must match exactly. Money within 0.01, shares within 0.00001.
  • After you submit, we run your main file again on fresh data with different values and check it the same way. So compute everything from the files at run time.
  • Reviewers read your Plan, code, review page and notes.

Time and AI

Plan 15 minutes (AI off), Build 75 minutes, Writeup 20 minutes: about 110 minutes in total. AI Assist is on in Build and Writeup.

Your plan
Write as much or as little as you need
Plan is AI-off. There is no minimum time or word count. Continue whenever you are ready.

Dataset schema reference

Data dictionary

chats

ColumnMeaning
chat_idThe chat, for example C-101
chat_atWhen the customer sent the message (store time)
customer_messageWhat the customer wrote, exactly as sent

orders

ColumnMeaning
order_idThe order number customers see, for example WB-4102
customer_emailThe email used for the order
placed_atWhen the order was placed
delivery_city, delivery_zipWhere the order is going now
order_totalWhat the customer paid for the whole order, in US dollars
packing_starts_atWhen the warehouse starts packing the order. Changes close before this.

order_items

ColumnMeaning
item_idOne item in one order
order_idThe order it belongs to
skuThe product code, including its size where it has one
product, size, colorWhat it is
price_paidWhat the customer paid for this item, in US dollars

stock

ColumnMeaning
skuA size a customer can swap to
product, size, colorWhat it is
priceToday's price, in US dollars
in_stocktrue or false

The interview room is built for a desktop browser, as candidates use it. Scroll sideways to explore, or open this page on a computer.

Fictional sample interview. The code runs in your browser and the AI replies are prewritten. In an interview, code runs in isolated sandboxes and the session is recorded.See the report your team gets