Google

What the interviewer is probing

This question tests whether the candidate immediately recognizes the network interference problem — the defining reason standard A/B testing fails here — and whether they can reason about the trade-offs of the available alternatives (geographic clustering, switchback/crossover, synthetic control). It also probes metric judgment: the distinction between local segment effects and network-wide effects is not cosmetic, and a strong candidate will articulate why only one of them represents the policy-relevant estimand. Finally, the follow-ups reveal causal discipline — can they enumerate the confounders a switchback design is exposed to and propose concrete countermeasures?

Strong answer outline

• Why standard A/B fails: routing changes are not independent across users — diverting user A changes congestion for user B, so individual-level randomization produces SUTVA violations everywhere. Treatment and control units interfere, making estimated effects meaningless.
• Randomization unit options and their costs:
• Geographic clustering (randomize by city or district): eliminates within-city interference but requires many cities for power; spillover at cluster boundaries remains.
• Switchback / crossover design (alternate treatment and control over time across the whole city): removes geographic spillover by giving the entire network the same treatment state at any moment. Cost: assumes the city reaches a new equilibrium quickly, and that consecutive-day effects don't bleed over (carryover bias).
• Synthetic control (match treated cities to untreated ones): useful for city-level rollouts but requires rich pre-treatment data and assumes no network ties between cities.
• Segment selection is a design choice, not a tuning knob: selecting ~100 historically congested segments creates a well-defined estimand (effect on those bottlenecks) but limits external validity to similar bottleneck types. Pre-registration matters here to prevent post-hoc fishing.
• Carryover bias in switchback: traffic patterns have memory (rush hours, weekly seasonality). Mitigation: enforce washout periods between arms, balance arm assignment by day-of-week, and model residual autocorrelation explicitly in the analysis.
• Analysis with hierarchical Bayesian model: pools information across cities and hours while allowing city-level heterogeneity; appropriate when per-city power is limited. Key output is a posterior distribution over the effect, not just a point estimate — which forces honest uncertainty communication.
• Interpreting the two effect sizes: the ~2% targeted-segment effect measures local decongestion; the ~0.35% network-wide effect is the policy-relevant number because it captures both the gains on cleared segments and the costs imposed on absorbing segments. Reporting only the larger local number would overstate societal benefit.
• Common wrong turns: claiming individual-level causal effects from a city-level design; ignoring that non-app users benefit (which is actually an argument for the intervention, not against measurement); conflating statistical significance with practical significance at 0.35%.

The underlying concept

Standard A/B testing rests on the Stable Unit Treatment Value Assumption (SUTVA): one unit's treatment does not affect another's outcome. Traffic networks violate this structurally — a rerouted vehicle changes travel times for every other vehicle on its new and old path, a phenomenon called interference or spillover. When interference is global (the whole network is one connected system), the only valid randomization units are ones that contain the interference: entire cities, or time periods in a switchback design. Switchback designs trade spatial isolation for temporal isolation, measuring the city's response to switching the entire system between states, but they introduce carryover risk whenever the system's equilibrium lag exceeds the switching interval. The causal estimand must be defined at the level of the randomization unit — network-wide average speed, not individual trip time — or the estimate is not causally identified.

Source

Derived from The power of collaboration: How we can reduce traffic congestion

What the interviewer is probing

This question probes whether the candidate can translate an economic constraint (pay-per-byte-hour) into an algorithmic design, rather than treating resource allocation as a static engineering parameter. It tests architecture judgment — separating the TTL assignment problem from the capacity management problem — and forces the candidate to reason about the feedback loops between a prediction model and the system it controls. The validation portion surfaces metric judgment: distinguishing between cost savings and latency degradation, and knowing which cache miss increases are acceptable.

Strong answer outline

• Frame the cost model first. Total cost = (memory cost × bytes cached × time held) + (miss penalty × miss rate). The insight is that holding a page whose next access is far away costs more in memory rental than the miss penalty you'd pay to re-fetch it. This reframes eviction as: "when should I stop renting this page's slot?"
• The ski-rental analogy. Each cached page faces the same rent-vs-buy decision under uncertainty about future access. The break-even rule: keep the page as long as its accrued memory cost is below the miss penalty; evict when it crosses. A randomized variant reduces expected cost versus the adversarial worst case.
• TTL assignment via a lightweight model. Features: data size (miss penalty scales with it), operation type (read vs. scan), historical inter-arrival time for this page, relative miss cost vs. memory cost ratio. Candidate should reach for a shallow decision tree or small lookup table — not a neural net — because inference runs on every cache hit in a hot path. Wrong turn: proposing a complex model without noting the latency budget.
• Capacity eviction as a backstop. When the cache fills, fall back to a generalized LRU (or GDSF to handle variable-size pages). TTL-based eviction shrinks the working set first; LRU handles physical overflow. These are complementary, not competing.
• Validation strategy. Offline: replay cache traces with the new policy vs. fixed-size baseline; measure integrated memory-time cost and miss rate. Distinguish miss rate by miss cost — a high miss rate on cheap-to-fetch rows is acceptable; misses on expensive cross-shard reads are not. Online: shadow the new policy against production traffic before cutover; track p99 latency, actual I/O cost per request, and memory footprint over time. Alert on tail-latency degradation, not just aggregate miss rate. The critical wrong turn: using miss rate alone as the health signal — a cost-aware system deliberately increases misses on cheap data.
• Adaptation. TTL model should be retrained on rolling windows of production access logs. Feature drift (e.g., a new query pattern) shows up as systematic TTL misprediction; detect this by monitoring model-predicted TTL vs. actual next-access gap for sampled pages.

The underlying concept

Static cache sizing is a peak-provisioning problem: you allocate for the worst hour and pay for all the idle hours in between. When memory is metered, this becomes a literal dollar cost rather than an amortized hardware cost, which makes the economics of dynamic sizing much more attractive. The ski rental problem is the canonical model for this class of decision: you must commit to a recurring cost (renting memory) or a one-time cost (paying the miss penalty) without knowing how long you'll need the resource. The break-even algorithm — rent until your total rental equals the buy price, then buy — is worst-case optimal within a factor of 2. In practice, access patterns are predictable enough that a learned model can beat the adversarial bound significantly. The deeper structural insight is that TTL-based eviction and capacity-based eviction solve different subproblems — TTL controls the holding duration under normal load, and LRU/GDSF controls behavior when the cache fills — and keeping them cleanly separated makes both easier to reason about and tune.

Source

Derived from Optimizing cloud economics with linear elastic caching

What the interviewer is probing

The question tests whether the candidate can reason about transfer learning not as a recipe but as a set of assumptions — about distribution shift, label density, and feature alignment — that must each be checked empirically. Metric judgment is probed through the carbon-accounting use case, which makes false-negative costs concrete. Causal reasoning about geographic clustering in the label set is the hardest dimension; strong candidates will recognize that spatially autocorrelated data requires spatial cross-validation, not random splits.

Strong answer outline

• Why a pre-trained backbone helps: satellite imagery shares low-level texture features (vegetation spectral signatures, edge patterns, shadow geometry) across geographies; a backbone trained on hundreds of millions of global images will have learned these representations far better than a model trained from scratch on a few hundred km² of annotations. The fine-tuning hypothesis: the head and upper layers need to specialize to local landscape character; the lower layers may transfer almost directly.
• Validation set design — the critical issue: spatial autocorrelation means that if you randomly split pixels or even patches, train and validation sets will be geographically adjacent and share context. Use spatial cross-validation: hold out geographically contiguous blocks (e.g., entire counties), not random patches. This gives honest generalization signal.
• Checking that pre-training actually helps: ablation — train the same head from random init vs. from the pre-trained backbone, on the same labeled data, evaluated on the spatial hold-out. If the backbone doesn't improve sample efficiency (fewer labeled km² needed for the same performance), the distribution shift is too large and the representations don't transfer.
• How much to freeze: start with the backbone frozen, establish a baseline, then progressively unfreeze upper layers and measure validation performance. The inflection point where unfreezing hurts (overfitting to the small labeled set) tells you where to stop. Monitor training vs. validation loss gap carefully.
• Distribution shift failure modes in deployment:
• Geographic bias in labels: if annotators covered accessible or visually distinct areas, the model learns those contexts. Field-interior hedgerows, less-managed boundaries, or features in remote uplands may be systematically missed.
• Seasonal/illumination shift: fine-tuning imagery may be from one season; deployment imagery from another — vegetation appearance changes significantly.
• Resolution or sensor shift: if inference imagery comes from a different satellite or processing pipeline, spectral statistics shift.
• Rare but high-value features: stone walls may be underrepresented in training; the model's recall on them may be near zero without explicit oversampling or a weighted loss.
• Asymmetric cost — false negatives: use a recall-weighted loss (e.g., focal loss or class-weighted cross-entropy biased toward the positive class), tune the classification threshold explicitly on the spatial validation set against a precision-recall curve, and report recall and false-negative rate per feature class as primary metrics, not aggregate pixel accuracy.
• Wrong turns: using random pixel splits for validation (falsely optimistic generalization); treating aggregate IoU as the primary metric when the use case is carbon accounting (hides per-class recall differences); freezing the entire backbone without testing whether fine-tuning upper layers helps on this specific landscape character.

The underlying concept

Transfer learning from large foundation models works when two conditions hold: the source pre-training distribution shares low-level structure with the target task (so early-layer features transfer), and the target dataset, while small, is representative enough of the deployment distribution that fine-tuning the upper layers doesn't overfit to annotation artifacts. Geographic data violates the standard i.i.d. sampling assumption — nearby locations are correlated — so a random train/val split produces optimistically biased validation metrics. Spatial cross-validation, which holds out contiguous geographic blocks, is the correct analog to time-series cross-validation in temporal data: it simulates the actual generalization challenge, predicting on regions the model has never seen. Asymmetric misclassification costs should be encoded in both the loss function and the evaluation protocol from the start, not corrected post-hoc.

Source

Derived from From pixels to planning: Earth AI for nature restoration

What the interviewer is probing

Whether the candidate understands that at extreme power the binary significance question dissolves and the real questions become effect size, practical value, and portfolio-level error control. Also probing knowledge of overlapping-experiment infrastructure — layered traffic allocation — which Google pioneered and which any candidate claiming experimentation depth should be able to reconstruct from first principles.

Strong answer outline

• Replace "significant?" with "big enough to matter?": pre-registered practical-significance thresholds per metric (minimum launch-worthy effect), confidence intervals reported against those thresholds rather than zero.
• Value framework: convert core metrics to a common currency (long-term value per user) with explicit exchange rates set by leadership — e.g., how much revenue a millisecond of latency is worth — so mixed-sign results are decidable rather than debated ad hoc.
• Long-term validity: tiny short-term gains can be novelty or cannibalization; use long-running holdbacks to measure cumulative and lagged effects, and validate proxy metrics against them.
• Concurrency: layered/orthogonal experiment infrastructure — divide the system into layers (e.g., UI, ranking, ads) where experiments within a layer split traffic exclusively but layers overlap freely via independent hashing; run explicit interaction tests only for suspected-conflicting pairs.
• Portfolio error control: with thousands of tests, control the expected number of false launches (higher evidence bars for launch-gating metrics, replication runs for surprising wins, holdback verification post-launch).
• Segmentation discipline: pre-registered segment analyses or hierarchical shrinkage estimates; exploratory segment findings get confirmed in a fresh experiment, never launched from the discovery data.

The underlying concept

Statistical significance answers "is the effect exactly zero?" — a question nobody asked. With enormous samples, standard errors collapse and every real-but-trivial effect clears p-thresholds, so decision theory must take over: minimum effects of interest, explicit trade-off prices between metrics, and error control at the portfolio level (false launches, not false positives). Overlapping-layer designs solve the concurrency problem by making experiment assignments statistically independent across layers, spending scarce exclusive traffic only where interference is real.

Source

Distilled Prep canon — built on Google's published experimentation methodology (overlapping experiments, long-term holdbacks).

Source: Google engineering blog

What the interviewer is probing

Production-ML maturity: the understanding that most real-world model failures are data failures — schema drift, distribution shift, broken upstream joins, feature pipeline bugs — and that reliability comes from treating data like code: schemas, validation, versioning, and gated promotion. The TFX worldview, reconstructable from first principles.

Strong answer outline

• Failure taxonomy: schema changes (renamed/dropped fields, type changes), distribution shift (gradual drift vs. sudden breakage), missing partitions, upstream logic changes silently redefining a feature, and label problems (delay, leakage).
• Data validation stage: maintain a versioned schema + expected statistics per feature (ranges, missingness, distributions); validate each training batch against it — hard-fail on schema violations, alert on statistical anomalies (drift scores vs. a reference window); quarantine bad batches rather than train on them.
• Skew prevention structurally: single feature-definition source compiled to both training and serving (shared transforms / feature store); continuously compare serving-time feature distributions against training distributions — skew alarms are among the highest value monitors.
• Training pipeline: versioned data snapshots + versioned code + config = reproducible artifacts; evaluate on holdout and on sliced metrics (segments where regressions hide) against the current production model as baseline.
• Promotion gates: offline eval thresholds → canary serving on small traffic with online guardrails → gradual ramp with automatic rollback; the previous model stays warm for instant reversion.
• Diagnosis loop: when production quality dips, the versioned lineage (which data, which code, which model) plus feature-distribution monitors localize whether it's data, model, or world; without lineage, every incident is archaeology.

The underlying concept

Continuous training turns ML into a data-processing system whose correctness depends on inputs no one reviews — so the engineering answer is to give data the same discipline code has: declared schemas, automated validation, version control, and staged rollout. The two structural insights: skew is prevented by construction (one definition, two consumers), not by vigilance; and models should be promoted like binaries — evaluated against the incumbent, canaried, and reversible — because "newer" is not "better" when the data decides.

Source

Distilled Prep canon — built on the themes of Google's TFX and ML production-systems publications.

Source: Google engineering blog

What the interviewer is probing

Whether the candidate can localize the strong-consistency requirement (budget commitment) and engineer around paying cross-continent latency on the hot path — the budget-reservation/quota-lease pattern. Also probing real distributed-systems literacy: consensus per shard, read-path options (leases, bounded staleness), and why globally ordered transactions need bounded clock uncertainty.

Strong answer outline

• Split the problem: the invariant (total spend ≤ budget) needs serialized commitment; the decisions (serve this ad?) need microsecond reads. Never put the second on the first's path.
• Budget ledger: shard by account/campaign; each shard is a consensus-replicated state machine (Paxos/Raft) across regions — writes commit with cross-region quorum (pay the ~100 ms once per budget mutation, which is rare traffic).
• Local spending at global safety: regions lease budget slices from the ledger (e.g., region takes a 5% tranche); ad servers decrement local tranches at memory speed and renew leases asynchronously. Overspend is bounded by outstanding tranche size — pick tranche sizes to trade slight budget under-utilization against overspend risk.
• Reads: serving reads hit local replicas/leases (bounded staleness by construction); reporting reads can request linearizable reads through the leader when exactness matters.
• Time: to give external consistency (transactions ordered as observed globally) you need bounded clock uncertainty — the TrueTime idea: expose uncertainty intervals and wait them out on commit; with plain NTP you either wait too long or violate ordering under skew.
• Partition behavior: a cut-off region can spend only its already-leased tranches, then fails closed (stops spending) — bounded damage, no global invariant violation; leases expire and return to the ledger.
• State the CAP position explicitly: choose consistency for the ledger; engineered availability comes from leases, not from weakening the invariant.

The underlying concept

Global strong consistency is affordable when you stop buying it wholesale: identify the single invariant that needs serialization, run consensus for exactly that, and convert the hot path into locally served, pre-authorized capacity — the quota-lease/escrow pattern that turns a global constraint into bounded local ones. The Spanner lesson completes it: global transaction ordering is fundamentally a clock problem, solved not by perfect clocks but by known-bounded uncertainty plus commit waits.

Source

Distilled Prep canon — built on the themes of Google's Spanner and planet-scale infrastructure work.

Source: Google engineering blog

What the interviewer is probing

Metric literacy in search: CTR measures the promise of a result (title/snippet attractiveness), not its delivery. Rising reformulations and pogo-sticking are classic signals that clicks are failing to satisfy. Strong candidates assemble a success-oriented metric story (abandonment done right, long clicks, session-level success) and slice by query intent before deciding.

Strong answer outline

• The reconciling story: the change likely surfaces clickable results — compelling titles/snippets — that under-deliver; users click, bounce back (pogo-stick), and re-query. CTR up + reformulation up = attractiveness up, relevance flat or down.
• Better metric set: long clicks (dwell above threshold) vs. short clicks; pogo-stick rate; session-level success (did the session end in a satisfied state?); time-to-result; and good abandonment (query answered on the results page itself) separated from bad abandonment.
• Diagnose by intent: navigational queries (one right answer), informational, transactional — a change can help one class and harm another; the aggregate hides it. Also slice by position: is added CTR coming from top positions or deep exploration?
• Careful with dwell: long dwell can mean satisfaction or struggle; calibrate time signals per query class against explicit-feedback panels or human relevance ratings.
• Recommendation shape: as described, the evidence pattern points against launch despite CTR; either iterate (snippet honesty, relevance) or launch only for the query classes where session success genuinely improved.
• Prevention: make session-success and pogo-stick guardrail metrics so attractiveness-only wins can't clear the launch bar again.

The underlying concept

Every proxy metric encodes an assumption about what it proxies for; CTR assumes clicking predicts satisfaction, and ranking changes can break exactly that link by optimizing the promise rather than the payoff. Search measurement matured by moving from action counts to session-level success models — the unit of value is the user's task, not the click. The generalizable habit: when two proxies disagree (clicks up, re-queries up), the disagreement itself is the finding, and the tiebreaker is the metric closest to the user's true goal.

Source

Distilled Prep canon — built on Google's published work on search quality measurement.

Source: Google engineering blog

What the interviewer is probing

Understanding of the core datacenter economics — services reserve for peak and idle most capacity, so reclaiming that idle capacity for preemptible batch work is worth billions — plus the mechanisms that make sharing safe: priorities/tiers, preemption, resource isolation, and overcommit informed by actual usage vs. reservations. This is Borg territory; Kubernetes-only intuition covers maybe half of it.

Strong answer outline

• Why mix: dedicated pools idle at 20-40% utilization because services size for peak; co-locating preemptible batch in the trough reclaims it. The danger is interference — batch stealing CPU cache, memory bandwidth, or IO from latency-critical work.
• Job spec: resource requests (CPU, RAM, disk, accelerators), priority tier (production vs. best-effort), constraints (region, hardware, anti-affinity), and disruption tolerance. Quota is enforced at admission, priority at runtime.
• Scheduling loop: feasibility filtering (which machines can host it) then scoring (bin-packing for utilization, spreading for failure domains, data locality); at this scale use equivalence classes / cached scores and randomized sampling of candidate machines rather than scoring the whole fleet; shard or replicate schedulers with optimistic concurrency on placement commits.
• Priority mechanics: the top-tier service preempts best-effort tasks — scheduler evicts enough batch (respecting disruption budgets), reclaims resources, places the service; batch tasks are checkpointed or simply rescheduled — their contract is "will finish eventually."
• Overcommit safely: schedule against predicted actual usage rather than reservations for lower tiers, with node-level isolation (cgroups-style) and a killer that reclaims from best-effort first when a node runs hot.
• Reliability: scheduler state rebuilt from node agents (nodes are the truth); running workloads unaffected by scheduler outages; masters consensus-replicated per cell; failure domains and maintenance handled via eviction with notice.

The underlying concept

Cluster scheduling is economically a statistical multiplexing problem: peak-reserved capacity is mostly air, and tiered priorities plus preemption convert that air into throughput while preserving the latency tier's guarantees. Technically it's constrained optimization under a latency budget — solved not by optimality but by good-enough placement found fast (sampling, caching, sharding). The deepest design rule: keep the data plane independent of the control plane, so the scheduler's own failures never take down what it scheduled.

Source

Distilled Prep canon — built on the themes of Google's Borg and cluster-management publications.

Source: Google engineering blog