Skip to content
Tech Interview Prep

Real Interview Questions for the Embedded Systems Engineer at SpaceX

New

Interview Q&A

25 questions · Flow follows Relevance until this set has a learning order.

Q&A-1Why do you want to work at SpaceX?

Reported in jobrise.io › en › blogSpaceX Software Engineer Interview Guide 2026 | JobRise

The 90-second answer

Give three things in order: a specific engineering reason tied to how SpaceX builds, one piece of evidence from your own work with a number in it, and a question back. Mission enthusiasm alone doesn't separate you: everyone in the lobby has it.

"Two reasons, and neither is the mission poster. First, the iteration rate. SpaceX flew 96 Falcon launches in 2023 against 26 in 2020: 91 Falcon 9 plus 5 Falcon Heavy in 2023, so roughly one every 3.8 days averaged across the year (SpaceX published launch counts; the 3.8 is my arithmetic, 365/96). Each of those is a full-stack exercise of the flight software, so a latent race or a badly chosen watchdog timeout surfaces in weeks rather than at the next review cycle. I want my code judged at that rate.

Second, the constraint set. Mass and power budgets on a vehicle are tight enough that redundancy has to be justified rather than assumed, and the decisions I find interesting sit on the boundary between the flight computer, the controller next to the actuator, and the fault logic that has to tell a dead sensor from a dead engine. That's the work I've been doing: I cut worst-case CAN receive jitter by moving timestamping into the ISR and replacing a heap-allocated queue with a fixed-size ring, and I can walk you through the measurement."

Substitute your own measured numbers there. If you don't have them, describe the mechanism and say what you measured and on what hardware: a made-up figure is worse than no figure, because the follow-up will find it.

Close with a real question: "Which team would I be joining, and does a new engineer start on the test bench before touching flight code?"

Calibrate to who is asking

InterviewerWhat they wantWhat to cut
Recruiter screenCadence, ownership, willingness to relocateJitter numbers, register-level detail
Embedded hiring managerOne subsystem you can name, one measurement you can defendMission narrative
Cross-team engineerEvidence you'll debug someone else's board at 2 a.m.Anything about your career plan

Follow-ups to expect

"Why not NASA or JPL?" Don't disparage them. The difference is flight rate and ownership: at a lower rate a firmware change waits a long time for its next integration opportunity, and review overhead scales with that. Say what you'd give up by going the other way: depth on one vehicle, far fewer chances to be wrong and find out.

"What would you want to work on?" Commit to one: engine control, GNC flight software, avionics bring-up, or test infrastructure. Test infrastructure is a legitimate answer and often the fastest way in: the hardware-in-the-loop benches are what make the cadence survivable.

"What would make you leave?" Answer straight. If it's "nothing for years," say what you'd need instead: ownership of a subsystem end to end, and a feedback loop that stays short.

Failure modes, and how the interviewer detects them

Mission-only answer. Detection: they ask "what would you work on?" and you stay at mission level. Cost: filed as a fan, and the loop ends at this screen. This question is often the first filter, so there is no later chance to recover.

Claiming a domain you can't describe. Detection: "walk me through what a flight software integration test looks like." If you can't name the bench, the harness, and the simulated sensors, the claim collapses and every earlier statement gets discounted.

Numbers you can't defend. Detection: "what did you measure, on what hardware, under what load?" A 10x improvement with no measurement behind it costs you credibility on the technical answers that follow, which is a much bigger loss than the number was a gain.

No questions back. Detection: silence when they ask "anything for me?" Cost: you leave without knowing whether the team is engine control or GNC, which is exactly the information that should shape your next answer.

Curated: · Written: · Reviewed:

Q&A-2Tell me about the hardest bug you personally tracked down.

Reported in Why the SpaceX interview feels nothing like a FAANG loop

The 90-second answer

A CAN node on an engine controller went bus-off on roughly 1 boot in 18 at −40 °C and never at ambient. The cause sat three layers below the symptom: the PLL lock-wait loop in clock_init polled a status register through a non-volatile pointer, so the compiler was entitled to assume the loop's exit condition never changes and drop it. The code then sized the CAN prescaler from a core clock sampled before the PLL locked: sized for the 16 MHz oscillator, running a 500 kbit/s bus at 2.5 Mbit/s. The fix was volatile on the register, a bounded timeout that faults the boot instead of continuing, and a compile-time clock constant in place of a runtime measurement.

The part worth probing is why it took a week: the symptom pointed at the transceiver, and the boot log already held the answer.

The failure chain

LayerObservedActually
Symptombus-off ~320 ms after power-onTEC hit 256
Cause 1every frame errored2.5 Mbit/s on a 500 kbit/s bus
Cause 2prescaler sized for 16 MHzclock read before PLL lock
Cause 3no wait ever executedpoll loop removed at -O2

Why the loop vanished

c static uint32_t *pll_status = (uint32_t *)0xFFF70000u; // not volatile

void clock_init(void) { *PLL_CTL = PLL_ENABLE; while ((*pll_status & PLL_LOCK) == 0) { } // loop-invariant, no side effects ram_bist(); // ~1.9 ms can_init(core_clock_hz()); // reads the mux, not the PLL }

Nothing in that loop writes through pll_status, so the condition is loop-invariant and the body has no side effects. C11 §6.8.5p6 lets an implementation assume a loop whose controlling expression isn't constant terminates, which licenses removing it and the code after it. I can't tell you which pass did it; I can tell you the shipped image had no branch between the PLL enable and the RAM BIST, which is what mattered. The bug was in the code, not the compiler.

Why only at −40 °C

Figures below are from that board's PLL datasheet and our chamber logs: the shape of the problem, not universal constants. Lock time is 45 µs at +25 °C; at −40 °C it is 1.2 ms median, 2.1 ms at p99, 2.6 ms worst, crystal startup dominating. The clock read lands ~1.9 ms after enable, behind the BIST. At ambient the PLL is locked forty times over before the read. At cold the read sits in the lock-time tail: 11 bus-offs in 200 chamber boots, consistent with the fraction still unlocked at 1.9 ms.

The experiment that settled it

Swapping transceiver and harness on a failing unit moved nothing. A debug build with the BIST padded to 5 ms went 200/200 clean at −40 °C. That pinned it to a race between the clock read and PLL lock, and it cost one afternoon once I stopped trusting the symptom.

The fix

c static volatile uint32_t * const PLL_STATUS = (volatile uint32_t *)0xFFF70000u;

bool clock_init(void) { *PLL_CTL = PLL_ENABLE; for (uint32_t us = 0; us < 5000; us++) { if (*PLL_STATUS & PLL_LOCK) { __DMB(); // order the lock against the mux switch can_init(80u * 1000u * 1000u); // constant, not measured return true; } delay_us(1); } fault(FAULT_PLL_NO_LOCK); // fail the boot, don't guess return false; }

Two deliberate choices. The clock is a compile-time constant, because a measurement taken during bring-up is exactly the thing that can be wrong. And the timeout faults rather than falling through: a board that refuses to boot is diagnosable, a board that boots at 5× the wrong bus rate is not. volatile alone isn't sufficient in general: it stops the load being hoisted but doesn't order it against the following peripheral writes, hence the CMSIS __DMB().

What changed after

Every MMIO pointer now comes from a volatile-qualified header, and CI greps for casts to non-volatile pointer types. The boot self-test cross-checks the core clock against the independent 32.768 kHz RTC and logs the ratio, so a 5× error is one log line instead of a week of chamber time. Cold soak is a gate on every flight build: 200 boots at −40 °C, not a spot check.

Curated: · Written: · Reviewed:

Q&A-3Classify the bug in this C snippet.

Reported in SpaceX Interview Questions & Process - Software Engineer

The 90-second answer

Classification is two questions, not one: when does the bug bite, and which rule does it break. When gives you compile/link, wrong-at--O0, wrong-only-at--O2, or only-on-hardware. Which rule gives you undefined behavior, implementation-defined, or well-defined-but-wrong. The label is the intersection of the two.

For the snippet on the board: a poll loop on a memory-mapped status register reached through a plain uint32_t *:

c #define SPI_SR (*(uint32_t )0x40013008u) / bit 7 = BSY */

void spi_wait_idle(void) { while (SPI_SR & (1u << 7)) { } }

the class is well-defined program with wrong semantics: missing volatile on an MMIO object. It is not undefined behavior, and saying so is the point. The abstract machine has no reason to believe the object at that address ever changes, so the compiler may load it once and loop on the cached value. On arm-none-eabi-gcc 13.2 -O2 the load is hoisted above the loop and the body collapses to a branch to itself: the function never returns, and the watchdog bites instead.

The classification table I'd run through

SymptomClassWhy
Hangs at -O2, fine at -O0missing volatile (defined, wrong)load hoisted; abstract machine sees no writer
for (uint8_t i = 0; i < 256; i++) never endsinteger promotion (defined, wrong)i promotes to int, 255 < 256, then wraps to 0
int32_t t = a * b; wrong sign only when optimizedUB: signed overflowoptimizer assumes it cannot happen
Struct over a peripheral has wrong field offsetsimplementation-definedpadding and alignment are not fixed by C
Fine on the bench, corrupt in flightrace on an ISR-shared variableneeds volatile and atomicity or a critical section

The fix, and what it does not buy you

c #define SPI_SR (*(volatile uint32_t *)0x40013008u)

bool spi_wait_idle(uint32_t timeout_cycles) { while (SPI_SR & (1u << 7)) { if (--timeout_cycles == 0u) return false; /* never hang the vehicle */ } return true; }

volatile forces a real load per iteration. It does not give atomicity and it is not a memory barrier: a read-modify-write on a shared register still needs a critical section or a peripheral-level atomic, and ordering against a DMA descriptor write needs __DMB(). A poll loop with no timeout is its own defect: a stuck peripheral should return an error the caller can log and recover from, not spin until the watchdog resets the board.

How you catch it

A host unit test compiled at -O0 against a plain variable will pass and prove nothing. Build the test at the same -O2 and the same target triple as flight, and keep the register behind a header so the volatile qualifier is auditable in one place. Static analysis and MISRA C:2012 rules on volatile and on integer-to-pointer conversions flag the declaration; code review flags the missing timeout. If the interviewer pushes to "is it UB?", the answer is no: it is a correct program that computes the wrong thing, which is why the compiler is not at fault and -Werror will not save you.

Curated: · Written: · Reviewed:

Q&A-4Explain what makes this code thread-safe?

Reported in SpaceX Interview Questions & Process - Software Engineer

The 90-second answer

That question is only answerable against specific code, so I'd start by pinning down three things: what state is shared, which contexts touch it (ISR, task, DMA, second core), and what the memory model actually guarantees. Then the claim reduces to one of four properties, and I want to hear which one the author intended, because "it has a mutex" is not the same as "it is correct".

  1. Nothing is shared. All state is on the stack or in registers, no statics, no globals, no shared peripherals. Reentrant by construction. This is the cheapest and most common answer in flight code.
  2. Shared but immutable after init. A const table written once before the scheduler starts and only read afterwards is safe with no synchronisation at all.
  3. Shared and serialised. A mutex, or a critical section that masks the interrupt, with a documented lock order. Correct only if every accessor takes it: one unguarded read of a 64-bit value on a 32-bit core breaks it.
  4. Shared and lock-free. Single-writer/single-reader with atomics and explicit ordering. Fast enough for an ISR, but the ordering is the whole proof.

What that looks like in code

An ISR-to-task queue is the case I'd expect at SpaceX: you cannot take a mutex in an ISR, so the handoff has to be lock-free:

c // SPSC ring, ISR produces, task consumes. C11 atomics, gcc 13 / arm-none-eabi-gcc 13. typedef struct { uint8_t buf[256]; _Atomic uint32_t head; // written only by producer _Atomic uint32_t tail; // written only by consumer } spsc_t;

bool spsc_push(spsc_t *q, uint8_t v) { uint32_t h = atomic_load_explicit(&q->head, memory_order_relaxed); uint32_t t = atomic_load_explicit(&q->tail, memory_order_acquire); if (h - t == sizeof q->buf) return false; // full q->buf[h % sizeof q->buf] = v; atomic_store_explicit(&q->head, h + 1, memory_order_release); return true; }

bool spsc_pop(spsc_t *q, uint8_t *v) { uint32_t t = atomic_load_explicit(&q->tail, memory_order_relaxed); uint32_t h = atomic_load_explicit(&q->head, memory_order_acquire); if (h == t) return false; // empty *v = q->buf[t % sizeof q->buf]; atomic_store_explicit(&q->tail, t + 1, memory_order_release); return true; }

The release on head and the acquire on head are what make it safe: the byte store to buf[5] cannot be reordered after the head = 6 store, so a consumer that observes head == 6 is guaranteed to see the byte. Drop them to relaxed and it still passes on x86 and fails intermittently on Cortex-M7 or A53.

Failure modes I'd name

SymptomCauseHow you catch it
Counter drifts, ~6 lost increments/svolatile uint32_t n; n++ in both ISR and task: load-add-store is three instructions, ISR lands in the windowScope on the variable, or a host test with an injected preemption point
Rare stale reads after a writeMissing acquire/release, or volatile mistaken for a barrierThreadSanitizer on the host build; disassembly review
Task hangs holding a lockPriority inversion: low-priority task holds a mutex a high-priority task needs, medium-priority task preemptsPriority-inheritance mutex, or a watchdog that logs the lock owner
Data looks correct in RAM but DMA reads old valuesCache coherency on a core with D-cache; needs clean/invalidate around the bufferAssert on alignment and cache-line padding
Deadlock at bootTwo locks taken in opposite ordersStatic lock-order checker, or a single documented hierarchy

That lost-update figure is illustrative: a 10-cycle read-modify-write on a 168 MHz core is a 60 ns window, and a 10 kHz ISR lands inside it about 0.06% of the time. Six lost counts per second, which looks like sensor noise until you find it.

How I'd verify the claim

I'd want the shared-state list written down, ThreadSanitizer or a model checker run on the host build of the module, and a stress test that hammers the ISR at its worst-case rate while the consumer runs. If the answer to "what makes this thread-safe" is a mutex, the follow-up is "show me every accessor": that is where these bugs live.

Curated: · Written: · Reviewed:

Q&A-5How does memory get laid out here?

Reported in SpaceX Interview Questions & Process - Software Engineer

The 90-second answer

"Here" depends on the target, so I'd pin that down first: which board and core, and whether we mean the linker's view at build time or the MPU's view at run time. Assuming a bare-metal or RTOS flight computer with an MPU and no MMU, the layout is decided by the linker script and enforced at run time by MPU regions.

There are three kinds of address space and they never mix: flash (execute in place), SRAM (read/write), and peripheral registers (MMIO. volatile, never in a normal section). Startup code copies .data from its flash image into RAM, zeroes .bss, and deliberately leaves .noinit alone so a warm reset can read the boot reason and crash log.

SectionContentsLives inStartup action
.vectorsvector table at the boot addressflashKEEP, never GC'd
.text / .rodatacode, const tables, string literalsflashnone
.datainitialised non-zero globalsRAM, image in flashcopy _sidata→_sdata.._edata
.bsszero-init globals, COMMONRAMzero _sbss.._ebss
.noinitboot reason, crash logRAMuntouched
.stacksone static array per task + ISR stackRAM or TCMRTOS at task create
.dmadescriptors and buffersnon-cached SRAMdriver
MMIOperipheral registersfixed addresseshardware

ld MEMORY { FLASH (rx) : ORIGIN = 0x00000000, LENGTH = 2M SRAM (rwx) : ORIGIN = 0x20000000, LENGTH = 512K DTCM (rwx) : ORIGIN = 0x30000000, LENGTH = 64K } SECTIONS { .vectors : { KEEP(*(.vectors)) } > FLASH .text : { (.text) (.rodata) } > FLASH .data : { _sdata = .; (.data) _edata = .; } > SRAM AT > FLASH _sidata = LOADADDR(.data); .bss : { _sbss = .; (.bss) *(COMMON) _ebss = .; } > SRAM .noinit (NOLOAD) : { (.noinit) } > SRAM .stack (NOLOAD) : { . += 0x2000; _stack_top = .; } > DTCM }

Checking it fits. I don't trust the script, I trust the map file. arm-none-eabi-size gives the four numbers; the .map gives the addresses. On a 512 KB SRAM part: .data 12 KB + .bss 180 KB + DMA 96 KB + main stack 8 KB + 12 tasks × 4 KB = 344 KB, leaving 168 KB. If that margin is under ~20% I want to know before the build is frozen, because a task stack that grows 1 KB in a later commit eats it silently.

Failure modes I actually look for.

  • Stack overflow into .bss. Nothing faults by default. Fix: an MPU guard region below each task stack, plus a canary word and high-water-mark check the RTOS reports.
  • DMA into cached SRAM. The CPU reads stale data because the line was never invalidated. Either place .dma in a non-cacheable MPU region or clean/invalidate by exact address range, and the buffer must be aligned to the cache line, or invalidation discards a neighbour's dirty data.
  • .noinit that isn't. It survives a warm reset, not a power-on. Guard it with a magic word plus CRC and treat a mismatch as "no log".
  • Copy length drift. If _sidata/_sdata/_edata are hand-maintained rather than emitted by the script, .data gets truncated and one global starts life wrong. The map file shows it immediately.
  • Packed structs on a core without unaligned access. A uint32_t at offset 2 in a packed telemetry frame faults on load. Either unpack explicitly or confirm the core handles it.

After init there is no malloc. Pools are fixed-size and sized at build time, so the worst-case memory footprint is a number I can state in a review rather than a hope.

Curated: · Written: · Reviewed:

Q&A-6This thread has the higher priority. Why is it starving?

Reported in Why the SpaceX interview feels nothing like a FAANG loop

Higher priority only decides who runs when both threads are runnable. So before calling it starvation I'd separate three states: Blocked, Ready but not scheduled, and Running but not making progress. They have different causes, and the first one is what an interviewer usually means.

Blocked: priority inversion. A low-priority task holds a mutex the high-priority thread needs. A medium-priority task preempts the holder, so the high-priority thread waits behind work that has nothing to do with it. The wait is unbounded because any number of medium tasks can extend it.

Concrete trace on a 1 kHz attitude-control loop:

TimeEventRunning taskControl task state
0 µsLogger (prio 1) takes bus mutex, enters 200 µs critical sectionLoggerReady
50 µsControl (prio 10) wakes, tries mutex, blocksLoggerBlocked on mutex
100 µsTelemetry (prio 5) wakes, preempts LoggerTelemetryStill blocked
4.1 msTelemetry finishes; Logger resumesLoggerBlocked 4 ms, missed 4 deadlines

With priority inheritance, Logger runs at priority 10 until it releases the mutex. Control's worst-case wait becomes the 200 µs critical section plus scheduler overhead: bounded, and inside the 1 ms period. Without it, a 4 ms telemetry job turns a 200 µs lock into a 4 ms latency spike.

In FreeRTOS that difference is one API call:

c // FreeRTOS 10.5, configUSE_MUTEXES = 1 SemaphoreHandle_t bus_lock = xSemaphoreCreateMutex(); // inherits priority // xSemaphoreCreateBinary(); // does NOT inherit xSemaphoreTake(bus_lock, portMAX_DELAY);

You can see the boost at runtime: uxTaskGetSystemState() fills TaskStatus_t, and the holder's uxCurrentPriority rises above its uxBasePriority while it holds the mutex.

Ready but not running. Check the effective priority, not the one in the source. A thread can be pinned to a core that is saturated by a higher-priority interrupt thread, or its priority was changed at runtime and never restored. On an SMP target, a high-priority thread can also spin on a lock held by a lower-priority thread on another core. Priority inheritance does not apply to spinlocks, so that is genuine starvation with the scheduler doing exactly what it was told.

Running but slow. Priority is working; the thread is losing to cache pressure, memory bandwidth, or DMA contention from lower-priority cores. That shows up as high CPU time with low forward progress, not as a scheduling gap.

Other causes worth naming: a binary semaphore used as a lock (no owner, so no inheritance), a condition variable, a long interrupt-disabled section in a low-priority task, or an interrupt storm that keeps the thread from being dispatched at all. On non-RT Linux, a lower-priority task inside a non-preemptible kernel path delays the high-priority thread regardless of its nice value; PREEMPT_RT converts most spinlocks to PI mutexes, but raw_spinlock_t sections still do not inherit.

My first three checks are: task state and blocked-on object, lock owner's current versus base priority, and interrupt/IRQ-thread counts per core. That usually identifies the inversion in one trace capture.

Curated: · Written: · Reviewed:

Q&A-7What does `new` cost you here, and where would you put the allocation instead?

Reported in Why the SpaceX interview feels nothing like a FAANG loop

First I'd pin down which new: global operator new backed by a heap, or a class-specific overload. Assuming the former on a flight processor running a 1 kHz control loop, the cost that matters isn't the average: it's the tail and the failure mode. A glibc-style allocator fast path is maybe 50–100 ns on a 200 MHz Cortex-R, but a free-list search, coalescing, or a lock shared with a lower-priority task can turn that into microseconds and jitter. Worse, it can fail after 72 hours of fragmentation, and new then throws std::bad_alloc or, with -fno-exceptions, calls std::terminate. In a loop with a hard deadline, that's not a performance cost; it's a schedulability and safety cost. So I don't allocate in the loop. I allocate at initialization into static storage or a fixed pool, and use placement new or an O(1) free-list allocator.

WhereWorst-caseFragmentationUse
global new/mallocunbounded (free-list walk, lock)yesboot only
fixed poolO(1)noneper-type objects in loop
arena/bumpO(1)none; reset per cyclescratch buffers
stackO(1)nonesmall, bounded lifetime

A fixed pool is the usual answer for objects that live across cycles:

cpp template <typename T, size_t N> class FixedPool { alignas(T) std::byte buf_[N * sizeof(T)]; T* free_[N]; size_t n_ = N; public: FixedPool() { for (size_t i = 0; i < N; ++i) free_[i] = reinterpret_cast<T*>(buf_ + i * sizeof(T)); } T* create() { return n_ ? new (free_[--n_]) T() : nullptr; } void destroy(T* p) { p->~T(); free_[n_++] = p; } };

For a command queue, I'd use a ring of fixed-size slots sized to the largest command, with a reject or drop-oldest policy. For telemetry, double-buffered static arrays. For a parser that needs temporary nodes, a bump allocator over a static arena reset at the start of each frame. The allocation decision moves to initialization or compile time; the loop only takes and returns slots.

The failure modes I'd test: pool exhaustion under worst-case command burst, priority inversion if a mutex guards the pool, and silent truncation. Detection is high-water marks per pool, an assert on nullptr in test builds, a telemetry counter for allocation failures, and a linker map check that the heap is zero-sized or bounded. On a flight build I'd also grep the map for malloc and operator new to prove no hidden allocation from std::string, std::function, or a library call. If the interviewer pushes on variable size, bound it: a fixed maximum and a pool of that size. Truly unbounded allocation is a design error in flight software, not a runtime problem to manage.

Curated: · Written: · Reviewed:

Q&A-8The watchdog resets the board every ninety seconds in the field but never on the bench. Where do you start?

Reported in Why the SpaceX interview feels nothing like a FAANG loop

The 90-second answer

Two numbers get conflated here: the watchdog window and the 90 s. Ninety seconds is time-to-failure, not the timeout. On an STM32F4 the IWDG tops out at 4095 × 256 / 32 kHz ≈ 32.8 s (RM0090), and even at the slow end of the LSI spec it reaches only ~62 s, so either there's an external watchdog with a long window, or the kicker works fine for ~90 s and then something stops it. I go looking for what has a ~90 s period and only runs in the field.

First, the reset cause. RCC->CSR latches IWDG, WWDG, BOR, POR, PIN and software reset independently and survives the reset; read it before writing RMVF, which clears all of them at once.

RCC->CSR flagMeaningWhere to look
IWDGindependent watchdog firedkicker stopped, or window too tight
WWDGwindow watchdog firedkick too early or too late; clocked from PCLK1, so no LSI drift
BORsupply fell below threshold28 V bus transient, heater inrush, TX burst
POR / PINcold start or NRSTharness, connector, ESD
SFTRSTsoftwarean assert in an error path

Then persist a breadcrumb in .noinit so the next boot can say what the last one was doing:

c typedef struct { uint32_t magic; /* 0xB0A7C0DE / uint32_t boots; uint32_t tick_at_death; uint32_t task_at_death; uint32_t reset_flags; / RCC->CSR, read before clearing */ int16_t temp_c; uint16_t bus_mv; } crumb_t;

attribute((section(".noinit"))) crumb_t crumb;

void boot_log(void) { uint32_t csr = RCC->CSR; /* read first: RMVF write clears it */ if (crumb.magic == 0xB0A7C0DE) crumb.boots++; else { crumb.magic = 0xB0A7C0DE; crumb.boots = 1; } crumb.reset_flags = csr; RCC->CSR |= RCC_CSR_RMVF; }

Prime suspect: the 90 s log flush

A 128 KB sector erase on an F4 is on the order of 1 s typical, 2 s max: the flash characteristics table in the part's datasheet (DS9405 for F427/429; check your exact part). During program/erase the flash array is not readable, so execution from flash stalls at the next fetch. The ART accelerator's instruction cache can serve a few already-cached lines, which is why this looks intermittent rather than deterministic. So the first thing I check is where the kicker is linked, not assume. On the 2 MB dual-bank F42x/F43x parts you can execute from one bank while erasing the other; on single-bank parts the kicker and its ISR have to be linked into SRAM with the vector table relocated via SCB->VTOR for the kick to survive an erase.

Bench versus field: no accumulated log on the bench, so no erase, so no reset. That asymmetry explains "field only" better than anything else I'd guess at.

Instrument rather than argue. Toggle a GPIO at entry and exit of the kicker task and scope it. Kicker stops → the task is starved or blocked. Kicker runs and the board still resets → the write stalls the bus, or interrupts are masked past the window.

Other candidates, in order

  • LSI drift. The F4 LSI is 32 kHz typical, but the datasheet's min/max span is roughly 17–47 kHz over temperature and process, about ±50%. IWDG timeout scales inversely: a nominal 1.0 s window is ~0.68 s at 47 kHz and ~1.9 s at 17 kHz. A window that's comfortable on a 25 °C bench goes marginal in a cold soak. Detect it by logging temperature alongside the reset flags and comparing window margin at both extremes.
  • Supply. A heater duty cycle or radio TX burst pulling the 28 V bus through a brownout shows up as BOR, not IWDG, which is why reading CSR first matters.
  • A window watchdog kicked too early in a fast path that only runs in flight.

What I would not do

Lengthen the timeout, or kick from a timer ISR. The second is worse: it hides the hang and the watchdog stops being a watchdog. Reproduce in a thermal chamber at −40 and +85 °C, on the vehicle bus voltage with the real load and the transmitter keyed, and fix what actually stalls.

Curated: · Written: · Reviewed:

Q&A-9You have a 20 ms budget end to end. Which of these three calls can you not afford?

Reported in Why the SpaceX interview feels nothing like a FAANG loop

The 90-second answer

The three calls are on the board in front of me, so I'd read them back before answering: the decision turns on worst case, not typical, and I want to be sure which number I'm being shown. Taking the set that comes up in this loop: a 32-byte SPI burst read of the IMU, a 64-byte CAN FD telemetry frame, and a 4 KB log-sector write to external NOR flash. The one I can't afford is the flash write. Its typical time is already more than twice the entire budget, and its datasheet maximum is twenty times the budget. The other two are bounded and fit.

CallTypicalWorst caseBounded byVerdict in 20 ms
SPI IMU burst, 32 B @ 8 MHz32 µs32 µs + driver timeoutClock and timeoutAffordable
CAN FD frame, 64 B data @ 2 Mbit/s~280 µsunbounded under arbitration lossTx queue depth + timeoutAffordable if you bound it
NOR 4 KB sector erase45 ms400 msSiliconNot affordable

The arithmetic. SPI: 32 bytes = 256 bits ÷ 8 MHz = 32 µs, plus a few bytes of command overhead. CAN FD: ~69 bytes on the wire at the 2 Mbit/s data phase ≈ 280 µs, plus ~40 µs of arbitration at 500 kbit/s. Flash, from the Winbond W25Q128JV datasheet: 4 KB sector erase is 45 ms typical and 400 ms maximum; even a 256-byte page program is 0.7 ms typical, 3 ms max. 45 ms against a 20 ms end-to-end budget is not a close call: the erase alone overruns by 2.25× before the CPU has done anything else.

Budget the path, not the call. 20 ms end to end is not 20 ms for this call. Subtract sensor sampling, the control law, actuator output and jitter. If the loop closes at 100 Hz you have 10 ms per period, so a 20 ms path already spans two periods: anything that pushes a period boundary is a deadline miss, not a slow response. That's the second reason the flash write is out: it doesn't just cost time, it costs a period.

The trap in the set is the CAN frame. It looks cheap and it usually is, but if the bus is loaded and your frame keeps losing arbitration, the worst case is unbounded: you're waiting on other nodes' traffic. That's affordable only because you can bound it: a Tx queue with a fixed depth and a timeout that drops and counts. Unbounded is the thing you can't afford; the flash erase is just the obvious instance of it.

What I'd do instead. Never erase on the critical path. Double-buffer the log across two sectors, erase the spare from a background task during a maintenance window, and have the control path only ever do a page program into the already-erased buffer: 0.7 ms typical, and still off the hot path. For parameters that change often, FRAM or MRAM: byte-writable, no erase cycle, writes in microseconds.

c /* Control path: enqueue only. ~1 us, never blocks, never erases. */ static uint8_t ring[1024]; static volatile uint16_t head, tail;

bool log_try(const char s, size_t n) { if ((uint16_t)(head - tail) + n > sizeof ring) return false; / drop + count */ for (size_t i = 0; i < n; i++) ring[(head + i) % sizeof ring] = s[i]; head += n; return true; }

Failure modes I'd watch for. If the code executes XIP from that same NOR part, the erase stalls instruction fetch: the CPU isn't waiting, it isn't running, and every task misses, not just the writer. Second: priority inversion on the flash mutex, where a low-priority logger holds the lock through an erase and the 100 Hz control task blocks behind it while looking perfectly schedulable on paper. Fix is a priority-ceiling mutex and keeping flash and the sensor bus out of the same critical section. Detection: toggle a GPIO at loop entry and exit, capture on a scope, and telemeter a deadline-miss counter. Worst-case timing is bounded by analysis plus measurement margin, never by testing.

Curated: · Written: · Reviewed:

Q&A-10Walk me through how you bound the worst-case timing on a real-time system.

Reported in SpaceX Interview Questions & Process - Software Engineer

The 90-second answer

A worst-case bound is not one number for one function: it is a bound on the whole latency chain from sensor read to actuator write, and it has to survive the scheduler, not just the compiler. I do it in three layers: get a WCET for each task that is an upper bound rather than an average, add the interference the scheduler is allowed to inflict, then prove the sum fits the deadline with margin left over.

Concretely, on a 100 Hz control loop (T = 10 ms, hard deadline 10 ms) I want the static-analysis WCET, not the measured one, because measurement only proves the paths you happened to execute. On a Cortex-M7 at 480 MHz I log the high-water mark from the DWT cycle counter, never the mean:

c uint32_t t0 = DWT->CYCCNT; control_step(); uint32_t dt = DWT->CYCCNT - t0; /* 1 cycle = 2.08 ns / LOG_MAX(dt); / tail, not average */

Measured max comes back at 1.8 ms. Static analysis of the same binary (abstract interpretation over the machine code, cache and pipeline modelled) gives 2.6 ms. I carry 2.6 ms: the gap is the branch that only runs on a mode transition, which the bench never exercises.

Adding the interference

ContributorBoundSource
control task WCET2.60 msstatic analysis
IMU ISR, T = 10 ms0.12 msstatic + longest mask window
comms task, T = 20 ms0.90 msstatic analysis
blocking on shared SPI bus0.15 mslongest critical section held by a lower-priority task

Response-time analysis, iterating R = C + B + Σ ⌈R/Tᵢ⌉·Cᵢ over higher-priority tasks:

  • R₀ = 2.60 + 0.15 = 2.75 ms
  • R₁ = 2.75 + ⌈2.75/10⌉·0.12 + ⌈2.75/20⌉·0.90 = 3.77 ms
  • R₂ = 2.75 + 0.12 + 0.90 = 3.77 ms: fixed point

3.77 ms against a 10 ms deadline. Utilization is 0.26 + 0.012 + 0.045 ≈ 0.32, so it is comfortably schedulable, and I still want the bound under half the deadline because the analysis will be wrong somewhere.

Where the bound actually breaks

Cache and bus contention from DMA. The bench run has warm caches and an idle radio; in flight the comms DMA steals memory-bus cycles and evicts L2 lines under the control task. I either lock the hot path in L2 or run the timing-critical section with caches off, and I re-measure with DMA saturated: a 15–30% growth in WCET is normal and has to be inside the number, not discovered later.

Interrupt masking. Every disable_irq() window adds directly to latency. I measure the longest mask window with a GPIO toggle on a scope, not by reading the code, and it goes into the ISR bound above.

Priority inversion on the shared bus. A low-priority task holding the SPI lock while the control task waits is exactly the 0.15 ms blocking term; if the RTOS does not do priority inheritance, that term becomes unbounded and the analysis is void. I would rather double-buffer the sensor data and remove the lock.

Rare paths. Static analysis catches the once-per-flight branch; measurement never will. That is the reason I quote the analysis number and treat the measured one as a sanity check that the tool did not model something absurdly.

Finally, the bound is enforced at runtime, not just on paper: a deadline check in the scheduler trips a watchdog and safes the vehicle if control_step overruns. The analysis tells you it should not happen; the monitor is what you have when it does.

Curated: · Written: · Reviewed:

Q&A-11Design the task decomposition for an attitude control loop that must run at 50 Hz with 20 ms jitter budget.

Reported in www.refontelearning.com › blog › space-softwareSpace Software Engineer Interview Questions in 2026: What to ...

First I’d clarify what the 20 ms applies to: peak-to-peak jitter on the actuator command, or one-sided lateness? I’ll assume it’s the maximum deviation of the actuator update from its nominal 20 ms grid, and that the control law deadline is the end of each 20 ms period. That budget is one full period, which is loose for attitude control; I’d design to a few milliseconds and use 20 ms as the failure threshold.

Decomposition: a 1 kHz hardware timer as timebase, a 50 Hz control task at highest priority, and lower-rate housekeeping. The control task is triggered by the timer compare, not by an RTOS tick, so its release jitter is the timer interrupt latency, tens of microseconds, not the tick period. It does four bounded steps: read the latest IMU frame from a double buffer filled by SPI DMA, run the estimator, run the control law, write the actuator command. The actuator write is the last thing in the task and is a single atomic register or CAN frame write.

TaskTriggerPriorityWCETPeriod
IMU DMA complete ISRSPI DMAISR20 µs1 kHz
Attitude controltimer compare104 ms20 ms
Actuator outputinside control100.5 ms20 ms
Telemetry/log10 Hz timer52 ms100 ms
Health/watchdog1 Hz11 ms1 s
Command handlerevent81 mssporadic

Utilization is 4/20 + 2/100 + 1/1000 + … ≈ 0.22, so rate-monotonic is comfortably schedulable. The control task’s worst-case response is its own WCET plus blocking from lower-priority critical sections; I’d cap those at 100 µs with priority-ceiling mutexes.

Trace (hypothetical): timer fires at t=0, ISR latency 30 µs, control task starts at 0.05 ms. It reads the IMU frame sampled at t=-0.2 ms, estimator finishes at 2.1 ms, control law at 3.0 ms, actuator write at 3.4 ms. Worst-case blocking adds 0.1 ms, so the command lands by 3.5 ms: 16.5 ms before the deadline. Measured jitter is the variation of that write time; I’d log it with a GPIO toggle and a scope.

Failure modes: a telemetry task holding the SPI bus can delay the IMU read; fix with a priority-ceiling mutex or a separate bus. DMA overwriting the buffer mid-read; fix with double buffering and a sequence counter. Estimator iteration count varying with sensor validity; bound it and fall back to a fixed-gain filter. A missed actuator write must be detected by a deadline-miss counter and the watchdog; after two consecutive misses, command safe mode. No dynamic allocation, no unbounded loops, and the control code in zero-wait memory.

Curated: · Written: · Reviewed:

Q&A-12Design the software that receives commands from the ground and executes them on the spacecraft.

Reported in www.refontelearning.com › blog › space-softwareSpace Software Engineer Interview Questions in 2026: What to ...

The 90-second answer

State the assumptions first: this is a design for a generic LEO vehicle with a store-and-forward ground link, and the framing family is CCSDS because that is what most missions and ground stations speak. The pipeline below is one defensible design, not the only one: a mission that uplinks absolute-time command loads as files would put CFDP (CCSDS 727.0-B) in front of the same execution core.

The rule I build around: the uplink is an untrusted byte stream until it has passed framing, authentication and a dictionary lookup, and every command is a state transition that must be legal in the current vehicle mode.

Pipeline: TC Transfer Frame (CCSDS 232.0-B) → COP-1 reliable delivery (232.1-B) → Space Data Link Security (355.0-B) → dictionary decode → mode and interlock check → handler with a bounded worst-case execution time → acknowledgement telemetry and CLCW back down.

StageRejects onReported as
Frame sync + CRCbit errors, bad lengthframe discarded, counter incremented
COP-1sequence count outside window, duplicateN(R) not advanced; retransmit requested
SDLStag mismatch, replay of an old frameCMD_AUTH_FAIL, security event
Dictionaryunknown APID/opcode, short payloadCMD_UNKNOWN, CMD_BAD_LEN
Interlockillegal in current mode, not armedCMD_INTERLOCK, CMD_NOT_ARMED
Handleroverrun of its time budgetexecution-time monitor trips, safe mode

One precision on the security stage: 355.0-B defines the security header and trailer and the authenticated/encrypted frame format, but it does not fix the cipher. The mission picks the suite; AES-GCM is specified in the SDLS Extended Procedures (355.1-B). So "verify the tag" means the receiver runs whatever suite was configured, and it does so before the payload reaches the command dictionary: an unauthenticated frame never gets a lookup.

Critical commands are two-step. A command that changes attitude, fires a valve or enables a thruster is marked critical in the table and requires an arm command followed by an execute command from the same ground session inside a fixed window. The window length is a mission parameter, not a standard: pick it from the round-trip time plus operator reaction time and write it into the requirements. Arm state clears on window expiry, on any mode change, and on loss of the uplink. That defeats the two failure modes that actually hurt: a corrupted frame that happens to pass CRC, and a stale command replayed from a recorded pass.

c typedef enum { CMD_OK, CMD_BAD_LEN, CMD_UNKNOWN, CMD_OUT_OF_RANGE, CMD_INTERLOCK, CMD_NOT_ARMED, CMD_BUSY } cmd_rc_t;

typedef struct { uint16_t apid; uint8_t opcode; uint8_t min_len; bool critical; /* arm + execute, same session */ cmd_rc_t (*handler)(const uint8_t *p, uint8_t len); } cmd_entry_t;

static const cmd_entry_t cmd_table[] = { /* fixed at build time */ };

Trace. Ground sends SET_ATT_TARGET (APID 0x1A2, opcode 0x07, 12-byte payload) in a TC frame with V(S) = 0x3F1. The receiver accepts it, COP-1 advances, the SDLS check passes, the table lookup finds the handler, the range check confirms the quaternion is normalised within tolerance, the mode check passes, and the handler runs in 40 µs against a 2 ms budget. The vehicle sends an acknowledgement carrying the same virtual-channel sequence number and result=0; the next frame from the ground carries the CLCS in the CLCW, closing the loop. If the frame is lost, the ground retransmits after its timeout, the receiver sees the duplicate sequence number, discards it and re-sends the acknowledgement: the handler does not run twice.

Failure modes. A command storm from a misbehaving ground script is caught by a per-APID rate limit and a bounded queue depth; queue-full is telemetried, not swallowed. A handler that overruns is caught by an execution-time monitor rather than a watchdog reset, so the vehicle enters a defined safe mode instead of rebooting. Uplink loss mid-sequence leaves COP-1 retransmitting until its retry count expires; what the vehicle does then is a requirement you write, not something the protocol decides: hold last commanded state, or hand authority to onboard fault management.

The trade-off worth stating. More onboard autonomy means the vehicle keeps doing something useful through a comms gap; more ground authority means a human with the full picture approves every state change, at the cost of the vehicle doing nothing new while the link is down. The command path has to be robust either way, which is why the arm window and the mode interlocks are the parts I would get reviewed first.

Curated: · Written: · Reviewed:

Q&A-13Design the fault detection, isolation, and recovery (FDIR) system for a small satellite.

Reported in www.refontelearning.com › blog › space-softwareSpace Software Engineer Interview Questions in 2026: What to ...

The 90-second answer

Assume a LEO smallsat with 4–6 ground passes a day of 6–10 minutes each. Anything that needs a human inside one orbit is a lost mission, so FDIR has to be autonomous and layered: the cheapest tier that can fix the fault acts, and the expensive tiers only fire when it demonstrably can't.

Three tiers, each with its own detection and its own authority:

  1. Driver-level: bus timeout, CRC failure, range check. Retry once, then mark the unit unhealthy. Milliseconds.
  2. Subsystem health manager: a table of liveness and sanity checks per unit, evaluated at 1 Hz. Two consecutive failures → isolate the unit (power it off, exclude it from the attitude solution) and reconfigure onto a redundant unit or a degraded mode. Seconds.
  3. Vehicle supervisor: hardware watchdog fed by a heartbeat from the mode manager. Miss it → processor reset. Reset count over a threshold in a window → latch into safe mode: sun-point, power-positive, omni antenna, beacon, wait for ground. Tens of seconds.

The rule that makes it work: detection lives as close to the fault as possible, and recovery authority escalates only after the lower tier has failed. One dropped star-tracker frame must not put the spacecraft in safe mode.

TierDetectsLatencyActionAuthority
Driverbus timeout, CRC, rangemsretry ×1, mark unhealthycomponent
Health manager2 consecutive sanity failures @ 1 Hz~2 spower-cycle, swap to redundant, drop from solutionmode manager
Supervisorheartbeat missing1–2 sprocessor resethardware watchdog
Safe modereset count > 3 in 10 min, or critical threshold~30 ssun-safe, beacon, await groundlatched

Isolation is the hard part, not detection. A range check cannot catch a stuck sensor: the value is legal, it just stopped changing. You need dissimilar cross-checks: IMU-A against IMU-B, and both against the rate derived from star trackers. Trace it:

  • t=0: IMU-A returns the same value for 50 samples. Range check passes. Driver sees nothing.
  • t=1 s: health manager compares IMU-A to IMU-B and to the star-tracker rate. Residual > 3σ on two consecutive cycles → IMU-A declared failed.
  • t=2 s: IMU-A powered off, IMU-B promoted, attitude filter reinitialised with inflated covariance so the estimator doesn't fight the swap.
  • If IMU-B also fails within 10 minutes, that is not a sensor problem: it's a common-mode fault (power rail, thermal), so the supervisor goes to safe mode rather than swapping again.

Failure modes worth naming. False positives are the expensive one: a threshold set from datasheet noise instead of measured in-flight noise gives you a spurious safe mode a week, and you lose the science. Set thresholds from telemetry. The FDIR can itself become the fault: a watchdog reset loop where boot takes 20 s and the fault recurs before the next pass, so persist the fault counter in MRAM across resets and latch after N. And the configuration table that defines all of this lives in RAM and will take a single-event upset; store it with a CRC and reload from a radiation-tolerant copy, or the detector silently changes its own thresholds.

Two things I would insist on in design review: every FDIR action is logged with a timestamp to non-volatile memory so the next pass can reconstruct what happened, and each tier has an independent ground-inhibit command. And test safe mode for real: command it in thermal-vac and confirm power-positive attitude, not on a bench where the sun is a lamp.

Curated: · Written: · Reviewed:

Q&A-14Design a telemetry pipeline that thousands of Starlink satellites report into. What do you aggregate on-orbit versus on the ground?

Reported in SpaceX Hardware Engineering interview prep guide

The 90-second answer

Clarify the constraints first: each satellite gets a few minutes of downlink per gateway pass, the telemetry allocation is tens of kilobits per second, and the bus produces raw sensor data at hundreds of kilobits. The split follows information density. Aggregate high-rate, predictable signals on orbit into fixed-size summaries. Keep raw only for discrete events and for a rolling black-box buffer you dump when something breaks. The ground does fleet-wide correlation and long-term storage: it has the compute and the cross-satellite view the spacecraft never will.

Signal classExampleOn-orbit treatmentDownlink rate
High-rate continuousIMU, bus ripple, thermistors1 s window: min, max, mean, std, last, sample count~50 kbps
Low-rate stateMode, SoC, attitude quaternionRaw at 1 Hz~2 kbps
Discrete eventsFault flags, command acks, resetsRaw, event-triggered, sequence-numberedburst
Raw high-rateFull IMU streamRing buffer, last 10 min, only on triggeron demand

The arithmetic that decides it: 200 channels at 100 Hz, 4 bytes per sample, is 80 kB/s. 640 kbps raw. Summarising each channel to 32 bytes once per second gives 6.4 kB/s, 51 kbps. That is a 12.5× reduction, and carrying min and max in the window preserves the 10 ms transient a mean would erase.

On-orbit

The aggregator runs as a fixed-rate task in flight software. Each summary is a CCSDS space packet: 16-bit APID per channel group, 14-bit sequence count, GPS timestamp. The sequence count is what lets the ground detect a dropped packet instead of silently interpolating over it. The raw ring buffer lives in MRAM or a protected RAM region: 10 minutes at 640 kbps is 48 MB, so size it against the part you can actually afford, not the ideal.

Event-triggered data is separate. A threshold crossing or mode change emits a raw packet immediately and marks the ring buffer for snapshot. That is the only way to answer "what happened in the 200 ms before the fault" without streaming everything.

On the ground

Ingest checks sequence gaps, writes to a time-series store, and runs fleet analytics: cross-satellite correlation, orbital-position binning, long-term drift. Ground aggregation is where you compare satellite 4,212 against its neighbours: the spacecraft cannot see that.

Failure modes

Aggregation hides transients: a 10 ms voltage spike averaged into a 1 s mean disappears. Min/max in every window is the mitigation, and it costs 8 bytes per channel per second.

Clock loss: if GPS drops, timestamps drift. Fall back to a monotonic counter and reconstruct on the ground from packet arrival order and known pass geometry.

Downlink congestion: telemetry competes with payload and command acks. Priority queue with a hard cap on low-priority summaries; drop oldest first, never drop events.

Silent aggregator bug: a stuck channel looks like a flat line. Ground-side gap detection plus a heartbeat counter per APID catches it.

Curated: · Written: · Reviewed:

Q&A-15A test stand run produced anomalous sensor data. Walk through separating a sensor problem from a real hardware problem.

Reported in SpaceX Hardware Engineering interview prep guide

The 90-second answer

Two hypotheses: the measurement chain lied, or the article did something. Separate them with evidence independent of the suspect channel. The chain is transducer → signal conditioning → ADC → bus → logger, and each stage fails with a recognisable signature. First check the sensor's own health: excitation present, 4–20 mA loop between 3.6 and 21 mA, ADC not pinned at a rail, calibration date and last shunt-cal result. Then check physics: does the anomaly break a conservation law, appear only on channels sharing a harness or ADC card, or sit at a frequency the transducer cannot legitimately produce? Then perturb: swap in a known-good transducer, inject a shunt cal, or move the channel to a different acquisition card and re-run.

SignatureLikely locus
One channel moves; redundant channel on separate harness, ADC and power stays nominalSensor or its conditioning
All channels on one ADC card or one harness move togetherAcquisition, grounding, EMI
Reading pinned to 0 V or full scale, or loop current <3.6 mAOpen circuit, short, failed transducer
Anomaly at a fixed frequency unrelated to commanded eventsAliasing, clock coupling, digital artefact
Anomaly scales with excitation voltageRatiometric sensor or reference fault
Anomaly repeats across runs and persists after stimulus removedReal hardware

Worked trace. 10 bar transducer, 0.5 V/bar, 12-bit ADC over 0–5 V, sampled at 12.4 kHz. At T−0.2 s the channel reads 4.98 V for three samples: 9.96 bar, essentially full scale. The redundant transducer on a separate harness and ADC card reads 6.2 bar at the same timestamp. The suspect channel's loop current is 22.1 mA, above the 21 mA fault threshold. That is a chain fault: transducer or wiring, not a 9.96 bar event. Had both channels read 9.9 bar, the overpressure is real and the next question is which fluid or structural model explains it.

Aliasing arithmetic. 12.4 kHz sampling gives a 6.2 kHz Nyquist limit. A 9 kHz structural ring aliases to |12.4 − 9| = 3.4 kHz. If the "anomaly" is a clean 3.4 kHz tone that no commanded event explains, check whether a 9 kHz resonance exists: a 9 kHz ring is plausible on a pump or valve; a 3.4 kHz one that appears only when the igniter fires is probably coupling.

Calibration drift is not a step. A 0.1% FS/day drift on a 10 bar sensor is 10 mbar/day. A 0.5 bar jump inside one run is not drift; it is a discontinuity, and discontinuities come from opens, shorts, railed ADCs or real events.

Failure modes. The expensive mistake is confirming the scary hypothesis. A pressure spike during a hot-fire looks real because it is the outcome you fear, so teams chase the plant and ignore a railed ADC. The opposite error, dismissing a real overpressure as a sensor glitch, is worse. The guard is a pre-agreed rule: no anomaly is classified as sensor-only unless a second, independent measurement of the same quantity disagrees, or the channel fails a shunt cal. Write that rule into the test readiness review before the run, not after. Log raw ADC counts and loop current alongside scaled engineering units, so a railed converter is distinguishable from a genuine 10 bar reading.

Curated: · Written: · Reviewed:

Q&A-16How do you write application software differently when a bug could scrub a launch or degrade a satellite constellation?

Reported in SpaceX Hardware Engineering interview prep guide

The assumptions I'd state first: this is flight application software on a vehicle or satellite bus, it can command actuators or change attitude, and recovery may be a launch scrub or a ground command that takes hours. Under those constraints I write the failure path before the happy path. Four things change: the language subset, input validation, failure behavior, and verification.

ConcernOrdinary applicationFlight/constellation application
Memorymalloc/free, grow as neededstatic pools; no allocation after init
Loopswhile(true) with a breakbounded iteration; watchdog kick on a known path
Inputtrust the schemaCRC, range, plausibility, and voting on redundant sensors
Failurelog and retryenter safe mode, emit a telemetry event, wait for ground
Verificationunit + integrationrequirements trace, HIL, fault injection, MC/DC where required

The code-level rule is that a function touching hardware has one bounded, observable failure mode. I use MISRA C:2012 and the JPL Power of Ten as the baseline: no recursion, no dynamic allocation after init, bounded loops, checked return values, smallest scope, and static analysis with warnings as errors. That is not ceremony; recursion and unbounded loops make worst-case stack and timing unprovable, and on a flight processor an unprovable stack is a reset waiting for the right input.

A command handler shows the pattern. The snippet is C99:

c typedef enum { CMD_OK, CMD_BAD_LEN, CMD_BAD_CRC, CMD_OUT_OF_RANGE } cmd_result_t;

cmd_result_t stage_thruster_cmd(const uint8_t *buf, size_t len, struct thruster_cmd *staging) { if (len != THRUSTER_CMD_LEN) return CMD_BAD_LEN; if (crc32(buf, len - 4) != read_u32(buf + len - 4)) return CMD_BAD_CRC; uint16_t duration_ms = read_u16(buf + 2); if (duration_ms > MAX_BURN_MS) return CMD_OUT_OF_RANGE; staging->thruster_id = buf[0]; staging->duration_ms = duration_ms; return CMD_OK; }

The caller applies the staging struct only on CMD_OK. That matters: a handler that writes the thruster ID before checking the CRC can leave a partially applied command. I test that with fault injection: flip bit 5 in a command frame, assert the function returns CMD_BAD_CRC, no actuator command changes, and telemetry carries the rejection reason. If the test sees a thruster ID written, the handler is wrong.

Failure behavior is designed, not discovered. Every state machine has a safe state reachable from any other state, and the transition is telemetered. On a constellation, safe mode has to be safe for the constellation, not just the vehicle: a satellite that stops responding to ground or emits on the wrong beam can degrade neighbors. So I bound autonomy: the satellite can enter safe mode without ground, but it cannot leave it without an authenticated command. I also size the watchdog to the longest bounded path plus margin. If safe-mode entry takes 200 ms and the watchdog fires at 100 ms, the processor resets before safe mode; I inject a hang at the worst point and measure.

Verification is where most of the cost goes. Requirements trace to tests, tests run on host and then on hardware-in-the-loop, and fault injection is a first-class test suite, not an afterthought. I also review for what the compiler cannot see: shared state between an interrupt and the main loop, priority inversion, and a sensor that returns a plausible but wrong value. The trade-off is speed. I tier by criticality: flight-critical code gets the full process; payload and ground tooling get less. Redundancy and voting add their own failure modes, so I prefer a simple single string with a watchdog and a reachable safe mode over triple redundancy I cannot test.

Curated: · Written: · Reviewed:

Q&A-17Implement a lock-free ring buffer for telemetry data ingestion.

Reported in SpaceX SDE interview questions coding and system design 2026 ...

The 90-second answer

First I'd pin down the topology, because "lock-free" means different things depending on it. Telemetry ingestion on a flight board is almost always single-producer/single-consumer: one DMA completion ISR or one sensor task writes, one downlink/logger task reads. That case has a genuinely wait-free implementation: no CAS, no retry loop, two index stores. If there are multiple producers I'd say so up front, because the design changes.

Assumptions I'd state: power-of-two capacity so the index wrap is a mask, monotonic 32-bit counters so full and empty are distinguishable by difference rather than equality, and a producer that must never block. Dropping a sample is acceptable, stalling the ISR is not.

c #include <stdatomic.h> #include <stdint.h> #include <stdbool.h>

#define RB_SIZE 4096u /* power of two */ #define RB_MASK (RB_SIZE - 1u)

typedef struct { _Alignas(64) _Atomic uint32_t head; /* producer writes / _Alignas(64) _Atomic uint32_t tail; / consumer writes */ _Alignas(64) uint8_t data[RB_SIZE]; } rb_t;

bool rb_push(rb_t *rb, const uint8_t src, size_t n) { uint32_t head = atomic_load_explicit(&rb->head, memory_order_relaxed); uint32_t tail = atomic_load_explicit(&rb->tail, memory_order_acquire); if (RB_SIZE - (head - tail) < n) return false; / full: drop */ for (size_t i = 0; i < n; i++) rb->data[(head + i) & RB_MASK] = src[i]; atomic_store_explicit(&rb->head, head + n, memory_order_release); return true; }

size_t rb_pop(rb_t *rb, uint8_t *dst, size_t n) { uint32_t tail = atomic_load_explicit(&rb->tail, memory_order_relaxed); uint32_t head = atomic_load_explicit(&rb->head, memory_order_acquire); size_t avail = head - tail; if (avail < n) n = avail; for (size_t i = 0; i < n; i++) dst[i] = rb->data[(tail + i) & RB_MASK]; atomic_store_explicit(&rb->tail, tail + n, memory_order_release); return n; }

The whole correctness argument is the ordering table, and this is where candidates get caught:

AccessOrderWhy
producer loads headrelaxedonly it writes head
producer loads tailacquiremust see the consumer's frees before overwriting
producer stores headreleasepublishes the payload writes before the index
consumer loads headacquirepairs with the producer's release
consumer stores tailreleasefrees slots for the producer

Get the acquire on the producer's tail read wrong and it passes every test on x86-64 (TSO won't reorder those stores) and corrupts data on the Cortex-A/R core in the flight computer, under load only. That is the failure mode I'd name: silent frame corruption that reproduces on target and never on the dev box.

Overrun policy and sizing. The producer drops newest and bumps a saturating dropped counter; the consumer reads that counter and emits a gap marker so ground never sees a silent hole. Drop-oldest is tempting but in SPSC only the consumer may write tail, if the producer advances it you've broken the single-writer invariant and the acquire/release pairing no longer proves anything. If you truly need drop-oldest, the consumer drains to the newest record on wakeup instead.

Sizing is arithmetic, not vibes. 2 kHz of 128-byte telemetry records is 256 KB/s; a 64 KiB ring is 256 ms of slack. If the consumer's worst stall (a flash erase, say) is 200 ms, that fits with margin; if it's 400 ms you drop, and you should know that before flight rather than after.

Things I'd flag as follow-ups. _Alignas(64) on head and tail is false-sharing defence: one shared line means every producer store invalidates the consumer's line, roughly 100 ns per bounce versus ~1 ns from L1. On Apple silicon the line is 128 B, so I'd pad to that if the code is portable. If the producer is a DMA engine rather than the CPU, the ring is only half the problem: you need cache invalidate-by-address on the buffer before the consumer reads it, and the head store must happen after the transfer-complete interrupt, not before. And on a Cortex-M0/M0+ there is no LDREX/STREX, so C11 atomics lower to library calls that mask interrupts: the code is still correct but it is no longer lock-free, and I'd say that out loud rather than let the label stand.

For multiple producers I'd reach for one SPSC ring per producer with the consumer draining round-robin, which keeps the wait-free property, over a CAS-based queue like Vyukov's where a contended CAS on Cortex-M runs 40–80 cycles and can livelock a high-rate ISR.

Curated: · Written: · Reviewed:

Q&A-18Here's broken code for a subsystem: what's the failure mode, and how do you fix it?

Reported in SpaceX SDE interview questions coding and system design 2026 ...

The 90-second answer

I don't start editing. I ask three things first: what the symptom is (hang, wrong value, intermittent), whether it reproduces on the bench or only in flight, and what optimisation level and toolchain built it. Then I read the code against the short list of failure classes that actually bite embedded subsystems, and pick the one whose signature matches the symptom.

ClassSignatureHow I confirm it
ISR/task shared statefails under load, or at a fixed periodmask interrupts around the read; if it vanishes, that's it
Missing volatile / reorderingclean at -O0, broken at -O2diff the disassembly
Stack overflowcorrupts an unrelated variablefill stack with 0xA5, read the high-water mark
Uninitialised peripheral or clockhangs on first register accessread back the RCC enable bits
Integer boundarywrong only at 0, 0x7FFF, 0xFFFFboundary tests, not random ones

Given a sensor-ingest path, my first bet is a 64-bit counter shared between an ISR and a task on a 32-bit core. It isn't atomic, and it only corrupts once every 2^32 samples: exactly the bug that passes a two-day bench run.

c /* broken: Cortex-M4, gcc -O2 / static volatile uint64_t samples_ok; / ISR writes, task reads */

void ADC1_2_IRQHandler(void) { uint16_t v = (uint16_t)ADC1->DR; ring[head & (N - 1)] = v; head++; samples_ok++; /* 64-bit read-modify-write */ }

uint32_t samples_since_last_check(void) { uint64_t now = samples_ok; /* compiles to two 32-bit loads */ uint32_t d = (uint32_t)(now - last); last = now; return d; }

volatile stops the compiler caching the value in a register. It does not make the access atomic. The ISR's increment is load-low, add, store-low, store-high; the task's read is load-low, load-high. Interleave them at the wrong instant and the task assembles a value that never existed.

The trace. Counter at 0x0000_FFFF_FFFF_FFFF (281,474,976,710,655), sampling at 10 kHz:

StepActorValue
1task loads low word0xFFFF_FFFF
2ISR completes the increment0x0001_0000_0000_0000
3task loads high word0x0001
4task assembles now0x0001_FFFF_FFFF_FFFF

The task reads 4,294,967,295 samples that don't exist. d truncates to 0xFFFF_FFFF, so whatever consumes it (a rate estimator, a dropout detector) divides by a nonsense delta. The low word rolls over every 4.29e9 / 1e4 ≈ 429,497 s ≈ 5 days, which is why it survives a bench campaign and surfaces on orbit.

The fix. I don't mask interrupts in a control path; that adds jitter to the loop that matters. Read high, low, high and retry if high moved:

c static inline uint64_t samples_snapshot(void) { uint32_t hi, lo, hi2; do { hi = (uint32_t)(samples_ok >> 32); lo = (uint32_t)samples_ok; hi2 = (uint32_t)(samples_ok >> 32); } while (hi != hi2); return ((uint64_t)hi << 32) | lo; }

If hi is unchanged across the window, no carry happened, so lo belongs to that hi. Cost is two extra loads and no lock: a handful of cycles on a 168 MHz M4.

The second thing I'd flag in the same function: ring[head & (N - 1)] = v has no full check. If the task stalls longer than N sample periods, the ISR silently overwrites unread samples while samples_ok keeps climbing, so the health check reports a perfectly healthy subsystem that is dropping data. That needs a head - tail == N test and a separate dropped counter.

How I'd prove it before changing anything. Seed samples_ok = 0xFFFF_FFFF in the debugger, breakpoint the task's read, and single-step the two loads with the ISR live. Or ship a debug build that snapshots twice and asserts monotonicity: at 10 kHz it fires within minutes once the boundary is seeded.

Curated: · Written: · Reviewed:

Q&A-19How would you design a fault-tolerant, real-time flight control system? What are the key software architecture considerations?

Reported in 30+ Popular SpaceX Engineering Technical Interview Questions ...

The 90-second answer

Assume a launch vehicle: three flight computers in a voting set, a 100 Hz control loop (10 ms hard deadline), and the requirement to stay fail-operational through ascent. One fault must not lose the vehicle. The architecture falls out of four decisions: how many redundant channels, how their outputs are voted, how faults are detected and isolated, and how the schedule stays deterministic.

Three computers, each running identical control software, cross-strapped to the same sensor set and to the actuator drivers. Each computes an actuator command every 10 ms; a voter (hardware, or a software voter on a non-voting monitor) takes the median. One channel can be wrong in any way, including Byzantine, and the median still tracks the two good ones. Two simultaneous faults need five channels; that's arithmetic, not preference.

Determinism is the second axis. The loop is a fixed-rate cyclic executive, not a preemptive RTOS whose priority inversions I have to reason about under fault. Every task has a measured WCET, the frame is laid out statically, and there is no dynamic allocation after init: no malloc, no exceptions, no unbounded loops. A 10 ms frame might budget:

PhaseBudgetNotes
Sensor acquisition2.0 ms3 IMUs + rate gyros over serial
Navigation + control law4.0 msWCET-measured, fixed iteration count
Actuator output1.0 msCAN to TVC and engine controllers
Telemetry + logging0.5 msnon-blocking ring buffer
Margin2.5 msabsorbs jitter, not new work

Voting and sensor selection

Mid-value select on every triplicated analog channel. Median-of-three tolerates one arbitrary faulty value: with readings 1.00, 1.02, 40.0 the output is 1.02, and with 1.00, 1.02, −40.0 it is 1.00. Two faults require five channels and a median-of-five; anything less is a single-fault-tolerant design pretending otherwise.

c /* median of three; NaN channels are dropped before this is called */ static float mid3(float a, float b, float c) { if (a > b) { float t = a; a = b; b = t; } if (b > c) { b = c; } return (a > b) ? a : b; }

NaN and out-of-range values are rejected upstream by a plausibility gate: rate limit against the previous sample, and comparison against the other two channels, because a NaN that reaches the median silently poisons the vote.

Fault detection and isolation

Layered, because no single mechanism catches everything:

FaultDetectionResponse
IMU stuck-highdisagreement vs. peers + rate limitdrop channel, continue on two, flag
FCC hangwindow watchdog on peer heartbeatpeers exclude it from the vote
Bus CRC errorscounter over thresholdswitch to redundant bus
Second channel lostvote set drops below twosafe mode or commanded abort

A window watchdog, not a plain timeout: it catches a task that finishes too fast as well as one that hangs, which is how you find a corrupted jump table before it flies. Every inter-computer link carries a CRC and a sequence counter, so a stale frame is discarded rather than voted on.

Failure modes worth naming

Common-mode failure is the one that kills TMR designs: three channels sharing one power rail, one clock, or one software bug. Independence has to be physical and in the code path: separate regulators, separate oscillators, and diverse implementations for the highest-criticality checks. The second is a voter that becomes the single point of failure; it gets its own watchdog and a defined output when it loses quorum. The third is timing: a fault handler that runs longer than the frame it interrupts turns a recoverable fault into a missed deadline, so handlers are bounded and the deadline monitor trips before the loop overruns.

Detection in test: hardware-in-the-loop with fault injection (force a channel high, kill a heartbeat, corrupt a CRC) and confirm the vehicle stays inside its control envelope. That's the evidence the design works, not the block diagram.

Curated: · Written: · Reviewed:

Q&A-20Explain the difference between a process and a thread. How does this matter for embedded/flight software?

Reported in 30+ Popular SpaceX Engineering Technical Interview Questions ...

The first 90 seconds

A process is the unit of isolation: its own virtual address space, its own file descriptors, its own heap and globals. A thread is the unit of scheduling inside a process: its own stack and register state, sharing everything else with its siblings. Two threads in one process can scribble on each other's memory by accident; two processes cannot without an IPC call the kernel mediates.

On flight software that is usually not a choice you get to make. Most flight computers (a Cortex-R, a LEON) either have no MMU or run an RTOS that doesn't enable one, so there is one flat address space and "threads" (VxWorks tasks, RTEMS tasks, FreeRTOS tasks) are all you have. The real question is what you gave up by not having processes, and what you do about it.

ProcessThread / RTOS task
Address spaceprivate, MMU-mappedshared, one flat map
Stackper thread, inside the processown stack, size fixed at create
Switch costregisters + TLB/ASID changeregister set only
Fault containmentfault kills the process; supervisor restarts itcorruption is silent until it isn't
CommunicationIPC: queues, shared memory, socketsdirect pointer, needs a lock or a lock-free structure

What the missing boundary costs, concretely

c /* FreeRTOS on Cortex-M7. Two tasks, one address space. */ static uint8_t telemetry_buf[512];

void vTelemetryTask(void p) { for (;;) { size_t n = build_frame(telemetry_buf, sizeof telemetry_buf); xQueueSend(telemetry_q, &n, portMAX_DELAY); / sends the length */ } }

void vDownlinkTask(void p) { size_t n; for (;;) { xQueueReceive(telemetry_q, &n, portMAX_DELAY); radio_send(telemetry_buf, n); / buffer may be half-rewritten */ } }

The queue passes a length, not the data. vTelemetryTask is higher priority, so it will preempt between the send and the radio read, and radio_send transmits a frame that is half old, half new. Under a process model with a message queue the kernel copies the 512 bytes and the bug cannot exist. Here the fix is to copy into the queue (512 bytes × depth) or double-buffer with an index handoff. That is the trade in one example: IPC costs a copy, and the copy is what buys the safety.

The arithmetic that decides it

Stack budget: 8 tasks × 4 KB = 32 KB of a 256 KB SRAM part. 12.5% of RAM on stacks before any heap. A process model adds page tables and region descriptors on top. On an ARMv7-M MPU you get 8 regions (16 on ARMv8-M), so you can carve out a handful of guarded regions, not one per task.

Switch cost: a Cortex-M PendSV switch saving r4–r11 plus EXC_RETURN is roughly 100–200 cycles. Under 1 µs at 200 MHz. A process switch on a Cortex-A with a cold TLB is tens of microseconds, and the TLB miss path is not bounded. In a 20 ms control loop, 1 µs is noise; an unbounded MMU walk is not, which is why hard-real-time flight code stays away from it.

Where processes do show up in flight software

  • cFS: one process, many tasks, on RTEMS or Linux. Isolation comes from the Software Bus and table validation, not the MMU.
  • ARINC-653 partitions (VxWorks 653, PikeOS): each partition gets its own memory region and a CPU budget inside the major frame. That is how a payload experiment and flight-critical guidance share one box.
  • Linux on a companion computer: processes for fault containment, and you accept scheduler jitter.

Failure modes to name

A thread overruns its stack into a neighbour's TCB: no fault, just wrong values three subsystems away. You catch it with MPU guard regions and stack high-water-mark checks, not with a crash. Priority inversion across a shared mutex disappears entirely once a queue replaces the mutex. And watchdog design follows from the same choice: per-thread heartbeats let one stuck thread hide behind the others still kicking, while a per-process watchdog resets everything. Pick which failure you would rather have.

What I'd say in the room: on flight hardware I default to one process, tasks, and MPU-guarded stacks, and I buy isolation with design (no shared mutable state, queues that copy) rather than with the MMU, because the MMU costs determinism I can't get back.

Curated: · Written: · Reviewed:

Q&A-21How do you guarantee message delivery during network failure?

Reported in SpaceX System Design Interview: The Complete Guide

The 90-second answer

You can't guarantee delivery across a link that fails; you guarantee a contract and make the failure explicit. Split traffic by class. Commands are at-most-once: each carries a 32-bit command ID, the receiver dedups on it, and the sender never auto-retries a destructive command (arm, burn, deploy) without fresh operator authorisation. Telemetry is at-least-once: sequence-numbered, persisted to a store-and-forward queue, retransmitted on ack timeout, deduplicated at the receiver. Both are bounded by storage, so the real design decision is the drop policy when the buffer fills, and that policy is written down and tested before launch.

ClassGuaranteeMechanismWhen the buffer fills
Command (safe)at-least-onceseq + ack + retransmit, dedup windownever drop; block producer
Command (destructive)at-most-oncenonce + explicit ack, no auto-retryreject at sender
Housekeeping telemetryat-least-oncepersistent ring, seq, gap markerdrop oldest, set gap flag
High-rate sciencebest-effortcircular buffer, downsampledownsample 10:1

Trace. Flight computer sends CMD seq=42 (set heater duty to 60%). Ground receives at t=2 ms, executes, sends ACK 42. The ack is lost in a fade. At t=500 ms the FC retransmits 42. Ground finds 42 in its 1024-entry dedup bitmap, re-sends ACK, does not re-execute. At t=502 ms the FC advances its tail pointer. One execution, two transmissions.

Code. The queue is a persistent ring in NOR flash; tail is the oldest unacked entry, head the next write.

c typedef struct { uint32_t seq; uint8_t cls; uint16_t len; uint8_t payload[64]; } msg_t; static msg_t log[4096]; /* 4096 * 72 B ≈ 288 KB */ static uint32_t head, tail;

void send_next(void) { if (tail == head) return; transmit(&log[tail % 4096]); /* ack handler: on ACK(seq), advance tail past seq */ }

Receiver side keeps a uint32_t seen[32] bitmap over a 1024-seq window; a seq older than tail - 1024 is treated as already delivered.

Sizing. A 100 kbps downlink for a 10-minute blackout is 100e3/8 × 600 = 7.5 MB. That does not fit an 8 MB NOR part alongside code, so housekeeping drops to 1 Hz and high-rate data lives in a 60 s circular buffer. The gap is marked in the next packet header, so ground processing knows the telemetry is incomplete rather than wrong.

Failure modes. Lost ack → duplicate suppressed by the dedup window; window wrap → 32-bit seq and a window smaller than half the space; buffer full → drop oldest telemetry, never commands; blackout longer than the buffer → downlink a summary and a gap marker; timeout tuning → base retransmit on link state (SNR, frame error rate), not wall clock, or you retransmit into a link that is still down.

Why not TCP. TCP gives retransmit and ordering, but head-of-line blocking puts a command behind 200 telemetry segments, and its backoff assumes round trips of milliseconds, not a 20-minute one-way light time. A custom protocol lets you prioritise commands and choose per-class semantics.

Curated: · Written: · Reviewed:

Q&A-22Write an algorithm to count the number of distinct islands in a grid where 1 represents land and 0 represents water.

Reported in 30+ Popular SpaceX Engineering Technical Interview Questions ...

The first 90 seconds

Two things decide the answer before I write a line: does diagonal adjacency count, and can I mutate the grid? I assume 4-connectivity and a mutable buffer, and I say so out loud. On a flight processor the third constraint is memory: no malloc in the scan loop, and no recursion. The stack on these targets is a few KB, and a snake-shaped island thousands of cells long would blow it. So: iterative flood fill with an explicit stack, marking cells as I push them so nothing is pushed twice.

c #include <stdint.h> #define ROWS 64 #define COLS 64

static uint16_t stack[ROWS * COLS]; /* each cell pushed at most once */

static void flood(uint8_t g[ROWS][COLS], int r0, int c0) { static const int8_t dr[4] = {-1, 1, 0, 0}; static const int8_t dc[4] = {0, 0, -1, 1}; int top = 0; g[r0][c0] = 0; stack[top++] = (uint16_t)(r0 * COLS + c0); while (top) { int idx = stack[--top]; int r = idx / COLS, c = idx % COLS; for (int k = 0; k < 4; k++) { int nr = r + dr[k], nc = c + dc[k]; if (nr >= 0 && nr < ROWS && nc >= 0 && nc < COLS && g[nr][nc]) { g[nr][nc] = 0; /* mark on push, not on pop */ stack[top++] = (uint16_t)(nr * COLS + nc); } } } }

int count_islands(uint8_t g[ROWS][COLS]) { int n = 0; for (int r = 0; r < ROWS; r++) for (int c = 0; c < COLS; c++) if (g[r][c]) { n++; flood(g, r, c); } return n; }

Checked against C11, gcc 13. Marking on push is what bounds the stack at ROWS*COLS entries: 8 KB here. Marking on pop lets the same cell be queued from two neighbours and the stack grows past that bound.

Trace on a 4x5 grid, row-major scan:

1 1 0 0 1 1 0 0 0 0 0 0 1 1 0 0 0 1 0 0

(0,0) is land → n=1, flood marks (0,0) (0,1) (1,0). (0,4) → n=2, flood marks it alone. (2,2) → n=3, flood marks (2,2) (2,3) (3,2). Every other cell is water. Returns 3. Each cell is examined once by the scan and each land cell pushed once, so time is O(R·C) and the stack is O(R·C) worst case.

Alternatives and when to pick them

ApproachTimeExtra memoryUse when
DFS/BFS flood fillO(R·C)O(R·C) stackgrid fits in RAM, one pass
Union-find, row by rowO(R·C·α)two rows of labels + equivalence tableframe too big for RAM, streaming
Recursive DFSO(R·C)O(R·C) call framesnever on a target with a small stack

A 4096x4096 frame at one byte per pixel is 16 MB: it does not fit in internal SRAM, so you stream it: label each row, union with the row above where land touches, then a second pass to resolve labels to roots. That is the version I would ship for imagery, or for connected-component labelling on a telemetry fault bitmap.

Failure modes I would name

  • Recursion depth. A 64x64 worst case is 4096 frames at roughly 40 bytes each, ~160 KB against a 4 KB stack. The symptom is a hard fault or a corrupted return address, not a wrong count. Detect with a stack canary and an MPU guard region.
  • Const input. If the grid is a DMA buffer you cannot write, keep a visited bitmap instead: ROWS*COLS/8 = 512 bytes for 64x64.
  • Index width. uint16_t overflows the moment ROWS*COLS > 65535; use uint32_t and size the stack accordingly.
  • Connectivity ambiguity. On 1 0 / 0 1 the count is 2 with 4-connectivity and 1 with 8-connectivity. That is a spec question, not an implementation detail: ask it before coding.
  • Reentrancy. The file-scope stack makes count_islands non-reentrant; if two tasks can call it, move the stack to the caller or guard it with a mutex.
  • Frontier sizing. A comb-shaped island can hold thousands of frontier cells at once in a BFS queue; size for the worst case or use the stack form above.

Curated: · Written: · Reviewed:

Q&A-23Walk me through how you would architect a fault-tolerant command propagation system.

Reported in SpaceX's Interview Process (2026) - TechPrep

The 90-second answer

I'll assume a ground station uplinking discrete commands to a flight computer over two redundant links, with cross-strapped processors A and B driving shared actuators. The goal is not "exactly once": that's unachievable across an unreliable link, since a lost ACK leaves you genuinely unable to tell "never arrived" from "executed, reply lost." The goal is at-most-once actuation: never fire twice, retransmit until acknowledged, and surface the ambiguous case to the operator instead of guessing.

That splits into three layers, each with its own job.

LayerMechanismCatches
FrameCCSDS TC transfer frame, 16-bit FECF, COP-1 go-back-N retransmitbit errors, dropped frames
CommandCRC over the command segment plus an authentication code (SDLS, or an HMAC keyed per vehicle)corruption, forgery
Executorepoch + sequence number, dedup window, cached ACKduplicates, replays, reordering

CCSDS TC gives you an 8-bit frame sequence count, so it wraps at 256; COP-1's window width is a managed parameter and has to stay under 128 or old and new frames become indistinguishable. I'd run 8 frames in flight. The retransmit timer comes from the round trip: LEO is tens of milliseconds, so 100 ms is generous; a GEO bird at 36,000 km has ~120 ms one-way light time, so the timer is nearer 400 ms.

Trace. Ground sends command 0x1A3F, sequence 0x2C01, to open a valve. Flight A validates CRC and authentication, opens it, sends ACK. The ACK dies in a fade. Ground retransmits after the timer. A finds 0x2C01 in its dedup window, replays the cached ACK, and does not pulse the valve again. B, in standby, records 0x2C01 in its own window without actuating. If A dies and B promotes, B still refuses to re-execute 0x2C01.

Failure modes. Both processors believing they're active is the one that hurts: a hardware ownership line plus a heartbeat lease, and a dual-active counter that forces both to standby. A stale time-tagged command after a clock step: reject anything older than a couple of seconds or with a mismatched epoch. Sequence wrap: bump the epoch on wrap and on every boot, so the dedup window resets cleanly. A bus stuck dominant: three consecutive framing errors declares the link failed and the redundant one takes over. Instrument per-link CRC error counts, command age, and ACK latency; a watchdog on the command handler resets it if its heartbeat stops.

Alternatives. Link-level retransmit alone can't stop a duplicate whose ACK was lost: executor-side dedup is mandatory, not belt-and-braces. Triple-modular voting covers compute faults but not the command path. Stored command sequences suit known events, but need the same authentication and the same dedup.

Curated: · Written: · Reviewed:

Q&A-24Build a telemetry packet parser.

Reported in SpaceX's Interview Process (2026) - TechPrep

The 90-second answer

I'd clarify the wire format and rate first: sync word, length field, CRC, max payload, byte rate, and whether the parser runs in interrupt or task context. Assuming a byte stream from a UART DMA ring buffer at up to 3 Mbaud, packets with a 2-byte sync, 16-bit length, 16-bit sequence, 32-bit timestamp, up to 256 bytes of payload, and a CRC-16/CCITT trailer, I'd build a non-blocking parser that scans for sync, validates length and CRC, and returns a pointer to the payload. It never allocates, has bounded work per call, and resyncs by advancing one byte after any error. That survives a noisy downlink without locking up.

FieldBytesEncoding
sync20xAA 0x55
length2little-endian payload length, 0–256
sequence2little-endian, wraps at 65535
timestamp4little-endian, microseconds since boot
payload0–256protocol-defined, big-endian fields
CRC2CRC-16/CCITT over header+payload

c #define SYNC0 0xAAu #define SYNC1 0x55u #define HDR 10u #define MAX_PAYLOAD 256u #define CRC 2u

typedef struct { uint16_t seq; uint32_t ts; const uint8_t *payload; uint16_t plen; } pkt_t;

/* returns bytes consumed; caller keeps the remainder */ size_t parse(const uint8_t *b, size_t n, pkt_t out) { size_t i = 0; while (i + HDR + CRC <= n) { if (b[i] != SYNC0 || b[i+1] != SYNC1) { i++; continue; } uint16_t plen = (uint16_t)b[i+2] | ((uint16_t)b[i+3] << 8); if (plen > MAX_PAYLOAD) { i += 2; continue; } size_t total = HDR + plen + CRC; if (i + total > n) break; / incomplete, wait */ uint16_t want = (uint16_t)b[i+total-2] | ((uint16_t)b[i+total-1] << 8); if (crc16_ccitt(b + i, total - 2) != want) { i++; continue; } out->seq = (uint16_t)b[i+4] | ((uint16_t)b[i+5] << 8); out->ts = (uint32_t)b[i+6] | ((uint32_t)b[i+7] << 8) | ((uint32_t)b[i+8] << 16) | ((uint32_t)b[i+9] << 24); out->payload = b + i + HDR; out->plen = plen; return i + total; } return i; }

Trace: a good packet AA 55 04 00 01 00 00 00 00 00 DE AD BE EF 12 34 has plen=4, total=16, CRC over the first 14 bytes matches 0x3412, so parse returns 16 and the caller dispatches the 4-byte payload. A corrupted length AA 55 01 01 gives plen=257, which exceeds MAX_PAYLOAD; the parser skips two bytes and scans for the next sync. A bad CRC advances one byte, so a false sync inside a payload costs one CRC attempt, not a lost stream.

Arithmetic: at 3 Mbaud 8N1, one byte every 3.33 µs, 300 kB/s. A 76-byte packet at 1 kHz is 76 kB/s, 25% of the link. A 4 KB DMA ring buffer gives 4 KB / 76 kB/s = 53 ms of slack. The scan loop does at most one CRC per byte position; a CRC-16 over 76 bytes is a few hundred cycles on a 100 MHz Cortex-M7, so a worst-case resync through a 4 KB buffer is millions of cycles: tens of milliseconds. That is too long for a 1 ms task, so I'd either bound resync work per call or use a byte-wise state machine that never re-scans.

Trade-offs: the scan-based parser is simple and stateless, but a payload full of AA 55 pairs can force O(n²) CRC work. For high-rate telemetry I'd use a byte-wise state machine that never re-scans, or a 4-byte sync and CRC-32 to make false syncs negligible. Zero-copy returns a pointer into the ring buffer; that requires the consumer to finish before the DMA wraps, so I'd copy into a static packet buffer if the consumer can block. CRC-16 catches burst errors up to 16 bits; CRC-32 if the link is noisier or packets are larger.

Failure modes: length-field corruption is contained by the max-length check before any buffer write. CRC collision is 1 in 65536, so I'd add sequence-gap detection and range checks on payload fields before acting on them. Unaligned casts fault on Cortex-M, so every multi-byte field is assembled byte by byte. A 32-bit microsecond timestamp wraps every 71.6 minutes; the consumer must handle wrap or the parser should emit 64-bit. And a parser that blocks on a partial packet at the end of a DMA buffer will drop the next packet: return consumed bytes and keep the remainder.

Curated: · Written: · Reviewed:

Q&A-25Design a control system for a physical actuator.

Reported in SpaceX's Interview Process (2026) - TechPrep

The 90-second answer

Before designing anything I'd pin down four numbers and one word: stroke and max rate, the load it has to hold and move, the accuracy and settling time, and the safe state. For a grid fin that's usually "hold position with damping"; for a propellant isolation valve it's "closed". Then the shape of the answer: this is a cascaded loop closed locally on the actuator controller, not on the flight computer.

Flight computer sends a position setpoint at 200 Hz over CAN FD with a rolling counter and CRC. The actuator controller, a lockstep MCU like a TMS570, runs three nested loops: position at 200 Hz, velocity at 1 kHz, motor current at 20 kHz on the PWM. Position feedback is an LVDT on the output ram, cross-checked against the motor encoder through the gear ratio and ball-screw lead.

Cascade because the inner current loop makes the motor look like a torque source, which linearises the plant and lets the outer loops be tuned one at a time. It also gives a hard torque limit for free: clamp the current setpoint and the actuator physically cannot exceed it, whatever the outer loop does.

Loop rates and why

LoopRateCrossoverFeedbackOutput
current20 kHz~1 kHzshunt + ADCPWM duty
velocity1 kHz~100 Hzencoder deltacurrent cmd
position200 Hz~10 HzLVDTvelocity cmd

Rule I use: crossover at 1/10 to 1/20 of the loop rate. Sampling delay is about 1.5 samples: at 20 kHz that's 75 µs, which costs 27° of phase at 1 kHz and leaves 45–60° of margin once the plant's own lag is counted. Same arithmetic on the position loop: 7.5 ms delay, 27° at 10 Hz.

The position loop

c /* 200 Hz position loop, C11, timer ISR on the actuator controller. Gains are Q12 fixed point so worst-case ISR cycles are identical every tick. */ static int32_t integ, prev_err;

void pos_loop(void) { int32_t cmd = setpoint_rate_limited(); /* counts, slew-limited upstream */ int32_t fb = lvdt_read(); if (!sensors_agree(fb)) { actuator_safe(); return; }

int32_t err = cmd - fb;
int32_t p   = (KP * err) >> 12;
int32_t d   = (KD * (err - prev_err)) >> 12;
prev_err = err;

int32_t i_next = integ + ((KI * err) >> 12);
int32_t u = p + i_next + d + vel_feedforward();

if (u > I_MAX)       u = I_MAX;          /* clamp, and don't wind up */
else if (u < -I_MAX) u = -I_MAX;
else                 integ = i_next;

current_setpoint(u);

}

Two things there matter more than the gains. The integrator only accumulates when the output isn't saturated: otherwise a jammed actuator winds up and then slams when it frees. And the setpoint is slew-limited upstream, because a step demand saturates the current loop and rings the gearbox.

Failure modes

  • Sensor disagreement. LVDT and encoder must agree within 2% of stroke. Three consecutive bad samples is a fault. Don't average them: a disagreement means one is lying and you don't know which.
  • Loss of command. Counter stops incrementing for 3 frames (15 ms) → hold last setpoint, ramp current to zero over 100 ms, then safe state.
  • Runaway. Position moves more than a set threshold while the command is constant → drop the enable line.
  • Thermal. I²t on the winding; derate the current limit above 120 °C.
  • The enable line is a separate GPIO driven by a dead-man timer in the ISR, not a register write in the control path. If the control task hangs, the hardware cuts PWM without software cooperation.

On the stand: sine sweep 0.1–50 Hz for the frequency response, step response for overshoot and settling, a stall test to confirm the clamp holds and the integrator doesn't wind up, and a check that the first structural mode sits well above the position crossover, if it doesn't, add a notch rather than detuning the loop.

Curated: · Written: · Reviewed:

Curated: · Written: · Reviewed: