Technical interview guide

Neural Network Fundamentals

Forward pass, backpropagation, and the activation functions that make deep networks work.

Read: 27 min
Practice MCQs: 25
Interview QA: 25
Edition: v3
Editorial status: Reviewed
Relevant for: AI EngineerMLOps EngineerMachine Learning Engineer

Scope: Google ML Crash Course neural-network guidance, TensorFlow Keras guides, and PyTorch stable reproducibility guidance accessed 2026-08-31..

Interview QA

Treat each question like a live interview question: answer out loud first (structure, assumptions, tradeoffs), then open the model answer to spot gaps and rehearse a tighter follow-up.

Curated: 2026-10-05 · Written: 2026-09-10 · Reviewed: 2026-10-03

QA-1

Explain how a feed-forward neural network makes a prediction.

QA-2

Why do neural networks need activation functions?

QA-3

Compare sigmoid, tanh, and ReLU.

QA-4

How do you choose an output layer and loss?

QA-5

Explain backpropagation.

QA-6

How would you debug a neural network whose loss does not decrease?

QA-7

How would you debug exploding or vanishing gradients?

QA-8

Why does weight initialization matter?

QA-9

Compare SGD, momentum, and adaptive optimizers.

QA-10

How do you choose learning rate and schedule?

QA-11

How does batch size change optimization and generalization?

QA-12

Compare batch normalization and layer normalization.

QA-13

How do dropout and batch normalization differ between training and inference?

QA-14

How do you regularize a neural network?

QA-15

How do you evaluate whether a neural network is better than a simpler model?

QA-16

How do you make neural-network experiments reproducible?

QA-17

What should a training checkpoint contain?

QA-18

How do you export and validate a neural network for serving?

QA-19

How do mixed precision and loss scaling affect training?

QA-20

How would you investigate training that works on CPU but diverges on GPU?

QA-21

How do you detect data leakage in a neural-network pipeline?

QA-22

How do you monitor a neural network in production?

QA-23

A model has great training accuracy but poor validation accuracy. What do you do?

QA-24

How would you design a neural-network training test suite?

QA-25