Q&A-1A teammate passes a CUDA tensor into np.array() and gets a RuntimeError, then tries a float64 numpy array multiplied by a float32 CUDA tensor and gets a different result than expected. Explain what each failure is, why it happens, and how you would write the code so the same pipeline works on CPU and GPU.
Answer
Short answer. A numpy array lives in host memory, and numpy's ufuncs run on the CPU. A torch tensor carries a device and a dtype, and its storage can sit on a CUDA device. Numpy has no concept of a device, so it cannot read GPU memory. np.array(t) goes through the array protocol, which calls .numpy(), and that fails because a CUDA tensor has no host buffer to return. The fix is an explicit device-to-host copy, .cpu().numpy(). That copy synchronises with the GPU and pays for a transfer.
Mixed-type arithmetic is a separate failure. The result dtype and device come from each library's promotion and dispatch rules, and numpy and torch do not agree on them. Do not predict the result from memory. Print it.
# PyTorch 2.x, NumPy 1.26 or 2.x. Check the exact exception text in your environment.
import numpy as np
import torch
t = torch.ones(3, device='cuda', dtype=torch.float32)
try:
np.array(t) # goes through the array protocol, which calls .numpy()
except Exception as e:
print(type(e).__name__, e) # the message names the cuda device
host = t.cpu().numpy() # explicit device-to-host copy, then a host view
Dtype promotion. Torch promotes a Python scalar against a tensor without widening the tensor, and a 0-dim tensor does not upcast a dimensioned tensor of the same category. Those rules cover tensor-with-tensor and tensor-with-scalar operations. An ndarray operand has to be converted first, so the float64 array times float32 CUDA tensor case is the one to test, not to reason about:
a = np.array([1.1, 2.2]) # float64, host
x = torch.tensor([1.1, 2.2], device='cuda') # float32, device
r = x * a
print(type(r), r.dtype) # check both the type and the dtype
Make the conversion explicit so the result type is fixed by your code rather than by operator dispatch:
x * torch.as_tensor(a, dtype=x.dtype, device=x.device) # float32 result on cuda
This converts the array once, to the tensor's dtype and device. The float64 to float32 rounding happens at that line, on purpose, and a reviewer can see it.
Same code on CPU and GPU. Keep data preparation in numpy on the host. Choose the device once, and convert at a single boundary:
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
batch = torch.as_tensor(np_batch, dtype=torch.float32, device=device)
From there, the training path stays in tensors and never calls back into numpy.
Failure modes and how to detect them.
-
Silent host round-trips.
float(t),t.item(),t.numpy()or.cpu()inside a training step forces a sync every iteration. Profile withtorch.profileror Nsight Systems. Step time rises, and the GPU timeline shows idle gaps around the call. Loggingloss.item()every step is the usual cause. -
Shared memory on CPU.
torch.from_numpyshares storage with the array, andtorch.as_tensordoes too when no conversion is needed. Writes through either side show up on the other, and an in-place op on the tensor changes the numpy array. Detect it by hashing the array before and after the call, or by checkingnp.shares_memory. Usetorch.tensor(a)when you need an independent copy. -
Dtype drift in metrics. Accumulating a loss in float32 over millions of steps drops low-order bits. Run a short job with a float32 accumulator and a float64 one, then compare the relative difference before choosing the accumulator dtype.
Curated: · Written: · Reviewed: