Conventionally, even with sensitive single-photon detectors, a large number of detected photons is needed at each sensor pixel location to mitigate Poisson noise. Here, using only the first detected photon at each pixel location, we capture both the 3D structure and reflectivity of the scene, demonstrating greater photon efficiency than previous work. Our computational imager combines physically accurate photon-counting statistics with exploitation of spatial correlations present in real-world scenes. We experimentally achieve millimeter-accurate, sub pulse width depth resolution and 4-bit reflectivity contrast, simultaneously, using only the first photon detection per pixel, even in the presence of high background noise. Our methods enable rapid, low power, noise-tolerant active imaging.