We explore channel capacity and rate-distortion for channels and sources sampled at sub-Nyquist rates. We first show that periodic sampling achieves capacity among all non-uniform sub-Nyquist subsampling structures. We also derive sampling methods that minimize the worst-case (minimax) capacity loss due to channel-independent (universal) sampling design. We next study rate-distortion of sub-Nyquist sampled stationary Gaussian sources corrupted by noise. In particular, we determine the generalized sampler and source encoder that minimizes distortion between the source and its reconstruction. We also show that with sufficiently high (sub-Nyquist) sampling rates, multibranch sampling achieves the lower bound on the distortion. Sub-Nyquist joint source and channel coding is also discussed.