A seminal paper of David Blackwell (published in 1956) introduces the notion of approachability. A key result in the study of repeated games, Blackwell's approachability theorem has been used extensively in online learning. We develop a framework for analyzing a wide range of sequential problems. First, the framework captures both statistical learning (that is, learning with i.i.d. data) and online learning (that is, worst-case sequences). Second, our framework includes Blackwell's approachability, internal and Phi-regret, prediction of individual sequences, calibration of forecasters, global non-additive notions of cumulative loss, and more. In particular, our generic results extend Blackwell's approachability theorem to games in (infinite-dimensional) separable Banach spaces. Further, we define complexity measures which capture the difficulty of learning in a sequential manner. Among these measures are analogues of Rademacher complexity, covering numbers and combinatorial dimensions from statistical learning theory. These can be seen as temporal generalizations of the classical results.