This paper considers randomized distributed control for automated demand response in the power grid, and other applications. The results are summarized as follows: (i) The Markov Decision Process framework of Todorov is extended to continuous time models, in which the "control cost" is based on relative entropy. (ii) Step (i) defines a family of Markovian generators, parameterized by a weighting parameter. A central control authority chooses this parameter in real-time based on the aggregate output of the Markovian agents; this is the basis of the mean field model. (iii) Provided the control-free system is a reversible Markov process, it is found that the linearized model is passive. In conclusion: control of the aggregate is easy with this architecture!