For a multiple input channel, one may define different capacity regions, according to the criterions of error, types of codes, and presence of the feedback. In this paper, we aim to draw a complete picture of relations among these different capacity regions. In particular, we prove that the average error capacity region of a multiple input channel can be achieved by a random code under the criterion of maximum error probability. Moreover, we show that for a non-deterministic or Gaussian multiple input channel with feedback, the capacity regions are the same under two different error criterions. To illustrate the relation, we discuss two special classes of channels. Our results are extended to conpund channels and AVC as well.