I will describe a two-stage approach to image segmentation. In the first stage, a bottom-up model produces a small diverse set of hypothesized segmentations. In the second stage, these are ranked, possibly with a more complex top-down model, to select the best hypothesis. I will focus on the second stage, discuss issues that arise in learning the ranking model, and briefly describe the results our approach achieves on some segmentation benchmarks.