For many random optimization problems we have by now very sharp estimates of the satisfiable regime. At the same time, though, all known polynomial-time algorithms only find solutions in a very small fraction of that regime. We study this phenomenon by examining how the statistics of the geometry of the set of solutions evolve as constraints are added. We prove in a precise mathematical sense that, for each problem studied, the barrier faced by algorithms corresponds to a phase transition in that problem’s solution-space geometry. Roughly speaking, at some problem-specific critical density, the set of solutions shatters and goes from being a single giant ball to exponentially many, well-separated, tiny pieces. All known polynomial-time algorithms work in the ball regime, but stop as soon as the shattering occurs. Besides giving a geometric view of the solution space of random optimization problems our results establish rigorously a substantial part of the 1-step Replica Symmetry Breaking picture of statistical physics for these problems.