We consider the problem of estimating the probability of a string
drawn i.i.d. from an unknown distribution. The key feature of this
problem is that the length of the observed string is of the same order
as the size of the underlying alphabet. The classical Turing estimator
(introduced by I. J. Good) provides the traditional answer to this
problem. We introduce a natural scaling formulation to study this
problem and use it to show that probability estimation using the
traditional (Good-)Turing estimator is not  consistent (in a
statistical sense). Drawing from the intuition derived from this
exercise, we introduce a novel probability estimator that is indeed
strongly consistent with the natural scaling model.