The spectral gap g of an ergodic and reversible Markov chain is an important parameter measuring the asymptotic rate of convergence. In applications, the transition matrix P may be unknown, yet one sample of the chain up to a fixed time t may be observed. Let m be the minimal value of the stationary distribution. Hsu, Kontorovich, and Szepesvari (2015) showed that if t = 1/(g^3 m), then g can be estimated to within multiplicative constants with high probability. We prove that in fact 1/(g m) steps suffice. When the stationary distribution is uniform, this is known to be optimal.