We consider the effects of incorporating prior knowledge of features which correlate with phoneme identity as well perceptual invariances to design SVM kernels for phoneme classification in high-dimensional spaces of acoustic waveforms of speech. To this end we explore products and linear combinations of  polynomial and radial basis function kernels to design composite kernels which are invariant to waveform sign and time shift, and capture the dynamics of energy evolution in the time-frequency plane.  Experiments show marked improvements in phoneme classification as a result of this custom kernel design. This demonstrates that even in high-dimensional feature spaces, careful kernel design based on prior knowledge of the problem domain can have significant payback.