Let A in be an n by n matrix and let B be an n by p matrix and consider the Lyapunov matrix equation AX+XA^T+BB^T=0. If A+A^T < 0, then the extended Krylov subspace method (EKSM) can be used to compute a sequence of low rank approximations of X. In this paper we show that any positive residual history is possible for the EKSM for Lyapunov matrix equations. In addition, we show how to systematically construct linear time invariant systems for which it is impractical to approximate the action of the product of the system Gramians using the EKSM. This is a property of the underlying Lyapunov matrix equations, rather than a defect of the algorithm.