We develop a high order accurate numerical method for solving the elastic wave equation in second-order form. We hybridize the computationally efficient Cartesian grid formulation of finite differences with geometrically flexible discontinuous Galerkin methods on unstructured grids by a penalty based technique. At the interface between the two methods, we construct projection operators for the pointwise finite difference solutions and discontinuous Galerkin solutions based on piecewise polynomials. In addition, we optimize the projection operators for both accuracy and spectrum. We prove that the overall discretization conserves a discrete energy, and verify optimal convergence in numerical experiments.