Two‐dimensional elastic and viscoelastic phononic crystals structures are investigated theoretically and experimentally. These are two‐dimensional arrays of cylinders of air inclusions in a solid silicone rubber matrix as well as arrays of silicone rubber cylinders in an air matrix. In this study, viscoelasticity is modeled with a compressible general linear viscoelastic fluid model. Numerical calculations of transmission spectra and band structures are conducted by extending the finite difference time domain (FDTD) method to account for linear viscoelastic materials which exhibit time‐dependent moduli. These systems demonstrate very wide band gaps in their transmission spectra that extend to frequencies in the audible range of the spectrum. Experimental and calculated transmission spectra for a rubber matrix/air inclusions phononic crystal show that the system behaves as a fluid/fluid composite with respect to the longitudinal polarization of acoustic waves due to the large contrast between the transverse and longitudinal speeds of sound. We also demonstrate that viscoelasticity can attenuate transmission over very wide ranges of frequency leaving only passing bands at very low frequency. These phononic crystals demonstrate the practical design of elastic or viscoelastic solid rubber/air acoustic band gap sound barrier with small dimensions.