We present a method for model-based viscoelastic characterization using shear wave elastography. The shear wave displacement signal is differentiated to a series of fractional derivative orders, and the group speed of the signal at each derivative order is calculated. These speeds are compared to a look up table built from simulations for a given viscoelastic material model. The best-fit material parameters are chosen to minimize the mean square difference between the group speeds from the look up table and experimental measurements across all fractional derivative orders. Various two- and three-parameter viscoelastic material models are tested. The technique is validated in simulations with ultrasonic tracking and demonstrated in three viscoelastic phantoms with material properties that match those associated with different degrees of liver fibrosis. Comparatively, phase velocity curves are computed, and parameters determined by fitting analytic expressions for phase velocity for a given material model. The fractional derivative group speed-based technique gives model parameters with lower mean square error than fitting material model parameters directly to phase velocities. Additionally, we conclude these viscoelastic phantoms are sufficiently characterized by a two-parameter material model and are better characterized by the Linear Attenuation model than the more commonly used Voigt model.