The recent wealth of discoveries in deep learning has coincided with the development of specialized automatic differentiation frameworks which can efficiently propagate gradients through repeating structures in artificial neural networks. For model-based approaches, automatic differentiation still performs relatively poorly and it is common to formulate gradients manually or to focus on low-dimensional problems. To accelerate research into model-based control of high-DOF robots such as humanoids with articulated hands and to enable hybrid approaches that combine model-based methods with deep learning, we develop a novel automatic differentiation framework that can evaluate gradients of robot models around previous candidate solutions multiple times faster than state-of-the-art methods.