It has been shown in previous work that the modular group acts projectively on the center of a factorizable ribbon Hopf algebra. The center is the zeroth Hochschild cohomology group. In this article, we extend this projective action of the modular group to an arbitrary Hochschild cohomology group of a factorizable ribbon Hopf algebra, in fact up to homotopy even to a projective action on the entire Hochschild cochain complex.