Using optical pump-terahertz (THz) time-domain spectroscopy measurements along with ab initio density functional theory (DFT) calculations, we studied the effect of 3,9-bis(2-methylene-(3-(1, 1-dicyanomethylene)-5-methylindanone)-5,5,11,11-tetrakis(4-hexylphenyl)-dithieno[2,3-d:2′,3′-d′]-s-indaceno [1,2-b:5,6-b′]-dithiophene (IT-M) molecules on the ultrafast dynamics and photoconductivity of the active layers of ternary organic solar cells (OSCs), which are composed by the conjugated polymer donor and fullerene derivative acceptor. The THz photoconductivity measurements show that the introduction of IT-M molecule increases the photoconductivity of the ternary OSC and the maximum value of photoconductivity is obtained in the sample with IT-M weight ratio of 15%. To reveal the mechanism of IT-M doping on the photoconductivity of OSC material, ab initio DFT calculations were performed to identify the band structures of the ternary OSC heterostructures. Resonant charge transfers induced by the band alignment of IT-M with the conjugated polymer donor and fullerene derivative acceptor, which lead to an increase of photoconductivity with a reduction of electron-hole recombination rate, were obtained from DFT calculations. In addition to the resonant charge transfer process, defect scattering induced by the doping of IT-M molecules leads to a reduction of photoconductivity. By competing these two effects, maximum photoconductivity of ternary OSC materials is obtained with IT-M weight ratio of 15%. Moreover, the charge carrier densities and scattering times of ternary OSC materials were derived from the measured THz photoconductivity via the classical Drude-Smith model. With these material properties, we identified that the maximum value of photoconductivity in OSC material with IT-M weight ratio of 15% is induced by its long scattering time instead of high carrier density.