Active galactic nuclei (AGNs) with jets seen at small viewing angles are the most luminous and abundant objects in the γ-ray sky. AGNs with jets misaligned along the line of sight appear fainter in the sky but are more numerous than the brighter blazars. We calculate the diffuse γ-ray emission due to the population of misaligned AGNs (MAGNs) unresolved by the Large Area Telescope (LAT) on the Fermi Gamma-ray Space Telescope (Fermi). A correlation between the γ-ray luminosity and the radio-core luminosity is established and demonstrated to be physical by statistical tests, as well as compatible with upper limits based on Fermi-LAT data for a large sample of radio-loud MAGNs. We constrain the derived γ-ray luminosity function by means of the source-count distribution of the radio galaxies detected by the Fermi-LAT. We finally calculate the diffuse γ-ray flux due to the whole MAGN population. Our results demonstrate that MAGNs can contribute from 10% up to nearly the entire measured isotropic gamma-ray background. We evaluate a theoretical uncertainty on the flux of almost an order of magnitude.