We investigate the decay of artificially created double occupancies in a repulsive Fermi-Hubbard system in the strongly interacting limit using diagrammatic many-body theory and experiments with ultracold fermions in optical lattices. The lifetime of the doublons is found to scale exponentially with the ratio of the on-site repulsion to the bandwidth. We show that the dominant decay process in presence of background holes is the excitation of a large number of particle-hole pairs to absorb the energy of the doublon. We also show that the strongly interacting nature of the background state is crucial in obtaining the correct estimate of the doublon lifetime in these systems. The theoretical estimates and the experimental data are in agreement.