Time-resolved soft-x-ray photoemission spectroscopy is used to simultaneously measure the ultrafast dynamics of core-level spectral functions and excited states upon excitation of excitons in WSe2. We present a many-body approximation for the Green's function, which excellently describes the transient core-hole spectral function. The relative dynamics of excited-state signal and core levels clearly show a delayed core-hole renormalization due to screening by excited quasifree carriers resulting from an excitonic Mott transition. These findings establish time-resolved core-level photoelectron spectroscopy as a sensitive probe of subtle electronic many-body interactions and ultrafast electronic phase transitions.