The structural response to high pressures as well as to high temperatures of (1−)PbTiO3−Bi(Zn0.5Ti0.5)O3 with =0.08, (1−)PbTiO3−Bi(Mg0.5Ti0.5)O3 with =0.17, and PbTiO3 single crystals was studied by in situ pressure-/temperature-dependent polarized Raman spectroscopy, complemented by synchrotron x-ray diffraction analysis at ambient pressure and different temperatures. The compositional and pressure dependencies of phonon anomalies indicate the existence of local-scale antiferrodistortive structural entities inside the perovskite (ABO3) polar tetragonal matrix, the fraction of which is enhanced by the substitution of BiO3 for PbTiO3. The type of cation (Zn2+ versus Mg2+) replacing Ti4+ at the B site affects the coherence within the dominant single-perovskite polar tetragonal matrix and the coupling between the “defect” double-perovskite antiferrodistortive entities. As a result, on increasing pressure, the partial substitution of Bi(Mg0.5Ti0.5)O3 triggers a structural instability at lower pressure values as compared to pure PbTiO3, whereas that of Bi(Zn0.5Ti0.5)O3 preserves the polar tetragonal phase over a wider pressure range. Furthermore, at ambient pressure B-site Zn2+ favors the coherence between local BO6 tetragonal distortions already above the Curie temperature , causing a larger unit-cell tetragonality below ; in contrast, B-site Mg2+ reduces the local BO6 anisotropy and disturbs the correlation between polar AO12 tetragonal distortions below , which results in a considerable reduction of both the unit-cell tetragonality and accumulated volume strain on cooling between and room temperature.