We perform large-scale ab initio density functional theory calculations to study the lattice strain and the vibrational properties of colloidal semiconductor core-shell nanoclusters with up to one thousand atoms (radii up to 15.6 Å). For all the group IV, III-V, and II-VI semiconductors studied, we find that the atom positions of the shell atoms seem unaffected by the core material. In particular, for group-IV core-shell clusters the shell material remains unstrained, while the core adapts to the large lattice mismatch (compressive or tensile strain). For InAs-InP and CdSe-CdS, both the cores and the shells are compressively strained corresponding to pressures up to 20 GPa. We show that this compression, which contributes a large blueshift of the vibrational frequencies, is counterbalanced, to some degree, by the undercoordination effect of the near-surface shell, which contributes a redshift to the vibrational modes. These findings lead to a different interpretation of the frequency shifts of recent Raman experiments, while they confirm the speculated interface nature of the low-frequency shoulder of the high-frequency Raman peak.