We derive force field potentials for CdS, CdSe, CdTe, ZnS, ZnSe, and ZnTe using the vibrational and elastic properties of the bulk materials based on density functional theory calculations and experiment. The potentials have bond-stretching (first- and second-nearest neighbor), bond-bending, as well as long-range Coulomb interactions. The bulk vibrational properties are very well reproduced for all materials considered. The vibrational properties of nanoclusters are compared to the results of large-scale density functional theory calculations with up to 1000 atoms. We obtain a very good agreement for all the core vibrational modes and an overall reasonable agreement over the entire spectrum. However, the low-frequency acoustic-type modes with surface character tend to be too soft in the force field calculations. We demonstrate the capability of the force field to obtain vibrational properties of realistic colloidal quantum dots with sizes up to 60 Å diameter.