Using ab initio density functional theory applied to semiconductor nanoclusters, we show that optical experiments where absorption is involved probe vibrations that are not necessarily the ones with large exciton-phonon matrix elements. The vibrations involved in these experiments must satisfy more stringent symmetry constraints than given by the selection rules of the exciton-phonon matrix elements. This resolves a long-standing observed discrepancy, while offering a viable theoretical approach to properly account for the experimental situation.