The dynamics of all four combinations of possible polarity and circularity states in a stack of two vortices is investigated by time-resolved scanning transmission x-ray microscopy. The vortex stacks are excited by unidirectional magnetic fields leading to a collective oscillation. Four different modes are observed that depend on the relative polarizations and circularities of the stacks. They are excited to a driven oscillation. We observe a repulsive and attractive interaction of the vortex cores depending on their relative polarizations. The nonlinearity of this core interaction results in different trajectories that describe a two-body problem.