The recent experimental condensation of ultracold atoms in a triangular optical lattice with a negative effective tunneling parameter paves the way for the study of frustrated systems in a controlled environment. Here, we explore the critical behavior of the chiral phase transition in such a frustrated lattice in three dimensions. We represent the low-energy action of the lattice system as a two-component Bose gas corresponding to the two minima of the dispersion. The contact repulsion between the bosons separates into intra-and intercomponent interactions, referred to as V-0 and V-12, respectively. We first employ a Huang-Yang-Luttinger approximation of the free energy. For V-12/V-0 = 2, which corresponds to the bare interaction, this approach suggests a first-order phase transition, at which both the U(1) symmetry of condensation and the Z(2) symmetry of the emergent chiral order are broken simultaneously. Furthermore, we perform a renormalization-group calculation at one-loop order. We demonstrate that the coupling regime 0 < V-12/V-0 <= 1 shares the critical behavior of the Heisenberg fixed point at V-12/V-0 = 1. For V-12/V-0 > 1 we show that V-0 flows to a negative value, while V-12 increases and remains positive. This results in a breakdown of the effective quartic-field theory due to a cubic anisotropy and, again, suggests a discontinuous phase transition.