A locally metric connection on a smooth manifold M is a torsion-free connection D on TM with compact restricted holonomy group Hol(0) (D). If the holonomy representation of such a connection is irreducible, then D preserves a conformal structure on M. Under some natural geometric assumption on the life-time of incomplete geodesics, we prove that conversely, a locally metric connection D preserving a conformal structure on a compact manifold M has irreducible holonomy representation, unless Hol(0) (D) = 0 or D is the Levi-Civita connection of a Riemannian metric on M. This result generalizes Gallot's theorem on the irreducibility of Riemannian cones to a much wider class of connections. As an application, we give the geometric description of compact conformal manifolds carrying a tame closed Weyl connection with non-generic holonomy.