We construct the actions of a very broad family of 2d integrable $\sigma$-models. Our starting point is a universal 2d action obtained in [arXiv: 2008.01829] using the framework of Costello and Yamazaki based on 4d Chern-Simons theory. This 2d action depends on a pair of 2d fields $h$ and $\mathcal{L}$, with $\mathcal{L}$ depending rationally on an auxiliary complex parameter, which are tied together by a constraint. When the latter can be solved for $\mathcal{L}$ in terms of $h$ this produces a 2d integrable field theory for the 2d field $h$ whose Lax connection is given by $\mathcal{L}(h)$. We construct a general class of solutions to this constraint and show that the resulting 2d integrable field theories can all naturally be described as $\mathcal{E}$-models.