We extract an invariant taking values in N boolean OR\{infinity\}, which we call the order of algebraic torsion, from the Symplectic Field Theory of a closed contact manifold, and show that its finiteness gives obstructions to the existence of symplectic fillings and exact symplectic cobordisms. A contact manifold has algebraic torsion of order 0 if and only if it is algebraically overtwisted (i.e. has trivial contact homology), and any contact 3-manifold with positive Giroux torsion has algebraic torsion of order 1 (though the converse is not true). We also construct examples for each k is an element of N of contact 3-manifolds that have algebraic torsion of order k but not k - 1, and derive consequences for contact surgeries on such manifolds. The appendix by Michael Hutchings gives an alternative proof of our cobordism obstructions in dimension three using a refinement of the contact invariant in Embedded Contact Homology.