Two-particle correlations in relative azimuthal angle (Delta phi) and pseudorapidity (Delta eta) are measured in root S-NN = 5.02 TeV p + Pb collisions using the ATLAS detector at the LHC. The measurements are performed using approximately 1 mu b(-1) of data as a function of transverse momentum (p(T)) and the transverse energy (Sigma E-T(Pb)) summed over 3.1 < eta < 4.9 in the direction of the Pb beam. The correlation function, constructed from charged particles, exhibits a long-range (2 < vertical bar Delta eta vertical bar < 5) ``near-side{''} (Delta phi similar to 0) correlation that grows rapidly with increasing Sigma E-T(Pb). A long-range ``away-side{''} (Delta phi similar to pi) correlation, obtained by subtracting the expected contributions from recoiling dijets and other sources estimated using events with small Sigma E-T(Pb), is found to match the near-side correlation in magnitude, shape (in Delta eta and Delta phi) and Sigma E-T(Pb) dependence. The resultant Delta phi correlation is approximately symmetric about pi/2, and is consistent with a dominant cos2 Delta phi modulation for all Sigma E-T(Pb) ranges and particle p(T).