Caterpillar crawling is distinct from that of other limbless animals. It is simple but efficient. This paper presents a novel mechanism to duplicate the movement to a modular caterpillar-like robot. First, how caterpillars move in nature is investigated and analyzed systematically. Two key locomotive properties are abstracted from the body shape of caterpillars during crawling. Then, based on a morphological mapping, a hypothesis of asymmetric oscillation with a ratio of two is proposed, followed by a thorough analysis of the kinematics of the caterpillar-like robot. The asymmetric oscillating mechanism is proved capable of generating stable caterpillar-like locomotion. Next, taking advantage of the two locomotive properties and the hypothesis, a new Central Pattern Generator (CPG) model is designed as the controller of the robot. The model can not only generate the signal as expected, but also provide explicit control parameters for online modulation. Finally, simulation and on-site experiments are carried out. The results confirm that the proposed method is effective for caterpillar-like locomotion.