This paper expounds the working principle of pharmacy robot vision system combining with the mechanical structure of the pharmacy robot. Firstly, the visual system takes the digital images of ampoules and penicillin bottles by the CCD camera. Secondly, we use the functions of OpenCV to extract the ROI of bottle image, filter, segment, and extract the characteristic parameters such as diameter, height, bottleneck position based on the method of pixel scanning. Finally, we can identify the kinds of the bottle through the trained Naive Bayesian Classifier. According to the diameter, height, bottleneck location and the kind of the bottle, the visual system can correctly guide pharmacy robot to complete tasks successfully.