This paper introduces the design and implementation of an augmented reality-based guidance system used for museum guidance. The objective of this study is to deliver the new museum guidance with intuitive and userfriendly interactions between man and machine based on computer vision technique in ubiquitous computing environment. In our design, a camera is used to capture hand motions and images of a printed guide, which are analyzed instantaneously to parse the designated actions the guidance system promptly responds with. The system can display and illustrate contextual information about 3D models of museum artifacts and multimedia materials to users in real time to compliment the museum experience. Moreover, the performance of the approach we proposed herein has been compared with that of the other alternatives with which the guidance system has also been installed for similar exhibitions. As a result, our proposed approach outperforms the other applications. The advantage of the proposed approach includes the exclusion of common tactile peripheral devices, for example, keyboard and mouse, which accordingly reduces hardware-related costs and reduces the risk of contamination in a public environment.