The two-dimensional (2D) and three-dimensional (3D) orthogonal moments are useful tools for 2D and 3D object recognition and image analysis. However, the problem of computation of orthogonal moments has not been well solved because there exist few algorithms that can efficiently reduce the computational complexity. As is well known, the calculation of 2D and 3D orthogonal moments by a straightforward method requires a large number of additions and multiplications. In this paper, an efficient algorithm for computing 2D and 3D Legendre moments is presented. First, a new approach is developed for computing Legendre polynomials with one variable; the corresponding results are then used to calculate 1D Legendre moments. Second, we extend our method to calculating 2D Legendre moments, a more accurate approximation formula when an analog original image is digitized to its discrete form is also discussed, and the relationship between the usual approximation and the new approach is investigated. Finally, an efficient method for computing 3D Legendre moments is developed. As one can see, the proposed algorithm improves the computational efficiency significantly and can be implemented easily for high order of moments.