We consider the problem of choosing optimal parameters in certain iterative procedures. Specifically, we are interested in finite-step processes for which it is possible to estimate the computational cost and the error relaxation in terms of the process parameters. The problem of finding the optimal process that provides the required error relaxation with a minimal total computational cost is defined and studied. To solve the problem, it is generally necessary to solve a series of mathematical programming problems with rapidly increasing dimension. We suggest two ways to avoid that difficulty. The first is to find a process that is close to the optimal process, by solving only one mathematical programming problem. The second is to define optimal processes in some special cases when this problem can be simplified. We define conditions under which processes with geometrically decreasing error are optimal or asymptotically optimal. The methods of finding parameters of such processes are also provided. We illustrate our ideas with two examples: the bilevel gradient method for unconstrained function minimization and the iterative process for solving an optimal design problem. ives Academic Press. Inc.