A Hopf algebra is a pair (A, 2) where A is an associative algebra with identity and 2 a homomorphism form A to A A satisfying certain conditions. If we drop the assumption that A has an identity and if we allow 2 to have values in the socalled multiplier algebra M(A A), we get a natural extension of the notion of a Hopf algebra. We call this a multiplier Hopf algebra. The motivating example is the algebra of complex functions with finite support on a group with the comultiplication defined as dual to the product in the group. Also for these multiplier Hopf algebras, there is a natural notion of left and right invariance for linear functionals (called integrals in Hopf algebra theory). We show that, if such invariant functionals exist, they are unique (up to a scalar) and faithful. For a regular multiplier Hopf algebra (A, 2) (i.e., with invertible antipode) with invariant functionals, we construct, in a canonical way, the dual (A , 2 ). It is again a regular multiplier Hopf algebra with invariant functionals. It is also shown that the dual of (A , 2 ) is canonically isomorphic with the original multiplier Hopf algebra (A, 2). It is possible to generalize many aspects of abstract harmonic analysis here. One can define the Fourier transform; one can prove Plancherel's formula. Because any finite-dimensional Hopf algebra is a regular multiplier Hopf algebra and has invariant functionals, our duality theorem applies to all finite-dimensional Hopf algebras. Then it coincides with the usual duality for such Hopf algebras. But our category of multiplier Hopf algebras also includes, in a certain way, the discrete (quantum) groups and the compact (quantum) groups. Our duality includes the duality between discrete quantum groups and compact quantum groups. In particular, it includes the duality between compact abelian groups and discrete abelian groups. One of the nice features of our theory is that we have an extension of this duality to the non-abelian case, but within one category. This is shown in the last section of our paper where we introduce the algebras of compact type and the algebras of discrete type. We prove that also these are dual to each other. We treat an example that is sufficiently general to illustrate most of the different features of our theory. It is also possible to construct the quantum double of Drinfel'd within this category. This provides a still wider class of examples. So, we obtain many more than just the compact and discrete quantum within this setting.