Latest advances in hardware technology and state of the art of computer vision and artiΓΏcial intelligence research can be employed to develop autonomous and distributed monitoring systems. The paper proposes a multi-agent architecture for the understanding of scene dynamics merging the information streamed by multiple cameras. A typical application would be the monitoring of a secure site, or any visual surveillance application deploying a network of cameras. Modular software (the agents) within such architecture controls the di erent components of the system and incrementally builds a model of the scene by merging the information gathered over extended periods of time. The role of distributed artiΓΏcial intelligence composed of separate and autonomous modules is justiΓΏed by the need for scalable designs capable of co-operating to infer an optimal interpretation of the scene. Decentralizing intelligence means creating more robust and reliable sources of interpretation, but also allows easy maintenance and updating of the system. Results are presented to support the choice of a distributed architecture, and to prove that scene interpretation can be incrementally and e ciently built by modular software.