Efficient protocol for clustering and backbone formation is one of the most important issues in wireless ad hoc networks. Connected dominating set (CDS) formation is a promising approach for constructing virtual backbone. However, finding the minimum CDS in an arbitrary graph is a NP-Hard problem. In this paper, we present a novel zone-based distributed algorithm for CDS formation in wireless ad hoc networks. In this Zone algorithm, we combine the zone and level concepts to sparsify the CDS constructed by previous well-known approaches. Therefore, this proposed algorithm can significantly reduce the CDS size. Particularly, we partition the wireless network into different zones, construct a dominating tree for each zone and connect adjacent zones by inserting additional connectors into the final CDS (at the zone borders). Our comprehensive simulation study using a custom simulator shows that this zone-based algorithm is more effective than previous approaches. The number of nodes in the CDS formed by this Zone algorithm is up to around 66% less than that constructed by others. Moreover, we also compare the performance of Zone algorithm with some recently proposed CDS formation protocols in ns2 simulator.