We studied the occurrence of phase separation and atomic ordering in quaternary In x Al y Ga 1ÀxÀy N layers by transmission electron microscopy. Three layers of different compositions were examined: one lattice-mismatched In 0.10 Al 0.02 Ga 0.88 N layer together with two lattice-matched In 0.12 Al 0.29 Ga 0.59 N and In 0.06 Al 0.18 Ga 0.76 N layers. The composition modulations were seen in all the layers. The wavelengths (k) of composition modulations resulting from phase separation were calculated using selected-area electron diffraction patterns. The smaller k (9 nm) in In 0.10 Al 0.02 Ga 0.88 N layer in comparison to that in In 0.12 Ga 0.88 N layer (k = 20 nm) suggests that the driving force for phase separation in In x Al y Ga 1ÀxÀy N layers is greater than that in In x Ga 1Àx N layers with similar In contents. Energy dispersive spectroscopy line profiles across In x Al y Ga 1ÀxÀy N/GaN interfaces revealed a gradual increase in Al and In incorporation. Additional (0 0 0 1) diffraction spots in an SADP taken on an h1 1 0 0i zone from the In 0.06 Al 0.18 Ga 0.76 N layer suggests that atomic ordering can occur in quaternary layers. Arguments are developed to rationalize these experimental observations.