Refractory objects such as Calcium, Aluminum-rich Inclusions, Amoeboid Olivine Aggregates, and crystalline silicates, are found in primitive bodies throughout our Solar System. It is believed that these objects formed in the hot, inner solar nebula and were redistributed during the mass and angular momentum transport that took place during its early evolution. The ages of these objects thus offer possible clues about the timing and duration of this transport. Here we study how the dynamics of these refractory objects in the evolving solar nebula affected the age distribution of the grains that were available to be incorporated into planetesimals throughout the Solar System. It is found that while the high temperatures and conditions needed to form these refractory objects may have persisted for millions of years, it is those objects that formed in the first 10 5 years that dominate (make up over 90%) those that survive throughout most of the nebula. This is due to two effects: (1) the largest numbers of refractory grains are formed at this time period, as the disk is rapidly drained of mass during subsequent evolution and (2) the initially rapid spreading of the disk due to angular momentum transport helps preserve this early generation of grains as opposed to later generations. This implies that most refractory objects found in meteorites and comets formed in the first 10 5 years after the nebula formed. As these objects contained live 26 Al, this constrains the time when short-lived radionuclides were introduced to the Solar System to no later than 10 5 years after the nebula formed. Further, this implies that the t ΒΌ 0 as defined by meteoritic materials represents at most, the instant when the solar nebula finished accreting significant amounts of materials from its parent molecular cloud.