In the free energy landscape picture of glassy systems, the slow dynamics characteristic of these systems is believed to be due to the existence of a complicated free-energy landscape with many local minima. We show here using light scattering that for a colloidal glassy material multiple paths can be taken through the free energy landscape, that can even lead to different 'final' non-ergodic states at the late stages of aging. We provide clear experimental evidence for the distinction of gel and glassy states in the system and show that for a range of colloid concentrations, the transition to non-ergodicity can occur in either direction (gel or glass), and may be accompanied by 'hesitations' between the two directions. This shows that colloidal gels and glasses are merely global free-energy minima in the same free energy landscape, and that the paths leading to these minima can indeed be complicated.