Maintaining a rich and coherent visual experience is a fundamental challenge to human visual system. Multiple physiological constraints limit the amount of visual input, and constant eye and head movements change the retinal projection every moment. Yet, the human mind effortlessly constructs a seamless visual world. This dissertation reports evidence showing how our brain is elegantly designed to overcome the limits of the visual system and achieve a continuous and integrated visual percept of the world. In Chapter 2 (Park, Intraub, Widders, Yi & Chun, 2007), I report functional magnetic resonance imaging (fMRI) evidence showing the extrapolation in the parahippocampal place area (PPA) and retrosplenial cortex (RSC) of scene layout information beyond what was physically presented. Using an illusion called boundary extension, results demonstrated that scene layout representation in the PPA and RSC are extrapolated beyond the confines of the perceptual input, as an adaptive effort to overcome the aperture-like visual constraint. Such extrapolation observed in Chapter 2 may provide an anticipatory representation of upcoming layout that may be fundamental to the integration of successive views. In Chapter 3 (Park & Chun, submitted), I tested how different snapshots of successive views are represented as an integrated scene in the brain. Maintaining a continuous visual experience across multiple views require computing specific differences across views as well as integrating similarities across views. I hypothesized that these distinct computations may occur differently in the PPA and RSC. Participants saw snapshots that represented different views, but appeared to come from the same panoramic scene. Results revealed that the PPA focuses on selective discrimination of different views while RSC focuses on the integration of scenes under the same visual context. This complementary functional architecture of the PPA and RSC may provide a mechanism for the brain to achieve visual integration over fragmented visual inputs. The ability to extrapolate beyond and across views may reflect the ability to imagine spatial layouts or mentally generate a cognitive counterpart of a novel map. Chapter 4 tested the nature of mentally generated scene representations in the PPA and RSC. Instead of actually viewing the scene, participants were asked to think back, that is, to refresh a scene that they just saw. Similarly to results in Chapter 3, the PPA showed view-specific representations and RSC showed view-invariant representations when participants mentally refreshed scenes. Furthermore, the act of refreshing facilitated the scene integration by foregrounding the information to facilitate the binding of the past and the current view. Altogether, this dissertation demonstrated that the brain achieves the visual stability through constantly extrapolating beyond the given physical input and representing multiple levels of scene information through complementary functions of the PPA and RSC.