Display pixel caching fills empty borders that occur when videos are displayed on a screen with mismatching aspect ratios and/or resolutions with spatially and temporally consistent information while preserving the original video format. It merges the motion information from subsequent frames to generate high-resolution panoramas in a consistent manner.