We calculate the intensity and photon spectrum of the intergalactic background light (IBL) as a function of redshift using an approach based on observational data obtained in many different wavelength bands from local to deep galaxy surveys. Our empirical approach allows us to obtain a completely model independent determination of the IBL and to quantify its observationally based uncertainties. Using our results on the IBL, we then place 68% confidence upper and lower limits on the opacity of the universe to $gamma$-rays, free of the theoretical assumptions that were needed for past calculations. We compare our results with measurements of the extragalactic background light and upper limits obtained from observations made by the {it Fermi} ~Gamma-ray Space Telescope.