We study diffuse gamma-ray emission at intermediate Galactic latitudes measured by the Fermi Large Area Telescope with the aim of searching for a signal from dark matter annihilation or decay. In the absence of a robust dark matter signal, constraints are presented. We set both, conservative dark matter limits requiring that the dark matter signal does not exceed the observed diffuse gamma-ray emission and limits derived based on modeling the foreground astrophysical diffuse emission. Uncertainties in several parameters which characterize conventional astrophysical emission are taken into account using a profile likelihood formalism. The resulting limits impact the range of particle masses over which dark matter thermal production in the early Universe is possible, and challenge the interpretation of the PAMELA/Fermi-LAT cosmic ray anomalies as annihilation of dark matter.