Centaurus B is a nearby radio galaxy positioned in the Southern hemisphere close to the Galactic plane. Here we present a detailed analysis of about 43 months of accumulated Fermi-LAT data and of newly acquired Suzaku X-ray data for the source. Centaurus B is detected at GeV photon energies, appearing as a steady $gamma$-ray emitter. The X-ray core of Centaurus B is detected as a bright source of continuum radiation. We do not detect however any diffuse X-ray emission from the known radio lobes, with the provided upper limit only marginally consistent with the previously claimed ASCA detection.

Two scenarios that connect the X-ray and $gamma$-ray properties are considered. In the first one, we assume that the diffuse non-thermal X-ray emission component is not significantly below the derived Suzaku upper limit. In this case, modeling the inverse Compton emission shows that the observed $gamma$-ray flux of the source may in principle be produced within the lobes. This association would imply that efficient in-situ acceleration of the radiating electrons is occurring and that the lobes are dominated by the pressure from the relativistic particles.

In the second scenario, with the diffuse X-ray emission well below the Suzaku upper limits, the lobes in the system are instead dominated by the magnetic pressure. In this case, the observed $gamma$-ray flux is not likely to be produced within the lobes, but instead within the nuclear parts of the jet. By means of synchrotron self-Compton modeling we show that this possibility could be consistent with the broad-band data collected for the unresolved core of Centaurus B, including the newly derived Suzaku spectrum.