MW campaign on Mrk 501 (March-May 2008):
Definition of format for data sharing
:
The reduced data from the different instruments will be available to all people participating in the campaign, prior to acceptance of the following policy specified here .
The analyzed data corresponding to each instrument will be produced and validated by the people working on that instrument. The provided reduced data are correct up to the best knowledge of the analyzers (collaborations).
In order to allow an efficient and easy use of such data sets, we request the analyzers to use a specific format for their data products. Such data format is specified below.
The reduced data will be provided in ascii format. Future MW campaigns might allow for root and fits files. But for the time being, we'll be keeping things simple.
Two different ascii files will be provided for flux and spectra measurements.
All fluxes (integrated in a specific energy range) will be provided in an single ascii file (e.g. InstrumentXX_Fluxes.txt ).
All spectra (over a specific energy range) will be provided in a single ascii file (e.g. InstrumentXX_Spectra.txt ).
In case different analysis were performed (on the data from a given instrument), then additional ascii files could be provided (e.g. InstrumentXX_Fluxes_analysis01.txt, InstrumentXX_Fluxes_analysis02.txt or InstrumentXX_Fluxes_EnergyRange01.txt, InstrumentXX_Fluxes_EnergyRange02.txt ). In such case, we suggest providing a README.txt file with short descriptions of the different analysese that were used.
For the sake of clarity, the information (fluxes/spectra and additional material like README files...) from the different instruments will be stored separately in different directories
The file should have a header with general information. This information is meant to be for general purposes. No specific format is required since it is not meant to be automatically parsed. We suggest that, at least, the following information appears in such header:
Instrument: name and web page.
Analyzers : specify name and contact e-mail addresses.
Date of ascii file production: This will be useful to keep track of potential different analyses
General notes: Write here anything you think it might be useful to understand/play with the data, as well as for combining the data with that of different instruments working at similar frequencies. For instance, in the case of a radio instrument one could write here the specific instrument (e.g. bolometer/heterodyne receiver etc.) and observing mode (cross-scans,on-off etc.).
The fluxes related to the different time bins (which could be weeks, day, hours...) will be specified independently. The different flux points will be preceded by the flag START_FLUX_REPORT. After writing all the flux points, the flag STOP_FLUX_REPORT should be used in the next line. Each single flux point should be specified following the format below:
The fields ending with a star are required (like MJD_START,MJD_END or FLUX). The other fields are optional, and might well depend on the type of instrument/energy range being covered (like UTC_date_START or FLUX_HostGalaxy).
The MJD_START and MJD_END time should be specified with at least 3 decimals.
UTC_date_START : (YYYYMMDD)
MJD_START* :
UTC_time_START : (HHMMSS at start of exposure)
UTC_date_END : (YYYYMMDD)
MJD_END* :
UTC_time_END : (HHMMSS at end of exposure)
Duration* : This is the net time observation (in seconds). Periods of time where instrument did not observe should be removed.
Mean_frequency : Mean frequency (energy) of the time bin (In Hz)
Lowest_frequency*: Min frequency of the time bin (In Hz)
Highest_frequency*:Max frequency of the time bin (In Hz)
FILTER : (H,J,K,B,V...etc); only required in case we deal with an optical instrument.
FLUX_UNITS* : (see below)
FLUX* :
FLUX_ERROR* :
FLUX_HostGalaxy : only applicable in case we deal with an optical instrument
FLUX_HostGalaxy_ERROR : only applicable in case we deal with an optical instrument
ANALSYS_FLAG* : (P=preliminary, F=final), this will be useful for filtering results
QUALITY_FLAG* : (B=bad, M=medium, G=good), this will be useful for filtering results
CALIBRATION : Specify here the observed/used calibrator sources and flux density scale (radio) or reference stars etc. (optical)
NOTES : Use this field to describe problems/issues during the observation night, as well as to describe holes/breaks in the observation carried out between the START time and STOP time (see remark 2). Leave blank if there were no problems and no breaks in the data acquisition.
Because of the broad energy coverage and instrument characteristics, several flux units are possible: Jy (for radio), mag (for optical... or should we use mJy ??), ph/cm2/s or erg/cm2/s (for X-rays and Gamma-rays).
Remark 1: The sign ":" will be used to separate the identifier (UTC date, UTC time...) from their value (20080501, 023000 ...). A single character string will be used at each side of the ":" sign. The text should not contain carriege returns in the middle; it should be a single line.
Remark 2: END-START >= Duration. The time window in which the flux point is derived, which is defined by UTC date/time START/END, could have holes; that is periods of time where the instrument was not observing. This could be the case in a flux point derived with an Cherenkov Telescope after integration of N days. The telescope is certainly not observing constantly during the N days, and consequently the duration of the observation is going to be smaller than the time window where the observations were carried on.
The file should have a header with general information. This information is meant to be for general purposes. No specific format is required since it is not meant to be automatically parsed. We suggest that, at least, the following information appears in such header:
Instrument: name and web page.
Analyzers : specify name and contact e-mail addresses.
Date of ascii file production: This will be useful to keep track of potential different analyses
General notes: Write here anything you think it might be useful to understand/play with the data, as well as for combining the data with that of different instruments working at similar frequencies. For instance, in the case of a radio instrument one could write here the specific instrument (e.g. bolometer/heterodyne receiver etc.) and observing mode (cross-scans,on-off etc.).
The spectra related to the different time bins (which could be weeks, day, hours...) will be specified independently. The different spectra will be preceded by the flag START_SPECTRA_REPORT. After writing all the spectra, the flag STOP_SPECTRA_REPORT should be used in the next line. Each single spectra should be specified following the format below:
The fields ending with a star are required (like MJD_START,MJD_END). The other fields are optional, and might well depend on the type of instrument/energy range being covered (like UTC_date_START).
The MJD_START and MJD_END time should be specified with at least 3 decimals.
UTC_date_START : (YYYYMMDD)
MJD_START* :
UTC_time_START : (HHMMSS at start of exposure)
UTC_date_END : (YYYYMMDD)
MJD_END* :
UTC_time_END : (HHMMSS at end of exposure)
Duration* : This is the net time observation (in seconds). Periods of time where instrument did not observe should be removed.
Table_START* : below there will be a table with all the energy bins FreqLow[Hz] FreqUp [Hz] E2*dF/dE [erg cm-2 s-1] Error in E2*dF/dE [erg cm-2 s-1]
Freq1_low Freq1_up EnergyDensity1 EnergyDensity1_Error
Freq2_low Freq2_up EnergyDensity2 EnergyDensity2_Error
...
FreqN_low FreqN_up EnergyDensityN EnergyDensityN_Error
Table_END* :
ANALSYS_FLAG* : (P=preliminary, F=final), this will be useful for filtering results
QUALITY_FLAG* : (B=bad, M=medium, G=good), this will be useful for filtering results
CALIBRATION : Specify here the observed/used calibrator sources and flux density scale (radio) or reference stars etc. (optical)
NOTES : Use this field to describe problems/issues during the observation night, as well as to describe holes/breaks in the observation carried out between the START time and STOP time (see remark 2). Leave blank if there were no problems and no breaks in the data acquisition.
Remark 1: The sign ":" will be used to separate the identifier (UTC date, UTC time...) from their value (20080501, 023000 ...). A single character string will be used at each side of the ":" sign. The text should not contain carriege returns in the middle; it should be a single line.
Remark 2: END-START >= Duration. The time window in which the flux point is derived, which is defined by UTC date/time START/END, could have holes; that is periods of time where the instrument was not observing. This could be the case in a flux point derived with an Cherenkov Telescope after integration of N days. The telescope is certainly not observing constantly during the N days, and consequently the duration of the observation is going to be smaller than the time window where the observations were carried on.
Remark 3: We recommend that, within one ascii file, the definition of energy bin boundaries is the same for all the spectra. This allows for a better comparison, and even combination, of the data points. Note however that different spectra (for different time periods) could have a different number of energy bins (spectra extended/reduced on the left/right side).
Additionally, the spectra could be fitted with a given function. The resulting fit parameters and goodness of the fit could also be reported. The following fields are thus optional. They are not meant to be parsed automatically.
Fit_function : formula
Fit_Range : Freq_low, Freq_high
Goodness_of_fit : Chi2/NDF (and/or Probability of getting higher Chi2)
FitResults : Parameter1 = val +/- error, Parameter2 = val +/- error ...
Instead of embedding the fit results in the file with the spectral points, additional files with the fit results can be provided. In this way, additional information can be better described (way of fitting, covariance matrix...).