Dialogue on repeatability and random numbers

Return-Path: <efrank>
From: efrank (Ed Frank)
Posted-Date: Wed, 8 Jan 1997 09:15:08 -0500
Received-Date: Wed, 8 Jan 1997 09:15:08 -0500
Message-Id: <199701081415.JAA15082@upenn5.hep.upenn.edu>
Subject: Re: Some Recent Developments in Svt Software
To: GRLynch@lbl.gov
Date: Wed, 8 Jan 97 9:15:07 EST
Cc: wenaus, jfkral@lbl.gov (J.F. Kral)
In-Reply-To: <no.id>; from "GerryLynch" at Jan 7, 97 5:48 pm
X-Mailer: ELM [version 2.3 PL11]

Gerry,

This note is in regard to your recent article, included below, about
random numbers in SVT code.  I want to point out that the problem you
point out goes beyond SVT: it is now unavoidable that the last phases
of event simulation take place in the framework because that is where
the trigger is simulated.  The L1 accept time is needed for correct
simulation of digitizations.  Also, if one is to include beam
backgrounds, then digitization must occur after mixing in the
backgrounds, and this too is done in the framework.  My point here is
that the event mixer and FCS clock simulation both consume random
numbers as will the digitization codes for other subsystems, all of
which will be running at once (in the same job).  We need a way for
all of this to coexist and be repeatable.

Right now there is (and has been) a module called trgRandom that is
used for controlling random numbers in the framework when event mixing
is being done.  The trgRandom module uses the bbsim seeds to reset
grndm at the start of each event.  The FCS clock simulations depend
upon this.  I have always realized that this needs to be taken further
in two regards.  First, some day the random number engine should be
changed from grndm to whatever simulation chooses, which I think means
using the G4 engines.  Let's leave that aside for the moment, except
to note that, preferably, the solution we chose today for providing
randoms in the framework will hide this inevitable change.  Second, a
means is needed for making simulations of independent subsystems
repeatable, even if some are turned off or on during a rerun.  I am
afraid that your work on randoms in SVT will either be shortlived, or
will not work at all, because of competition for use of grndm from
other places, e.g., FCS clocks (which you must use to do digis.  Fabrizio
has already started using it).

I have thought a little bit about this, but really not fully.  I meant
to post a note about this long ago but have been working on other
things, and now regret it.  Do the issues I raise above seem real or
pressing to you?  If so, maybe I should post this short reply to
the discussion list.

                -Ed

> Date: Tue, 7 Jan 97 17:48:48 -0500
> From: GRLynch@lbl.gov (GerryLynch)
> Subject:  Some Recent Developments in Svt Software
>   *** Discussion title:  Silicon Vertex Tracker software
[...]
>         We, expecially Bill Lockman, are working actively to fix this
> non-repeatability.  The random number seeds that are used in bbsim are
> being put into the event header and are accessible to us.  We can use
> these to initialize the random generator grndm that we use to get our
random
> numbers.  Unfortunately SvtSim does not always use grndm.  It uses
Cernlib
> routines norran, poissn and funran that do not use grndm.  The
conversion
> from poissn and norran to routines that use grndm is straight-forward.
The
> generation of the tails of a Gaussian distribution that is now done with
> funran can be done a different way using the Cernlib inverse error
function
> routine gausin.  We hope to have this implemented soon.
> 
>         In the investigation of where random numbers are used in SvtSim,
I
> discovered that for some time we have not been adding electronic noise
to
> our signals.  We do add isolated electronic noise digitizations, but
have
> not been adding the noise to real signals.  This has been fixed in the
> version of ss_digi_driver that is at the head of cvs.  When this bug was
> fixed, the resolutions at small angles got worse by about 10% and the
> resolutions at large angles by more than that.
> 

-
-------------------------------------------------------------------------------
Ed Frank                                             Department of Physics     
Office:  (617)496-2362                               University of
Pennsylvania
efrank@upenn5.hep.upenn.edu                          Philadelphia, PA  USA

> Return-Path: efrank@upenn5.hep.upenn.edu
> Received: from upenn5.hep.upenn.edu (UPENN5.HEP.UPENN.EDU
[130.91.48.31]) by noc4.dccs.upenn.edu (8.8.4/8.7.3) with ESMTP id
NAA21046; Wed, 8 Jan 1997 13:25:34 -0500
> Received: by upenn5.hep.upenn.edu
>       id NAA18527; Wed, 8 Jan 1997 13:25:33 -0500
> From: efrank@upenn5.hep.upenn.edu (Ed Frank)
> Posted-Date: Wed, 8 Jan 1997 13:25:33 -0500
> Message-Id: <199701081825.NAA18527@upenn5.hep.upenn.edu>
> Subject: Re: Some Recent Developments in Svt Software
> To: GRLynch@LBL.GOV
> Date: Wed, 8 Jan 97 13:25:33 EST
> Cc: lockman@slac.stanford.edu, bob_jacobsen@LBL.GOV,
wenaus@llnl.Lbl.Gov,
>         jfkral@LBL.GOV
> In-Reply-To: <970108101658.2bc04792@csa6.lbl.gov>; from
"grl@Csa.LBL.Gov" at Jan 8, 97 10:16 am
> X-Mailer: ELM [version 2.3 PL11]
> 
> 
> >   So Bill has borrowed the guts of trgRandom
> > and put it into SvtMakeDigis.  If we were to pursue this model, then
> > DchSim should do its own initialization in a similar way.  Likewise
the
> > trigger module that uses random numbers should do its own
initialization.
> 
> Would you please explain this further?  I don't see how each of these
> modules would determine a starting seed.  My trick in trgRandom of
> using the seed from bbsim seems to give just one sequence unless
> one does some slight of hand.

        -Ed

-
-------------------------------------------------------------------------------
Ed Frank                                             Department of Physics     
Office:  (617)496-2362                               University of
Pennsylvania
efrank@upenn5.hep.upenn.edu                          Philadelphia, PA  USA

Return-Path: <grl@csa2.lbl.gov>
From: grl@csa2.lbl.gov
Date:    Wed, 8 Jan 97 10:38:54 PST
Subject: copy of letter sent to Ed Frank
To: wenaus@llnl.gov
Reply-To: GRLynch@LBL.GOV
X-St-Vmsmail-To: ST%"Wenaus@LLNL.gov" 

Hi Ed,

        Thanks for your letter on random numbers.  

        I was aware of your module trgRandom.  I tried it out and it
worked
as advertized.  We decided not to use it because it did not give us
independence from other packages: the answers would depend on the order in
which DchSim and SvtSim is done.  So Bill has borrowed the guts of
trgRandom
and put it into SvtMakeDigis.  If we were to pursue this model, then
DchSim should do its own initialization in a similar way.  Likewise the
trigger module that uses random numbers should do its own initialization.

        Bill has worked fast.  He has now put at the head of cvs a version
of SvtSim that uses grndm everywhere and is indeed repeatable. This has
already allowed me to learn more about one bug than I knew before.

        I had not thought of the complication that FCS clocks uses random
numbers, but I see no fundamental problem here.

        I would suggest that we adopt this way of proceeding for all
modules
in the analysis framework that use random numbers.  Its a separate
question
which random number generator to use.   We chose grndm because 1) its a 
fine random number generator, 2) we use it already in many places, and 
3) you used it in trgRandom.  If Bob and Torre decide that we need to use
a 
different generator in the future, so be it.  

                                                                Gerry

From: grl@csa2.lbl.gov
Date:    Wed, 8 Jan 97 10:42:53 PST
Subject: trying to clarify the random number proposal
To: efrank@upenn5.hep.upenn.edu, lockman@slac.stanford.edu,
        bob_jacobsen@lbl.gov, wenaus@llnl.gov, jfkral@lbl.gov
Reply-To: GRLynch@lbl.gov
ST%"bob_jacobsen@lbl.gov",ST%"wenaus@llnl.gov", ST%"jfkral@lbl.gov"

Ed,

        The method that I am proposing is that at the event entry point
of each module that uses random numbers one initializes grndm with seeds
that are determined from the seeds that are in the event header.  The
procedure is the same as as you do in trkRandom, but each module has its
own unique sequence.  I expect that it would not be noticeable if each 
module has the same sequences for a particular event, but it's simple to
modify this so that each module has a different sequence, but still
determined
from the same numbers.
                                                        Gerry

Return-Path: <efrank@upenn5.hep.upenn.edu>
From: efrank@upenn5.hep.upenn.edu (Ed Frank)
Posted-Date: Wed, 8 Jan 1997 14:03:30 -0500
Subject: Re: trying to clarify the random number proposal
To: GRLynch@LBL.GOV
Date: Wed, 8 Jan 97 14:03:29 EST
Cc: lockman@slac.stanford.edu, bob_jacobsen@LBL.GOV, wenaus@llnl.gov,
        jfkral@LBL.GOV
In-Reply-To: <970108104253.2bc04792@csa6.lbl.gov>; from
"grl@Csa.LBL.Gov" at Jan 8, 97 10:42 am
X-Mailer: ELM [version 2.3 PL11]

>       The method that I am proposing is that at the event entry point
> of each module that uses random numbers one initializes grndm with seeds
> that are determined from the seeds that are in the event header.  The
> procedure is the same as as you do in trkRandom, but each module has its
> own unique sequence.

This sounds fine, but isn't there considerable difficulty underlying
the "initializes grndm with seeds that are determined from the seeds in
the event header?"  Would you please sketch what you have in mind?

Selecting seeds is actually tricky when you need to guarantee that
different sequences are unrelated.  While there are games like
playing with the system clock time, these games are not provably
correct.

The single seed in the event header is actually quite special.  The
simulation production is designed so as to guarantee unique start
seeds for every event.  But that is all you get- one seed.  There is
also no guarantee about how many randoms you can generate from that
seed before overlapping the sequence used for the next event.  I think
this is an oversight in the simulation production design that could be
fixed.

Here is something that is (perhaps) provably correct.  For the
L'ecuyer generator (grndm) one can produce generators that give
interleaved sequences.  Basically, one forms a family of N generators.
They all start from the same seed, but they only take every N'th
element in the sequence.  It is like a braided rope.  This introduces
two new problems.  First, the effective sequence length is shortened
by a factor of N.  Second, I think (but am not sure) that one worsens
the sequences statistical properties.

> I expect that it would not be noticeable if each 
> module has the same sequences for a particular event,

I'm not sure about this.  I suspect you are right, but would rather
avoid the issue if we can.

        -Ed

Return-Path: <LOCKMAN@SLAC.Stanford.EDU>
From: LOCKMAN@SLAC.Stanford.EDU
Date: Wed, 08 Jan 1997 14:02 -0800 (PST)
Subject: Re: trying to clarify the random number proposal
To: efrank@upenn5.hep.upenn.edu, grl@lbl.gov
Cc: wenaus@llnl.gov
X-Envelope-To: wenaus@llnl.gov
Content-Transfer-Encoding: 7BIT

In-Reply-To: efrank@upenn5.hep.upenn.edu -- 01/08/97 11:02

There are several requirements that we are trying to satisfy.
Some of them are

1.   reproducibility
This means that in each package using random numbers, the same sequence
of random numbers is generated in a given event on a given platform.
Gerry checked that this is satisfied for SvtSim by processing a given
event twice and comparing the digi output. This requires that the same
underlying generator be used everywhere in a package.

2.   independence
This means that the random numbers for different packages are handled
independently of each other. One way to implement this is to require
that the SVT calls SvtGrndm to generate its random numbers where each
call to SvtGrndm sets JSEQ(1) to some agreed upon value, then calls
grndm. DCH calls DchGrndm and sets JSEQ(1) to a different value, etc.

3.   randomness
Currently, SVT's random numbers are seeded from values stored in the
header. If each package uses the same seeds for its version of grndm,
then some correlations could creep in. One way around this would be to
seed a long period generator like CERN's ranlux generator with the
header seed and from this generate a new set of seeds for SvtGrndm,
DchRndm, etc.

Im sure Ive overlooked some requirements and there may be better ways to
implement those listed above. Perhaps the first step is to come up with
a formal list of requirements and then discuss how they might be
implemented.

-
--------------------------------------------------------------------------
Bill Lockman (UCSC)
email: lockman@slac.stanford.edu voice: (415) 926-3435 fax: (408) 459-5777

Date: Fri, 14 Feb 1997 01:50:58 -0800 (PST)
From: lockman@SLAC.Stanford.EDU (Bill Lockman)
Subject: Random numbers and reproducibility in SvtSim
In-Reply-To: <"/recoCommon"@babar-hn.slac.stanford.edu>
To: recoCommon-hn@MAILBOX.SLAC.Stanford.EDU
Errors-To: Owner-HyperNews@SLAC.Stanford.EDU
Newsgroups:
http://babar-hn.slac.stanford.edu:5090/HyperNews/get/recoCommon.html

*** Discussion title: Design of the Common Reconstruction Software
Email replies to recoCommon-hn must include:
  In-Reply-To: <"/recoCommon/136"@babar-hn.slac.stanford.edu>
  Subject: ...change this to be about your reply.

SvtSim is the package which simulates signals in the SVT wafers due to
charged particles and the subsequent electronics response. In the
reconstruction framework, the SvtGHits are read in from an xdr file.
SvtSim uses these to produce the electronics response (SvtDigis). Since
we simulate stochastic process in both the silicon wafers and readout
electronics, we need to sample random numbers from many different
distributions. One requirement imposed by the need to debug code in the
framework is reproducibility: a set of digis produced from a particular
set of hits in an event should always be the same, regardless of what
transpired in previous events.

This requirement implies that the sequence of random numbers used in a
particular event must always be the same. To achieve this, we first
modified all of our nonuniform random number generators to use the same
base generator. The base generator was chosen to be grndm, the Geant3.21
generator. The nonuniform generators presently in use are: poisson
(gpoiss), Landau (inline code) and gaussian above a cutoff (inline code
using erfc and grndm).

We then read the 2 32-bit integer seeds from the event header and used
these to seed grndm (sequence 1). The sequence is reseeded for each
event. The code to access and seed grndm came from Ed Frank. It is in
SvtSim/SvtMakeDigis.cc:

AbsEvent*
SvtMakeDigis::event( AbsEvent* anEvent, int )
{
// Section to get random number seeds from the event header
// Stolen from Ed Frank's trgFrame/trgRandom.cc
//  cout getGEventAList()->first();
  if ( NULL == header ) {
    cout nrndm1();
  int seed2 = header->nrndm2();
  int iseq = 1;

  grndmq_( seed1, seed2, iseq, "S", 1);
                   .
                   .
                   .

This could be turned into a function callable from any package. One
might improve on the present scheme of using just one random number seed
pair to seed many packages. This could be done by using the header seeds
to seed a completely different base generator which produces a new set
of seeds for grndm, one seed pair per package. One candidate for this
generator might be the CERN package, RANLUX. Another improvement would
be to have each package own its own copy of grndm.