Global Grid Forum 6 (GGF6) meeting

Held at Sheraton Hotel October 14-17,  2002

Rough Notes by Les Cottrell

Introduction - Charlie Catlett & Cees DeLaat

This is the 6th meeting of the GGF. It was focused on working groups/BOFs, i.e. working sessions as opposed to plenaries or general interest talks. There were about 400 attendees.  There was a "GGF chairs workshop" given by Cees DeLaat (based on the IETF training) attended by about 40 people. 

Lot of work is going on to come up with legalize. IBM recently agreed to make use of patents for grid free and open.

GGF mission: create specs & best practices; determine need for specs & CPs, create specs and pre-CPs (done by Research Groups); develop & manage architecture/frameworks; technology conduit between long-term research exploratory development, commercial application.

There is increased effort to improve the quality of sessions, this includes ensuring sessions are well run (e.g. chairs are trained), that it is clear what the purposes of the sessions are, that there are minutes, attendees list. There will be different types of sessions, e.g. document review sessions, regular WG sessions of 90 mins, there may be workshops on more research oriented topics.

Document process: GFD-C.1.pdf, governance at GGF-Governance-Apr2002.pdf

Role of WG: develop ideas, review proposals, jury, find consensus, produce specs, recommend specs to GGF via GFSG.

WG face-to-face meeting organization: announce 1 month before, publish agenda at the same time; all drafts must be sent 2 weeks before meeting to the drafts repository. For each meeting: get 2 note takers (secretary + one) notes will be edited and put up on the web site within 2 weeks, , circulate attendance lists (blue sheets), put agenda up for bashing, go through items, one chair keeps timing & process flow, other chair start out giving an overview of the drafts status.

Consensus must include the mailing list, does not have to be unanimous.  Humming and/or show of hands can be used to find how people think. Predefine agenda, prevent discussions going in circles.

Network Measurement Working Group  (NMWG)


Discuss characteristics of document: purpose, terminology, characteristics hierarchy, highlights of discussion
Interface discussion
Type of document
What needs to be done before submitting document
Tomorrow NMWG/CIM/DAMED meeting 2pm in Sheraton Ohio room

NMWG is part of Performance & Information  Systems area, mailing list, web page

Need to classify measurements: what does it measure? Sometime more important than how. Not necessarily a new schema: should be a good schema for network measurements, not all systems are/should be organized this way. Can be used as annotation in any schema. Goal is an agreed-upon classification  of measurements taken, to allow measurement methodologies to classify their measurements to enable portability.

A measurement is represented by 2 elements: Characteristics (what is measured, network characteristic is an "intrinsic property", not an observation), a network entity. Then there is the measurement methodology, observation, statistics. There can be raw and derived (aggregate or estimate from other characteristics) measurements. Discussion on a top down approach where classify as to usefulness for applications.

Observations: singleton=smallest possible observation; sample=several singletons together; statistic=derived from sample by computing a statistic.

Topology, 2 types: physical & functional. 

The Node attributes may need to add others besides QoS & protocol stack.

Under characteristics will remove stability.

Available bandwidth issues include: TCP implementation, buffer tuning, host issues, slow-start, intrusiveness, application bursts. May need to divide available bandwidth into: achievable, available (=capacity-utilization). May not need available since it is derived.  However, available is really a property of the network, but achievable includes the end hosts. Might also include application. Will probably need an iteration as one tries to develop the schemas and also the meeting with the DAMED etc. groups.

Jitter is an attribute of one-way delay, round trip jitter is a derivation (convolution) of two one-way delays.

Under routes there was a comment that need to have routes at all layers (many people associate routes with layer 3 only, whereas have layer 2 (switch/CAM), layer 4 (application layer switching), layer 1 (lambda switching)). May need to remove route metrics.

Added Closeness/Cost as an attribute to Others.

We  need an interface specification.

Will have sidebar meetings, then come back on Thursday 8:00am.

Sidebar meetings

The NMWG authors met after lunch for 1.5 hours to go over input. Major issues were the attributes for the nodes (e.g. MTU, IPv4/IPv6, host bus, speed, NIC card type, OS + version,  encryption, TCP stack type, (Reno/Tahoe/SACK/HPTCP ...). We also decided to exclude much mention of multicast. In fig 2 the Network Path box was changed to Network Path with sub classes of Hop and  ... It was decided to change Route in fig 3 to Hop-list and have 2 boxes underneath it (sub classes) labeled: physical and virtual, but are these attributes of host). Also in fig 3 jitter is to connected via an open arrow and dashed link. In fig 3 host is an end host (i.e. not a multi-homed forwarding device, they are included as nodes).  Also in fig 3 we added achievable as a sub class of bandwidth. This also introduced the notion that in some cases available=capacity-utilized, though it can be measured separately and so needs its own name. For bandwidth it is necessary to specify the layer (attribute) at which it is measured.

The next steps are to make the updates and then post to trhe email list for 2 weeks, then if no comments send to Charlie Catlett as a GGF submission.

Discovery and Monitoring Event Descriptions (DAMED) Working Group

This was chaired by Dan Gunter of LBNL and Brian Tierney of LBNL. See

Experiences with the "Top N" event names.  They are storing the info in a MySQL archive, they use the NetLogger transport. Have applied to iperf, netest, ping. Have not done traceroute yet and it might be a challenge. Unique event names are necessary, "single valued" event are convenient. Want to be able to strip off prefixes from the name. Typically the network. was redundant, name was already long, prefix on its own is not useful. They did not include the tool name in the event. Netlogger (transport) provides that information, so may be an implementation issue. Need additional event parameter (e.g. period reporting from iperf) to report the "singleton" and period and possibly the type of aggregation. To search the archive need a standard way to represent targets, they provided SRC & DST host for all measurements (even CPU use, the report DST=SRC), also needed HOP_NUM for hop-by-hop measurements. If have a measurement series need to distinguish between one measurement in series and  ..

Tried to synchronize with NMWG names.

Conclusion was that event names worked well, and have defined a mapping to XML/SQL/NetLogger (LDAP not done) so have completed task, and the question is what comes next. A next step is to map more networking events, extend beyond networking. The DMTF group has created a CIM (Common Interface Management?) document for desktops etc. Needs a second (beyond LBNL's)  implementation to become a standard. This is only an informational document so a second implementation in not mandatory. SLAC might be interested in an implementation.

How well does it extend to other areas such as performance analysis, resource usage, for feedback to an application. Nobody was against submitting as a document.

DAMED-WG and CIM-WG and NM-WG Interactions

Q: Does CIM schema cover all of DAMED information.  Suggestion take information from NMWG and DAMED and push through CIM. CIM does not cover network events.  Could form 2 groups one for host events, one for application events and have them CIM-ize the events. The NMWG will flesh out some of their stuff, e.g. add the attributes.

DAMED group have finished. Its too early to do schemas for network measurements, CIM is not ready to start on this, the NMWG is

Functional Topologies - Augusto Ciuffoletti


Grid computation requires 3 resources: computing, storage, communication. Currently Grid concentrates on CE & SE. Communications Elements (NE) are more complex. NE requires where the measurement is made. First step is to introduce a specialized entry in charge of measuring NE: the Theodolite Entity.  The 2nd step consists in localizing communication in a couple of nodes in a graph (he binds himself only to point to point communications). Each node may contain a SE, CE & TE. The SE/CE/TE are then in domains, there can be more than one SE, CE or TE per domain. Must ensure bottlenecks are between TEs. It is assumed that internal communications (within a  domain) are better than external. The characteristic used to design the network should be static (e.g. capacity) or change only in exceptional cases (e.g. route changes). Internal paths should be measured but not published. Then uses an SQL syntax to discover information stored.

Updates to NMWG document & Next Steps

Measurements represented by 2 elements: characteristic and attributes. A network path can be either a host-to-host path or a hop. A hop can be a level2 or level 3 hop. Nodes and paths should have physical/functional attributes.

Replaced routes with forwarding to remove paler implication. May have multiple hoplists for a given path, may need a stability attribute. May also have related measurements such as hoplist being needed together with delay. May need to add more text to indicate the measurements do not end at the network but include the host.

Types of GGF document submissions possible: Informational "here is something useful", community  practice - many people in the community believe this is the right way, GGF recommendation - this is the way to do it. Recommendation will require an implementation. Recommendation can be experimental or standards track. Experimental documents stay experimental for ever then if a new document is written it can become a standard.

Possible recommendation: add a new milestone to charter and those in the DAMED group interested in developing the network schema would join the NMWG. There was considerable interest in this. The CIM seemed a bit heavy so might develop schemas in XML.

It should not matter who funds the measurements, or where or possibly how the data is stored (apart from efficiency), what matters is the ability to access the data in a standard way (the schema).

Next steps include deciding on attributes, and then need to define schema (in XML fragments?). We also discussed applying schemas to measurement tools. The Internet 2 people say they have a web page that categorizes network measurement tools in different ways to the NLANR & SLAC web sites, and is also kept current. They will send me the URL.

Will replace the NMWG mailing list.


I met with Olivier Martin of CERN. We discussed moving the PingER monitor host from the old to the new sunstats (pcgiga2) host at CERN. The new sunstats is a Linux host inside the firewall. The iepm-bw should be moved to  pcgiga which is a production host on the external network with a 622Mbps link. There are also new servers (with complex names) that connect to the 2.5Gbps link and are used for experiments. WE also discussed CERN running the iepm-bw monitoring code. Olivier is interested in this for a few sites in Europe. One thing he is interested in is the impact of reordering of packets by the GEANT Juniper routers. Olivier said though TCP works around this, the extra CPU cycles used by the end nodes can impact the performance and he would like to measure this. Olivier is also interested in getting single stream performance.

Richard Hughes-Jones of Manchester has provided us with disk space there, so now is very interested in getting uth/ working. I worked on installing it but ran into needing Java to be installed. He also showed me results of iperf throughput made by Yee (of UCL) and Gareth (of Manchester) using Sally Floyd's HSTCP.

Maxim Grigoriev of FNAL is working with Jerrod to get the iepm-bw monitoring toolkit running for D0, CDF, SSDS, CMS.

I met with Eric Boyd, who with Russ Hobbes, is leading the Internet 2 End-to-End Performance Initiative PiPES project (see They made available to us a host at Internet 2 that we installed iepm-bw on so they could evaluate to see whether parts of iepm-bw could fit into Pipes, and/or to make suggestions for iepm-bw. I also talked to Peter Clarke (Yee Ting Li's (who is coming back to SLAC this winter for a month, I believe) boss, and also heavily involved with Eu DataGrid) of UCL, and they are very interested in following the Pipes work for Eu DataGrid etc. measurements. There is a lot of complementarity between all our efforts and we should try and keep informed of each others plans etc. and focus on complementary areas and hopefully leverage and use each others efforts. When I talked to Eric I mentioned that we are working on a new version of iepm-bw that has improved ability to add new observations to the analysis/reporting stages andf since he has a student available to evaluate iepm-bw it would be timely for us to get Eric the latest version.

Eric also said he was hoping someone from SLAC would attend the Internet 2 meeting at the end of October. He asked if we would like to make a presentation on our work. I agreed that we would like someone from our  group to attend, however, there is a lot of activity at this time of year (GGF, SC2002), we (SLAC) will try and send someone.

Following the joint NMWG/DAMED meeting George Brett of Internet 2 (who is joining the joint NMWG/DAMED group) approached me about collaborating and in particular to point me to the NLANR/DAST Advanced Applications data base at that describes and annotates major network applications. There is considerable interest in this cince coming up with a description of network tools was one of the proposed next activities of the NMWG.

I had a discussion with Ruth Pordes of FNAL, who is interested in migrating the PPDG to using the DataGrid/Trilium (PPDG/GriPhyN/iVDGL) GLUE schema.  GLUE has schemas for computation (CE) and storage elements (SE), but not yet for network elements (NE). The EDG contact person for understanding the GLUE schemas is Sergio Andreatti of DataTAG. There is a lot of preliminary work going on in the GGF NMWG (of which I am a member) and the DAMED working groups to define a hierarchy of names and attributes for network elements (NMWG), and to come up with the actual hierarchical names and provide an implementation for some network tools such as iperf and ping (DAMED). However there are multiple schemas around (e.g. EDG's)  and Trilium is scheduled to start  work on their NE schemas in Sept '02 . Ruth thus wants to start the NE work now and make an interim recommendation  before the end of 2002. So to short cut the process, she suggest we evaluate existing schemas and make an interim recommendation for PPDG. Since she wants something that is usable in a short amount of time (before the end of the year) to provide network information, that is available for cost functions for resource brokers, this also implies that besides the schemas one needs a set of tools such as MDS (available now for for CE/SE) to go from the network measurements and their schemas to the resource brokers. Currently in MDS, the consumers of the information (e.g. a resource broker who wants a cost function to make a decision) accesses the information database via the GIIS (Grid Information Index Server).  The GIIS communicates with a GRIS (Grid Resource Information Server) to access information which the GIIS will cache and provide to the consumer. There is a GRIS  on each resource (e.g. a cluster). This is all part of the current implementation of MDS. The communication between the database and the rest of the world is currently via LDIF/LDAP (there is a script that provides this functionality). The Trilium group (uses the current LDAP based MDS CE/SE services for publish (by producers)/subscribe(consumers). There is information at on the EDG NE/MDS stuff. It appears Ruth believes that she wants SLAC (for Trilium)  and Richard Hughes-Jones (for EDG) to lead the effort to make recommendations on how to proceed with the NE schemas.

After further discussing with Richard Hughes-Jones and Peter Clarke of the EU DataGrid it appears the main person developing the new schema for the EU Datagrid network measurement toolkit (iperf, PingER etc.) and MDS is Paul Mealor of UCL. We discussed the possibility of  Paul visiting SLAC. Peter is supportive of this.

Others involved include: Brian Tierney of LBNL - who is the US/PPDG/Trillium co-lead of the overall glue-schema group. Cristina Vistoli is the DataTAG co-lead.

The next step might be to nominate the right p[erson from SLAC to work on this, understand MDS, review the EDG implementation of MDS/NE, discuss with Paul Mealor, set up the EDG/MDS/NE at SLAC. We should work with Paul Mealor to get the right version.

There are a lot of concerns about the applicability of using LDAP/LDIF as the communication mode. In the long run MDS will probably move to web services. The EDG is developing a new version to replace LDAP/LDIF with RGMA (Relational Grid Measurement Architecture). The migration scheme involves replacing GRIS/GIIS with GIN/GOUT. For the migration/continuity the information from the information databases will be provided in both LDIF/LDAP format and RGMA format. I  discussed RGMA with its architect, Steve Fisher of RAL ( The new version is ready to be put into the EDG testbed (5 sites) this week. There is a script where one provides plugins to take a flat file and convert it into the right format to put the data into an RGMA circular database. The RGMA database is accessed by the archiver which in turn is accessed by the consumer. I have transparencies from a RGMA presentation. From our point of view the schema definitions are factorizaebl from teh access mechanism (MDS), so we will stay neutral on how MDS should look and focus, initially at least, on the schema.

Peter Clarke is also interested in getting more involved with developing a monitoring architecture. 

Brian Carpenter of IBM (ex CERN) and I discussed how the GGF NMWG/DAMED work relates to CIM. CIM has been created by the Desktop Management Task Force (DMTF) an industry consortium (not part of IETF or GGF) to define schema etc. for compute and storage elements.  It is a very large effort with many documents (in some cases very large) that it daunting in size and complexity. It would be extremely desirable for the evolution of the NMWG/DAMED groups to have someone on the team who is a liaison to CIM or at least knowledgeable in CIM. Brian said there are such people that he knows in IBM who could help in this. I agreed to send email to Brian <> with a cc to the rest of the NMWG soliciting help in this.