argo-bann.jpg (13838 octets)

Data Management Workshop
Brest (France)
  3-4-5 Octobre   

 

Contribution : Data Transmission by Etienne Charpentier

 

Two issues are being discussed here:

  1. Formats for GTS distribution of float data
  2. Data to submit to the AIC
  1. Formats for GTS distribution of Argo float data

    2.  

    Only the following code formats may be used for GTS distribution of Argo float data. Other formats would not comply with GTS regulations. All data distributed on GTS are directly and automatically assimilated by numerical meteorological or oceanographic models operated by major meteorological centres. Any other format would oblige each centre to develop specific data assimilation schemes and would encourage other applications to define their specific formats with the risk of eventually having a multitude of formats to deal with.

     

    1. Possible GTS formats:

      2. FM 64-XI Ext. TESAC (KKYY): The floats presently reporting on GTS are using TESAC format (temperature/salinity) which permits a resolution of 0.01 Celsius and 0.01 part per thousand.

      FM 63-XI Ext. BATHY (JJVV) only permits a resolution of 0.1 Celsius and 0.1 part per thousand. It is also limited to 20 points in the upper 500 meters. BATHY is used principally for XBTs.

      FM 18-X BUOY (ZZYY) code permits a resolution of 0.01 Celsius and 0.01 part per thousand. This code is used for drifting and moored buoys. It could also be used for floats.

      FM-94-X Ext. BUFR permits any resolution and provides for inclusion of associated data (e.g. QC flags), metadata as well as specific identification fields (e.g. Argos ID). Format is binary and permits compression, a feature which might be interesting for profile data. It is more complicated than character codes but encoders and decoders can be obtained from meteorological services

      FM-95-XI Ext. CREX permits any resolution and provides for inclusion of metadata as well as specific identification fields (e.g. Argos ID). This is a character code (i.e. a person can read it) which uses the same structure than BUFR. It also uses the same code tables than BUFR to describe the data. However, CREX does not permit compression nor associated fields (e.g. QC flags).

       

      Summary table:

      Format Current version Type Temp.

      resolution

      		 Sal.

      resolution

      		 Depth

      resolution

      		 Advantages Drawbacks
      TESAC FM 64-XI Ext.

      (KKYY)

      		 ASCII 0.01C 0.01 psu 1m. Significan points Resolution Rigid

      Poor depth resolution

      Eventually abandoned (2005)
      BATHY FM 63-XI Ext.

      (JJVV)

      		 ASCII 0.1C 0.1 psu 1m. Significant points (limited to 20) Widely used (XBTs) Resolution (D,T,S)

      Rigid

      Limited to 20 points in the upper 500m

      Eventually abandoned (2005)
      BUOY FM 18-X

      (ZZYY)

      		 ASCII 0.01C 0.01 psu 1m. Significant points (limited to 20) Resolution

      Widely used (buoys)

      		 Rigid

      Poor depth resolution

      Eventually abandoned (2005)
      BUFR FM 94-X Ext.

      (BUFR)

      		 Binary

      Table driven

      		 No limitation No limitation No limitation Flexible

      Self describing

      Metadata

      QC flags

      Compression

      		 Complex (encoders/decoders exist)
      CREX FM 95-XI Ext.

      (CREX)

      		 ASCII

      Table driven

      		 No limitation No limitation No limitation Flexible

      Self describing

      Metadata

      Readable

      		 Complex (encoders/decoders exist)

      Eventually abandoned (2009)

       

      All code forms are formally documented in the WMO manual on codes, WMO No. 306, Part A - Alphanumeric codes, and part B & C - Binary codes & Common features to Binary and Alphanumeric Codes.

       
    2. Choosing among character code:

      3. Among character codes, BUOY and TESAC are better. TESAC might be preferable to BUOY because users looking for sub-surface temperature and salinity data would rather check TESAC reports first. TESAC is in fact dedicated to temperature and salinity profile data. So for float data, unless other instruments are installed on the floats, TESAC should be preferred to any other character code. Details regarding the BATHY and TESAC formats can be found via the SOOPIP web site at http://www.brest.ird.fr/soopip/gts.html. It is now extremely difficult to modify character codes in order to encode new variables for which there is presently no provision in those codes. This is because the Commission for Basic Systems (CBS) of WMO now encourages the use of table driven codes.
    3. Choosing among table driven codes:

      4. Among table driven codes, BUFR is better than CREX for Argo needs because it permits encoding of QC flags and permits compression. CREX is a character code while BUFR is a binary code so CREX permits human readability but this is not a specific requirement for Argo data which are distributed on GTS primarily for direct data assimilation by numerical weather prediction models. CREX also will probably eventually be abandoned.

    4. Character codes versus table driven codes:

      5. Character codes are indeed easier to implement because they are simple but all character codes will eventually be abandoned probably around 2005, i.e. when Argo will be fully implemented. On the other hand, encoders for BUFR can be obtained from NOAA (http://www.nws.noaa.gov/tdl/iwt/) or ECMWF. Considering above discussion we probably have to make a choice between TESAC and BUFR. According to requirements expressed by the Argo Science Team, it is clear that BUFR is preferable to TESAC because it permits encoding of Quality Control flags plus encoding of specific fields such as float id, Argos or Orbcomm ID, and profile ID. It is relatively easy to add new entries in BUFR or CREX tables. Those changes should be submitted to the CBS. For example, the Data Buoy Cooperation Panel succeeded in adding specific table entries for surface drifters. Same mechanisms can be used. Details regarding BUFR tables can be found via the WMO web site at http://www.wmo.ch/web/ddbs/Code-tables.html . Details regarding CREX tables can be found via the WMO web site at http://www.wmo.ch/web/ddbs/CREX-Tables.html

       
    5. Proposed approach for GTS distribution of Argo data:

Considering above discussion, a proposed approach could be:

    1. Immediate distribution of all Argo float data in TESAC.
    2. Every float operator to take steps to eventually and gradually use BUFR instead of TESAC.

Before Argo data are actually distributed in BUFR, Argo operators must agree on what information exactly to distribute in BUFR reports and in what resolutions. It might indeed be required to request new table entries or to modify existing ones. In addition, specific templates might be proposed in order to minimize the size of BUFR reports. Based on those requirements, the Argo Coordinator will then submit s specific request to CBS. Meanwhile, float operators can work on the development of BUFR encoders or someone could be tasked to develop a software that would automatically translate standard Argo data sets into BUFR (e.g. Net-CDF to BUFR).

  1.  
  2. Data to submit to the Argo Information Center

    3.  

    1. Need for a list of operational floats

 

Meteorological or Oceanographic Centres assimilating Argo float data from the GTS data-flow do not necessarily know which of the floats are truly Argo floats, who operate the floats, and cannot easily access to crucial metadata (e.g. float type, stated sensor accuracies etc.). Typically, the only information they have to deduce the float operator is the WMO number. On the other hand, they can potentially provide float operators with valuable feed-back information regarding the quality of the float data (e.g. comparisons of the observations with the model first guess field or analysis). Maintaining a list of Argo floats which would among other things include the WMO number, float ID , Argos or Orbcomm ID, the name of the float operator, and certain metadata is therefore important.

The list could also be used by the meteorological services to sort out truly Argo floats from the GTS dataflow and produce specific products, including statistics regarding the overall quality of Argo floats, delays, availability, etc.

Such a list must be centralized and easily accessible (e.g. via the web). It is proposed that such a list be maintained by the Argo Coordinator at the Argo Information Center (AIC). To realize this, float operators are invited to agree to submit information about the floats they deploy to the Argo Coordinator. Basically, information to provide would include:

  1. Float ID
  2. Satellite system used (Argos, Orbcomm..)
  3. Formally Argo float (Y/N)
  4. Telecomm ID (Argos or Orbcomm)
  5. WMO number
  6. GTS bulletin header
  7. Deployment position
  8. Parking depth
  9. Cycle
  10. T measured (Y/N) and resolution
  11. S measured (Y/N) and resolution

 

After discussion on this topic at the Argo data management meeting in Brest, October 2000, the Argo coordinator will submit a proposal to the Argo Science Team regarding how to submit the data and in what format. This could be done in conjunction with the float deployment notification mechanism in order to avoid having float operators to submit the same information twice.

    1. Float positions (IOC resolution XX-6, EEZ issue)

To comply with the IOC XX-6 resolution, member states must be informed about the floats which enter their EEZs. This will be done through the AIC web site. Argo data centres must therefore routinely (real-time or daily) provide the AIC with the positions of all Argo floats. Minimal information to submit would include for all collected float observations:

    1. Float ID
    2. Date and time of observation
    3. Float position

 

And in order to offer other products, additional details such as these listed below could be provided as well:

    1. Profile depth
    2. Availability of T (Y/N)
    3. Availability of S (Y/N)

The meeting is invited (i) to agree on what data to submit, and (ii) suggest a mechanism for submitting the data routinely to the AIC.