Revision to leap seconds and calendar conventions (v3)

appa.adoc

@@ -386,7 +386,7 @@ In cases where there is a strong constraint on dataset size, it is allowed to pa
 | **`units_metadata`**
 | S
 | C, D, BI
-| <<units>>, and <<time-coordinate>>
+| <<units>>
 | Specifies the interpretation of a unit of measure appearing in the **`units`** attribute.
 
 | **`valid_max`**

appk.adoc

@@ -1,4 +1,3 @@
-
 [[appendix-mesh-topology-attributes, Appendix K, Mesh Topology Attributes]]
 
 [appendix]

appl.adoc

@@ -0,0 +1,48 @@
+[[appendix-leap-seconds, Appendix L, Leap Seconds]]
+:doc-part: L
+:figure: 0
+[appendix]
+== Leap Seconds
+
+This appendix describes the treatment of leap seconds in CF in more detail than <<calendar>>.
+Since precision to the second has rarely been needed in the climate and forecast community, leap seconds have typically been ignored, including in many datasets and earlier versions of the CF standard.
+
+If you are producing real-world datasets with datetimes in UTC, and if it's important for your time coordinates and datetimes to be accurate to the second, the **`utc`** calendar is recommended.
+When a datetime is converted to a time coordinate or vice-versa in the **`utc`** calendar, any leap seconds must be counted that occurred between the instant concerned and the reference datetime in the **`units`** attribute.
+Therefore both the writer and the user of the dataset will need leap-second-aware software for conversion between datetimes and time coordinates.
+
+For example, a leap second was added to UTC at the end of 2016.
+Therefore if **`calendar="utc"`** and **`units="seconds since 2016-12-31 23:59:58"`**, a value of 4 for the time coordinate represents the datetime 2017-01-01 00:00:01, because four seconds had elapsed by that instant since 2016-12-31 23:59:58 (seconds ending at 2016-12-31 23:59:59, 2016-12-31 23:59:60, 2017-01-01 00:00:00, 2017-01-01 00:00:01).
+In the **`utc`** calendar, the elapsed time between the same time of day on consecutive days is normally 86400 seconds, but increases to 86401 seconds when a positive leap second intervenes.
+For example, with respect to the the reference datetime in **`units="seconds since 2016-12-31 23:59:58"`**, the datetime 2017-01-01 23:59:58, which is one calendar day later, is represented by a time coordinate of 86401.
+
+In all calendars except **`utc`**, datetimes within leap seconds are not valid, reference datetimes in the **`units`** attribute must not contain seconds equal to or greater than 60, and the elapsed time between the same time of day on consecutive days is always 86400 seconds.
+
+If you are producing model-generated datasets with datetimes that follow the Gregorian calendar but do not include leap seconds, the **`proleptic_gregorian`** calendar is recommended.
+This calendar may also be used for real-world datasets with UTC datetimes to be converted to time coordinates, with two caveats:
+
+* Datetimes within leap seconds cannot be converted to time coordinates.
+* The difference between two time coordinates will differ from the duration of the time interval between the two instants by the net number of leap seconds that occurred between them.
+
+The advantages of the **`proleptic_gregorian`** calendar are:
+
+* The user can recover the datetimes exactly from the time coordinates.
+* Leap-second-aware software is not required for conversion between datetimes and time coordinates.
+
+Because the **`proleptic_gregorian`** calendar includes no leap seconds, the time coordinate for a given datetime and **`units`** may differ between this calendar and **`utc`**.
+For example, if **`calendar="proleptic_gregorian"`** and **`units="seconds since 2016-12-31 23:59:58"`**, the datetime 2017-01-01 00:00:01 is represented by a time coordinate of 3, because the UTC leap second (from 2016-12-31 23:59:60 to 2017-01-01 00:00:00) does not exist in the **`proleptic_gregorian`** calendar, whereas the time coordinate is 4 in **`utc`** with the same **`units`** attribute.
+In the **`proleptic_gregorian`** calendar, the time coordinate for 2017-01-01 00:00:58 is 60, and for 2017-01-01 23:59:58 is 86400.
+
+The **`standard`** calendar is not recommended for new datasets because of its uncertainty about leap seconds.
+The **`standard`** calendar has been used in existing datasets both for real-world data, and for data generated by models that follow the Gregorian calendar and do not include leap seconds in the timeline.
+For real-world data in the **`standard`** calendar since 1972, datetimes are in UTC or a local time zone offset from UTC.
+Hence leap seconds may have occurred between the instant of the reference datetime in the **`udunits`** and the instant referred to by a time coordinate.
+Some real-world datasets take leap seconds into account, and others do not.
+For example, equipment that automatically records data with UTC datetimes can insert a new leap second only if it has prior knowledge of when this is to be done, or is appropriately synchronized to UTC time services that deliver leap second support.
+
+As a consequence of such differences, users of real-world data in the **`standard`** calendar need to be aware that the difference between two time coordinates might not exactly equal the duration of the time interval between the two instants, but could be inaccurate by a number of seconds.
+Similar uncertainties apply if you do not know whether the data is observational or model-generated.
+Furthermore, the time coordinates for the same real-world instant in two datasets, both using the **`standard`** calendar, could disagree by some number of seconds.
+
+CF version 1.12 introduced possible values of the form "**`leap_seconds:`** __value__" for the **`units_metadata`** attribute, in order to specify the treatment of leap seconds in the **`standard`**, **`julian`** and **`proleptic_gregorian`** calendars.
+CF version 1.13 withdrew this use of **`units_metadata`**, in favour of the recommendations given above that the **`utc`** or **`proleptic_gregorian`** calendars should be used by data-producers instead of **`standard`**.

bibliography.adoc

@@ -19,6 +19,7 @@ Federal Geographic Data Committee, FGDC-STD-001-1998.
 OGC document 12-063. 1st May 2015.
 - [[[OGP-EPSG_GN7_2]]]  link:$$https://epsg.org/guidance-notes.html$$[OGP Surveying and Positioning Guidance Note 7, part 2: Coordinate Conversions and Transformations including Formulas].
 - [[[SCH02]]] link:$$https://doi.org/10.1175/1520-0493(2002)130<2459:ANTFVC>2.0.CO;2$$[A new terrain-following vertical coordinate formulation for atmospheric prediction models]. C Schaer, D Leuenberger, and O Fuhrer. 2002. _Monthly Weather Review_.  130. 2459-2480.
+- [[[SI]]] link:$$https://doi.org/10.59161/AUEZ1291$$[SI Brochure: The International System of Units (SI)], 9th edition, BIPM, 2019. 
 - [[[Snyder]]]  link:$$https://doi.org/10.3133/pp1395$$[Map Projections: A Working Manual]. USGS Professional Paper 1395.
 - [[[UDUNITS]]]  link:$$https://doi.org/10.5065/D6KD1WN0$$[UDUNITS Software Package].  UNIDATA Program Center of the University Corporation for Atmospheric Research.
 - [[[UGRID]]]  link:$$https://ugrid-conventions.github.io/ugrid-conventions$$[UGRID Conventions for storing unstructured (or flexible mesh) data in netCDF files]

cf-conventions.adoc

@@ -1,6 +1,6 @@
 include::version.adoc[]
 = NetCDF Climate and Forecast (CF) Metadata Conventions
-Brian{nbsp}Eaton; Jonathan{nbsp}Gregory; Bob{nbsp}Drach; Karl{nbsp}Taylor; Steve{nbsp}Hankin; John{nbsp}Caron; Rich{nbsp}Signell; Phil{nbsp}Bentley; Greg{nbsp}Rappa; Heinke{nbsp}Höck; Alison{nbsp}Pamment; Martin{nbsp}Juckes; Martin{nbsp}Raspaud; Randy{nbsp}Horne; Jon{nbsp}Blower; Timothy{nbsp}Whiteaker; David{nbsp}Blodgett; Charlie{nbsp}Zender; Daniel{nbsp}Lee; David{nbsp}Hassell; Alan{nbsp}D.{nbsp}Snow; Tobias{nbsp}Kölling; Dave{nbsp}Allured; Aleksandar{nbsp}Jelenak; Anders{nbsp}Meier{nbsp}Soerensen; Lucile{nbsp}Gaultier; Sylvain{nbsp}Herlédan; Fernando{nbsp}Manzano; Lars{nbsp}Bärring; Christopher{nbsp}Barker; Sadie{nbsp}Bartholomew; Thomas Lavergne
+Brian{nbsp}Eaton; Jonathan{nbsp}Gregory; Bob{nbsp}Drach; Karl{nbsp}Taylor; Steve{nbsp}Hankin; John{nbsp}Caron; Rich{nbsp}Signell; Phil{nbsp}Bentley; Greg{nbsp}Rappa; Heinke{nbsp}Höck; Alison{nbsp}Pamment; Martin{nbsp}Juckes; Martin{nbsp}Raspaud; Randy{nbsp}Horne; Jon{nbsp}Blower; Timothy{nbsp}Whiteaker; David{nbsp}Blodgett; Charlie{nbsp}Zender; Daniel{nbsp}Lee; David{nbsp}Hassell; Alan{nbsp}D.{nbsp}Snow; Tobias{nbsp}Kölling; Dave{nbsp}Allured; Aleksandar{nbsp}Jelenak; Anders{nbsp}Meier{nbsp}Soerensen; Lucile{nbsp}Gaultier; Sylvain{nbsp}Herlédan; Fernando{nbsp}Manzano; Lars{nbsp}Bärring; Christopher{nbsp}Barker; Sadie{nbsp}Bartholomew; Thomas{nbsp}Lavergne; Seth{nbsp}McGinnis; Patrick{nbsp}Van{nbsp}Laake
 :revnumber: {current-version}
 :revdate: {docprodtime}
 :doctype: book
@@ -73,6 +73,8 @@ include::toc-extra.adoc[]
 * Christopher Barker, NOAA
 * Sadie Bartholomew, NCAS and University of Reading
 * Thomas Lavergne, MET Norway
+* Seth McGinnis, NCAR
+* Patrick Van Laake, Independent data scientist
 
 Many others have contributed to the development of CF through their participation in discussions about proposed changes.
 
@@ -148,6 +150,9 @@ include::appj.adoc[]
 :numbered!:
 include::appk.adoc[]
 
+:numbered!:
+include::appl.adoc[]
+
 :numbered!:
 include::history.adoc[]
 

ch01.adoc

@@ -84,7 +84,7 @@ The coordinate values must be in strict monotonic order (all values are differen
 Missing values are not allowed in coordinate variables.
 To avoid confusion with coordinate variables, CF does not permit a one-dimensional string-valued variable to have the same name as its dimension.
 
-datetime:: The set of numbers which together identify an instant of time, namely its year, month, day, hour, minute and second, where the second may have a fraction but the others are all integer.
+datetime:: The set of numbers which together identify an instant of time, namely its year, month, day, hour, minute and second, where the second may have a fraction but the others are all integer. See <<time-coordinate-units>>.
 
 grid mapping variable:: A variable used as a container for attributes that define a specific grid mapping.
 The type of the variable is arbitrary since it contains no data.

ch03.adoc

@@ -13,6 +13,7 @@ But since it is an optional attribute, applications that implement these standar
 
 The **`units`** attribute is required for all variables that represent dimensional quantities (except for boundary variables defined in <<cell-boundaries>> and climatology boundary variables defined in <<climatological-statistics>>).
 The **`units`** attribute is permitted but not required for dimensionless quantities (see <<dimensionless-units>>).
+If multiplication by a dimensionless constant and addition of a dimensionless constant are the only operations required for the value of a dimensional quantity expressed in one unit to be converted to the value expressed in another unit, we describe the two units as __physically equivalent__.
 
 The value of the **`units`** attribute is a string that can be recognized by the UDUNITS package <<UDUNITS>>, with the exceptions that are given in <<dimensionless-units>> and <<units-multiples>>.
 Note that case is significant in the **`units`** strings.
@@ -111,7 +112,7 @@ For instance, when temperature is on-scale, a value in `kg degree_C m-2` can be
 [[units-multiples, Section 3.1.3, "Scale factors and offsets"]]
 ==== Scale factors and offsets
 
-UDUNITS recognises the SI prefixes shown in <<table-supported-units>> for decimal multiples and submultiples of units, and allows them to be applied to non-SI units as well.
+UDUNITS recognises the <<SI>> prefixes shown in <<table-supported-units, Table 3.1>> for decimal multiples and submultiples of units, and allows them to be applied to non-SI units as well.
 UDUNITS offers a syntax for indicating arbitrary scale factors and offsets to be applied to a unit.
 (Note that this is different from the scale factors and offsets used for converting between **`units`**, as discussed for temperature in <<temperature-units>>.)
 This UDUNITS syntax for arbitrary transformation of **`units`** is not supported by the CF standard, except for the case of specifying reference time (<<time-coordinate>>).
@@ -163,13 +164,13 @@ standard name:: The name used to identify the physical quantity.
 A standard name contains no whitespace and is case sensitive.
 
 canonical units:: Representative units of the physical quantity.
-Unless it is dimensionless, a variable with a **`standard_name`** attribute must have units which are physically equivalent (not necessarily identical) to the canonical units, possibly modified by an operation specified by the standard name modifier (see below and <<standard-name-modifiers>>) or by the **`cell_methods`** attribute (see <<cell-methods>> and <<appendix-cell-methods>>) or both.
+Unless it is dimensionless, a variable with a **`standard_name`** attribute must have units which are physically equivalent (not necessarily identical, see <<units>>) to the canonical units, possibly modified by an operation specified by the standard name modifier (see below and <<standard-name-modifiers>>) or by the **`cell_methods`** attribute (see <<cell-methods>> and <<appendix-cell-methods>>) or both.
  +
  +
 Units of time coordinates (<<time-coordinate>>), whose **`units`** attribute includes the word **`since`**, are _not_ physically equivalent to time units that do not include **`since`** in the **`units`**.
-To mark this distinction, the canonical unit given for quantities used for time coordinates is **`s since 1958-1-1`**.
-The reference datetime in the canonical unit (the beginning of the day i.e. midnight on 1st January 1958 at 0 `degrees_east`) is not restrictive; the time coordinate variable's own **`units`** may contain any reference datetime (after **`since`**) that is valid in its calendar.
-(We use `1958-1-1` because it is the beginning of International Atomic Time, and a valid datetime in all CF calendars; see also <<leap-seconds>>.)
+To mark this distinction, the canonical unit given for quantities used for time coordinates is **`s since 1972-01-01`**.
+The reference datetime in the canonical unit (the beginning of the day i.e. midnight on 1st January 1972 at 0 `degrees_east`) is not restrictive; the time coordinate variable's own **`units`** may contain any reference datetime (after **`since`**) that is valid in its calendar.
+(We use `1972-01-01` because it is when the current definition of UTC came into force, and a valid datetime in all CF calendars; see also <<calendar>>.)
 In both kinds of time **`units`** attribute (with or without **`since`**), any unit for measuring time can be used i.e. any unit which is physically equivalent to the SI base unit of time, namely the second.
 
 description:: The description is meant to clarify the qualifiers of the fundamental quantities such as which surface a quantity is defined on or what the flux sign conventions are.