Revision to leap seconds and calendar conventions (v3) #612
Conversation
ch04.adoc
Outdated
| In this table, "/" means "or", "-∞" means "any date (provided it is earlier than the **Last date**, if any)" and "+∞" means "any date (provided it is later than the **First date**, if any)". | ||
| In the **Months** column, "usual" means there are 12 months with the usual lengths expected by calendar software (31 days in January, etc.). | ||
| These month lengths are the same in all calendars marked "usual", except that the calendars differ regarding the rules for determining leap years, when February has 29 days instead of 28. | ||
| In the **Leap days** column, "Gregorian" and "Julian" identify different rules for determing which years are leap years. |
There was a problem hiding this comment.
In the **Leap days** column, "Gregorian" and "Julian" identify different rules for determining which years are leap years.
There was a problem hiding this comment.
I assume you mean the typo in "determining"? Thanks for spotting it. Fixed.
ch04.adoc
Outdated
| time:axis = "T"; // axis attribute is optional | ||
| time:standard_name = "time" ; // standard_name attribute is optional | ||
| time:calendar = "standard" ; // calendar attribute is recommended | ||
| time:units = "days since 1990-1-1 0:0:0" ; // units attribute is mandatory |
There was a problem hiding this comment.
| time:axis = "T"; // axis attribute is optional | |
| time:standard_name = "time" ; // standard_name attribute is optional | |
| time:calendar = "standard" ; // calendar attribute is recommended | |
| time:units = "days since 1990-1-1 0:0:0" ; // units attribute is mandatory | |
| time:axis = "T" ; // axis attribute is optional | |
| time:standard_name = "time" ; // standard_name attribute is optional | |
| time:calendar = "standard" ; // calendar attribute is recommended | |
| time:units = "days since 1990-1-1 0:0:0" ; // units attribute is mandatory |
Just add som space to make it easier to read what is CDL and what is comment
There was a problem hiding this comment.
Thanks. I've aligned the // vertically now.
ch04.adoc
Outdated
| The acceptable units of measure for time are given by UDUNITS. | ||
| The most commonly used of these strings (and their abbreviations) are **`day`** (**`d`**), **`hour`** (**`hr`**, **`h`**), **`minute`** (**`min`**) and **`second`** (**`sec`**, **`s`**). | ||
| The CF standard follows UDUNITS (<<units>>) in the definition of the acceptable units of measure for time. | ||
| The most commonly used of these units strings (and their symbols) are **`day`** (**`d`**), **`hour`** (**`h`**), **`minute`** (**`min`**) and **`second`** (**`s`**). |
There was a problem hiding this comment.
| The most commonly used of these units strings (and their symbols) are **`day`** (**`d`**), **`hour`** (**`h`**), **`minute`** (**`min`**) and **`second`** (**`s`**). | |
| The most commonly used of these units (and their symbols) are **`day`** (**`d`**), **`hour`** (**`h`**), **`minute`** (**`min`**) and **`second`** (**`s`**). |
ch04.adoc
Outdated
| | **`tai`** | 365/366 | usual | Gregorian | no | 1958-01-01 | +∞ | no | ||
| |=============== | ||
| In this table, "/" means "or", "-∞" means "any date (provided it is earlier than the **Last date**, if any)" and "+∞" means "any date (provided it is later than the **First date**, if any)". | ||
| In the **Months** column, "usual" means there are 12 months with the usual lengths expected by calendar software (31 days in January, etc.). |
There was a problem hiding this comment.
I agree that the word is not optimal, but neither I can find a better one. ChatGPT made several suggestions: of which the only reasonable one is "J/G" for Julian/Gregorian, which is essentially what these calendars are about (in addition to the algoritm for leap years). But this is just a suggestion to consider.
There was a problem hiding this comment.
I suggest that we can solve this problem by deleting the "Months" column. I think it's implied by the length of the year. Therefore I've deleted the column and the table caption text about it, and I have modified the table caption comment about the Leap days column to read as follows (new text in bold):
In the Leap days column, "Gregorian" and "Julian" identify different rules for determining whether a given year is a common year (with 365 days) or a leap year (with 366). By the Gregorian rule, etc. [as before]
I have moved the paragraph in the main text about lengths of months to become the first paragraph after the table, with the following text (new text in bold):
The lengths of the months defined by the Julian and Gregorian calendars (12 months in the year, 31 days in January, etc.) are used in all calendars except 360_day (in which there are 12 months of 30 days each), none (see <<none-calendar>>) and explicitly defined calendars (see <<explicit-calendar>>). The calendars differ in their treatment of leap years (when there are 29 days in February instead of 28).
ch04.adoc
Outdated
| By the Gregorian rule, a year is a leap year if either (i) it is divisible by 4 but not by 100 or (ii) it is divisible by 400. | ||
| By the Julian rule, any year that is divisible by 4 is a leap year, even if it is also divisible by 100. | ||
| In the **Year 0** column, "yes" and "no" respectively indicate that there are, or are not, valid dates in year 0 in this calendar. | ||
| In calendars marked with *, dates in year 0 are climatological; this use of year 0 is deprecated (see <<climatological-statistics>>). |
There was a problem hiding this comment.
| In calendars marked with *, dates in year 0 are climatological; this use of year 0 is deprecated (see <<climatological-statistics>>). | |
| In calendars marked with *, dates in year 0 used to indicate a climatology, but this use is deprecated (see <<climatological-statistics>> for the recommended mechanism). |
There was a problem hiding this comment.
Thanks. Adopted ".. in year 0 are used to indicate ...".
ch04.adoc
Outdated
|
|
||
| In a given calendar, each valid datetime identifies a particular instant in the continuous physical dimension of time. | ||
| The reference datetime in the **`units`** of a time coordinate variable must be a valid datetime in the calendar of the variable, and identifies a particular instant according to that calendar. | ||
| A datetime which is invalid in a given calendar cannot be converted into a time-coordinate in that calendar. |
There was a problem hiding this comment.
| A datetime which is invalid in a given calendar cannot be converted into a time-coordinate in that calendar. | |
| A datetime which is invalid in a given calendar cannot be converted into a time coordinate value in that calendar. |
ch04.adoc
Outdated
| For example, a time coordinate of **1** with **`units="days since 2020-02-28 23:10"`** represents the datetime 2020-02-29 23:10 in the **`standard`** calendar, but 2020-03-01 23:10 in the **`no_leap`** calendar (in which 2020 is not a leap year). | ||
|
|
||
| In the **`julian`** and the **`standard`** calendar, dates in years before year 0 (i.e. before 0-01-01 00:00:00) are not allowed, and the year in the reference datetime of the units must not be negative. | ||
| In these calendars, year zero has a special use to indicate a climatology (see <<climatological-statistics>>), but this use of year zero is deprecated. |
There was a problem hiding this comment.
This is an exact (at least very nearly) duplicate of what was stated just two paragraphs up:
Delete, or shorten substantially and merge into next sentence.
There was a problem hiding this comment.
The first occasion is in the footnotes to the table, but this isn't clearly delimited. I will see if I can improve the formatting. In any case, I agree we shouldn't have excessive duplication. Therefore I have changed:
- the column heading from Year 0 to Years <1.
- the table footnote to read:
In the Years <1 column, "yes" and "no" respectively indicate that there are, or are not, valid dates before year 1 in this calendar. In calendars marked with *, dates in year 0 are used to indicate a climatology, but this use is deprecated (see
<<climatological-statistics>>for the recommended mechanism).
in which the second sentence is unchanged.
- the main text to read:
In the
julianand thestandardcalendar, dates in years before year 1 (i.e. before 1-01-01 00:00:00) are invalid, and must not be used in the reference datetime of theunitsattribute. In other calendars, year 0 is the year before year 1, and negative years are allowed.
I believe this is just a re-expression, not a change in content, except that the old text seemed to allow a reference datetime in year 0 in standard and julian, which certainly should not be allowed. Dates in the deprecated climatological year 0 are not really valid dates. No instant of time is identified by their coordinates. This re-expression reduces the visibility of the deprecated convention and make the text simpler. I hope that's all right.
There was a problem hiding this comment.
Ah, now I remember why reference datetimes are allowed in climatological year zero; it's because you couldn't otherwise define time coordinates in this year. Looking again at the table, I think we don't actually need the "year 0" or "years <1" column. The "first date" column already correctly states the first valid date (year 1 for standard and julian). Therefore I have deleted the column and the text about it just below the table, and mentioned the year 0 matter only in the main text, as follows:
In all calendars except
julianandstandard, year 0 is the year before year 1, and negative years are allowed. In thejulianandstandardcalendars, dates in years before year 0 are invalid and must not be used in the reference datetime of theunitsattribute. In these calendars, datetimes in year 0, including as the reference datetime inunits, are allowed only to indicate a climatology. This use is deprecated (see<<climatological-statistics>>for the recommended mechanism).
There was a problem hiding this comment.
I find it very confusing to write "... are allowed ... is deprecated". While there is a fine difference between "allow" (or, more correctly, the opposite "disallow" or "forbid" ) and "deprecate", this is nevertheless confusing, if not outright contradictory. The use of year 0 to indicate climatology has been recommended against in slightly different wording all the way back to CF-1.1:
"We do not recommend this convention because (a) it does not provide any information about the intervals used to compute the climatology, and (b) there is no standard for how dates since year 1 will be encoded with units having a reference time in year 0, since this year does not exist; consequently there may be inconsistencies among software packages in the interpretation of the time coordinates."
In the light of this long-standing deprecation, and the strong arguments why this is the case, I find it unhelpful to write "are allowed" (present tense) and only in a separate sentence afterwards state that this is deprecated. In this context of explaining calendars, it would be much more appropriate the simply state that this use of year 0 is deprecated, the reference to Chapter 7.4, where the text I cited appears, is already in place.
There was a problem hiding this comment.
I know this is an ancient recommendation. I've changed it to "In these calendars, datetimes in year 0 indicate a climatology. This use is deprecated (see <<climatological-statistics>> for the recommended mechanism)."
There was a problem hiding this comment.
I think we should strengthen this by writing "In these calendars, datetimes in year 0 were previously used to indicate a climatology. This use is deprecated (see <> for the recommended mechanism)."
appl.adoc
Outdated
|
|
||
| For example, a leap second was added to UTC at the end of 2016. | ||
| Therefore if **`calendar="utc"`** and **`units="seconds since 2016-12-31 23:59:58"`**, a value of 4 for the time coordinate represents the datetime 2017-01-01 00:00:01, because four seconds had elapsed by that instant since 2016-12-31 23:59:58 (seconds ending at 2016-12-31 23:59:59, 2016-12-31 23:59:60, 2017-01-01 00:00:00, 2017-01-01 00:00:01). | ||
| In the **`utc`** calendar, two instants with the same time of day on different days, which would always be separated by a multiple of 86400 seconds if there were no leap seconds, will have a few more seconds between them if positive leap seconds intervene. |
There was a problem hiding this comment.
I find this sentence very difficult to "parse". Here is an attempt to simplify:
| In the **`utc`** calendar, two instants with the same time of day on different days, which would always be separated by a multiple of 86400 seconds if there were no leap seconds, will have a few more seconds between them if positive leap seconds intervene. | |
| In the **`utc`** calendar, the elapsed time between the same time of day on consecutive days is normally 86400 seconds, but increases to 86401 seconds when a positive leap second intervenes. |
There was a problem hiding this comment.
+1 on Lars' suggestion
There was a problem hiding this comment.
Thanks. Included.
appl.adoc
Outdated
| == Leap Seconds | ||
|
|
||
| This appendix describes the treatment of leap seconds in CF in more detail than <<calendar>>. | ||
| Most people ignore the existence of leap seconds, including many data producers and the CF standard before version 1.12. |
There was a problem hiding this comment.
The current wording (‘Most people ignore…’) may suggest that this is due to misunderstanding or neglect, whereas the actual reason is that the climate and forecast community has rarely needed second-level time precision in either observational or model data. Therefore, here are several alternative formulations that convey this point more accurately:
“Because there have been few use cases requiring second-level precision in the climate and forecast community, leap seconds have generally been ignored, including in many datasets and in the CF standard before version 1.12.”
“As second-level precision has rarely been needed in the climate and forecast community, leap seconds have typically been ignored, including in many datasets and earlier versions of the CF standard.”
“Since the climate and forecast community has had little need for second-level precision, leap seconds have usually been omitted in practice, both in datasets and in the CF standard prior to version 1.12.”
“Because there have been no concrete use cases requiring second-level precision in the climate and forecast community, leap seconds have generally been ignored, including in many datasets and in the CF standard before version 1.12.”
“Historically, the climate and forecast community has had little need for second-level precision, so leap seconds have typically been ignored in practice, including in datasets and in the CF standard prior to version 1.12.”
There was a problem hiding this comment.
Of course, the suggested sentences can be sharpened by removing
"generally",
"typically",
"usually",
"generally",
"typically".
This would then (more) accurately describe the situation as it is/was in CF prior to 1.12.
There was a problem hiding this comment.
I vote for:
“As second-level precision has rarely been needed in the climate and forecast community, leap seconds have typically been ignored, including in many datasets and earlier versions of the CF standard.”
There was a problem hiding this comment.
Thanks for considering this. I've put in the sentence Chris has restated. The text we had before wasn't written for this version, by the way. It was part of what we intended to be an easy-to-understand summary of the issue in CF 1.12.
appl.adoc
Outdated
| For example, with respect to the the reference datetime in **`units="seconds since 2016-12-31 23:59:58"`**, the datetime 2017-01-01 23:59:58, which is one calendar day later, is represented by a time coordinate of 86401. | ||
|
|
||
| In all calendars except **`utc`**, datetimes within leap seconds are not valid, and reference datetimes in the **`units`** attribute must not contain seconds equal to or greater than 60. | ||
| In these calendars, when a time coordinate value is calculated from a datetime, or the reverse, it is assumed that the coordinate value increases by exactly 60 seconds from the start of any minute (identified by year, month, day, hour, minute, all being integers) to the start of the next minute. |
There was a problem hiding this comment.
| In these calendars, when a time coordinate value is calculated from a datetime, or the reverse, it is assumed that the coordinate value increases by exactly 60 seconds from the start of any minute (identified by year, month, day, hour, minute, all being integers) to the start of the next minute. |
This sentence does not add anything.
There was a problem hiding this comment.
agree with Lars on that.
There was a problem hiding this comment.
I agree, it doesn't say anything different. It states the situation for the opposite case, to be clear. I have deleted this sentence and added a phrase to the previous sentence, referring back to Lars's new version of the statement about the length of the day in UTC. The sentence now reads: "In all calendars except utc, datetimes within leap seconds are not valid, reference datetimes in the units attribute must not contain seconds equal to or greater than 60, and the elapsed time between the same time of day on consecutive days is always 86400 seconds."
appl.adoc
Outdated
| For example, if **`calendar="proleptic_gregorian"`** and **`units="seconds since 2016-12-31 23:59:58"`**, the datetime 2017-01-01 00:00:01 is represented by a time coordinate of 3, because the UTC leap second (from 2016-12-31 23:59:60 to 2017-01-01 00:00:00) does not exist in the **`proleptic_gregorian`** calendar, whereas the time coordinate is 4 in **`utc`** with the same **`units`** attribute. | ||
| In the **`proleptic_gregorian`** calendar, the time coordinate for 2017-01-01 00:00:58 is 60, and for 2017-01-01 23:59:58 is 86400. | ||
|
|
||
| The **`standard`** calendar is not recommended for new datasets because of its uncertainty about leap seconds. |
There was a problem hiding this comment.
Do we really want/need to make this blanket recommendation against using the standard calendar? As we know, almost all use cases do not need (or want) this second-precision accuracy. We must not be unaware of the fact that making the recommendation as stated may initiate a lot of work on changing workflow and software systems. Or, alternatively, that people and organisations decide not to follow the stated recommendation, which only undermines the status of the CF Conventions.
| The **`standard`** calendar is not recommended for new datasets because of its uncertainty about leap seconds. | |
| The **`standard`** calendar is not recommended for new datasets where current or future use may require second-precision time data. |
There was a problem hiding this comment.
As I put in the issue comments, I think we should generally recommend the standard calendar for model results.
The standard calendar is for when leap seconds aren't relevant (i.e. from above ", leap seconds have typically been ignored") -- that is still the case.
Granted the other difference between standard and proleptic_gregorian is the "proleptic" part -- but that's a whole other question -- if folks are encoding time before pope gregory, the leap seconds are certainly not an issue ...
appl.adoc
Outdated
| For real-world data in the **`standard`** calendar since 1972, datetimes are in UTC or a local time zone offset from UTC. | ||
| Hence leap seconds may have occurred between the instant of the reference datetime in the **`udunits`** and the instant referred to by a time coordinate. | ||
| Some real-world datasets take leap seconds into account, and others do not. | ||
| For example, equipment which automatically records data with UTC datetimes can insert a new leap second only if it has has prior knowledge of when this is to be done. |
There was a problem hiding this comment.
| For example, equipment which automatically records data with UTC datetimes can insert a new leap second only if it has has prior knowledge of when this is to be done. | |
| For example, equipment which automatically records data with UTC datetimes can insert a new leap second only if it has has prior knowledge of when this is to be done, or is appropriately synchronized to UTC time service providers that deliver full leap second support. |
See comment in email.
There was a problem hiding this comment.
there's a typo: "has has" ...
but I like the rest of your suggestion.
There was a problem hiding this comment.
I would also change from "equipment which" to "equipment that"
There was a problem hiding this comment.
I think it should be "synchronized to UTC time services" not "... time service providers". Also, I don't think "full" adds well-defined information, so we can omit. That makes the sentence: For example, equipment that automatically records data with UTC datetimes can insert a new leap second only if it has prior knowledge of when this is to be done, or is appropriately synchronized to UTC time services that deliver leap second support.
appl.adoc
Outdated
| As a consequence of such differences, __users of real-world data in the **`standard`** calendar need to be aware that the difference between two time coordinates might not exactly equal the duration of the time interval between the two instants, but could be inaccurate by a number of seconds.__ | ||
| Similar uncertainties apply if you do not know whether the data is observational or model-generated. | ||
| Furthermore, __the time coordinates for the same real-world instant in two datasets, both using the **`standard`** calendar, could disagree by some number of seconds.__ |
ch04.adoc
Outdated
| The CF standard follows UDUNITS (<<units>>) in the definition of the acceptable units of measure for time. | ||
| The most commonly used of these units (and their symbols) are **`day`** (**`d`**), **`hour`** (**`h`**), **`minute`** (**`min`**) and **`second`** (**`s`**). | ||
| Plural forms are also acceptable. | ||
| In CF, following UDUNITS, any unit may optionally have one of the decimal prefixes for multiples and submultiples (<<table-supported-units, Table 3.1>>) e.g. **`millisecond`** or **`ms`**; however, we note that <<SI>> does not allow these prefixes for any unit of time other than **`second`**. |
There was a problem hiding this comment.
| In CF, following UDUNITS, any unit may optionally have one of the decimal prefixes for multiples and submultiples (<<table-supported-units, Table 3.1>>) e.g. **`millisecond`** or **`ms`**; however, we note that <<SI>> does not allow these prefixes for any unit of time other than **`second`**. | |
| In CF, following UDUNITS, any unit may optionally have one of the decimal prefixes for multiples and submultiples (<<table-supported-units, Table 3.1>>) e.g. **`millisecond`** or **`ms`**. |
The SI part of this sentence may well imply that you shouldn't, or even can't, have units such as kday, when CF has no such restrictions.
There was a problem hiding this comment.
The actual use of millihour, megaminute, kday, etc., are very rare I would say, in particular in the context of sciences (excluding science fiction as it seems). And the CF text before your suggested change does not say anything about it, except for noting that if you want to follow the SI then you should (must?) not use such units. And I think that this a good balance, not least when taking interoperability into account.
But I suggest that we should be more specific in our reference to the SI Brochure; something like:
| In CF, following UDUNITS, any unit may optionally have one of the decimal prefixes for multiples and submultiples (<<table-supported-units, Table 3.1>>) e.g. **`millisecond`** or **`ms`**; however, we note that <<SI>> does not allow these prefixes for any unit of time other than **`second`**. | |
| In CF, following UDUNITS, any unit may optionally have one of the decimal prefixes for multiples and submultiples (<<table-supported-units, Table 3.1>>) e.g. **`millisecond`** or **`ms`**; however, we note that Section 4 of <<SI>> does not allow these prefixes for any unit of time other than **`second`**. |
There was a problem hiding this comment.
Is it too late for us to say that CF does not recommend using pre/suffixes for any time unit other than second?
(sorry I can't think of the "it's allowed but you really shouldn't do it" word)
I'd be happy to completely disallow it, but that's tricky, given backward compatibility and punting to UDUNITS ....
There was a problem hiding this comment.
Thanks Lars, that's all fine by me.
(excluding science fiction as it seems).
Indeed! - I have read various books that like kilohours (something about imposing human-centric expressions of time on exoplanets with very different orbital characteristics, I guess). Since CF is bound to still be about in thousands of years, these may yet come into their own as we spread out ... :-)
|
I would just to say that I've been very impressed with the recent work on this, and am happy with the PR as it stands, bar one comment (https://github.com/cf-convention/cf-conventions/pull/612/files#r2553958552), but it is not a blocker if you want to ignore it :) (That link doesn't seem to get you all the way there - it's on line 241 in Chapter 4 on the sentence "In CF, following UDUNITS, any unit may optionally have one of the decimal prefixes for multiples and submultiples (<<table-supported-units, Table 3.1>>) e.g. |
ch04.adoc
Outdated
| * ±__n__ or ±__nn__, the hour alone, of one or two digits e.g. **`-6`**, **`+2`**, **`+11`**. This format is sufficient for many time zones. Zero hours must have a positive sign e.g. **`+0`**, which means the same as **`Z`**. | ||
|
|
||
| ** __H__:__M__, where _H_ is hour and _M_ minute, each of one or two digits, e.g. **`5:30`**. | ||
| * ±__n__:__nn__ or ±__nn__:__nn__, the hour and minute, separated by a colon **`:`**, where the hour is one or two digits, and the minute is two digits e.g. **-10:00`**, **`-3:30`**, **`+5:30`**, **`+12:45`**. |
There was a problem hiding this comment.
Format fix (added backtick "`" for -10:00):
| * ±__n__:__nn__ or ±__nn__:__nn__, the hour and minute, separated by a colon **`:`**, where the hour is one or two digits, and the minute is two digits e.g. **-10:00`**, **`-3:30`**, **`+5:30`**, **`+12:45`**. | |
| * ±__n__:__nn__ or ±__nn__:__nn__, the hour and minute, separated by a colon **`:`**, where the hour is one or two digits, and the minute is two digits e.g. **`-10:00`**, **`-3:30`**, **`+5:30`**, **`+12:45`**. |
5 participants
See issue #613 for discussion of these changes.
Release checklist
history.adocup to date?