Flagging Duplicates/Triplicates in Extracted Daily Data

[jkmusyoka]

When extracting daily data from Climsoft into a CSV file:

• For rainfall, the summary applied is total.
• For temperature, the summary applied is the mean/average.

If there are duplicates/triplicates ... daily values that differ in the database, the extracted data becomes incorrect and leads to misleading summaries or analyses. From my experience, users and even Climsoft admins may unknowingly work with inaccurate data.

It would be helpful to have a way of identifying whether each extracted daily value is based on a single record or a summary of multiple records. This would allow users to flag years/months with multiple values and investigate whether the multiple values are identical or not. Would this be simpler to implement than including a duplicate check as part of quality control checks?

[smachua]

It's actually not a case of duplicates but a configuration issue when Climsoft administrators adjust settings for daily observation hour severally (different countries use different hours hence need for settings in Climsoft). This may result in daily data occurring in different hours for the same day especially if received from different sources e.g. key entry or imports.
This occurrence has been delt with as a QC issue. With Climsoft version 4.2.7, it is resolved through a QC operation where observation hour in daily data is updated according to Climsoft settings. Where different values occur a QC report is produced for manual confirmation.