-
Notifications
You must be signed in to change notification settings - Fork 40
ADD: Function to create dummy QC variable from multiple ancillary QC variables. #964
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
This reverts commit 876763e.
|
Left some comments, but also some PEP8 issues as well. |
|
@AdamTheisen Whenever you have time, this PR and the other should be good for review. |
|
@zssherman I took a look at this one and am wondering if we need to revamp a little based on my comments above. I'll try and find some time to dig in deeper but I want to make sure that it's not going to conflict with any standards when we go to write out files. |
|
@rcjackson just as a heads up, I was playing around with this and I added in some modifications to rely more on the already built functions and I think it's working pretty good. One of the big changes is that I am assuming the first variable in the list of ancillary QC variables is the main one to merge the other QC variables into, so we don't have to deal with dummy variable. It also brings over the test meanings and assessments resulting in slightly different results. If you're okay with it, I'll update things accordingly and commit them to your PR/branch.
|
Fine with me @AdamTheisen |
| to ensure the dataset was updated from ARM QC to the | ||
| correct standardized QC. | ||
| flag_type : boolean | ||
| flag_type : booleany |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added y after boolean that might be on accident
| flag_type : booleany | ||
| Indicating the QC variable uses flag_values instead of | ||
| flag_masks. | ||
| ignore_dims : booleany |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Same here
|
@rcjackson @AdamTheisen Two possible spelling mistakes and some linting errors. |
|
Looks good merging! |


It was noted in #952 that the SMPS datastream has multiple ancillary QC variabiles that are related to QC checks from various machine learning algorithms. Therefore, I have added a function in the
qcfilteraccessorqcfilter.create_dummy_qc_variableto support merging together these QC flags into a dummy QC variable by performing the logical OR on the QC masks.qcfilter.create_dummy_qc_variablefunction supports specifying which ancillary QC flags to use and which tests to include in the mask. In addition, once the dummy variable is created, the various QC display functions and masking function will automatically detect the presence of this dummy variable when it is created.Here is an example of its use:
`ds = act.io.arm.read_arm_netcdf(act.testssample_files.EXAMPLE_SMPS,
cleanup_qc=True)
var_name = 'merged_dN_dlogDp'
title = 'Merged Number Size Distribution'
cbar_title = 'dN/dlogD$_p$ (1/cm$^{3}$)'
`