Add logic for running offline IODA tools.

[delippi]

DESCRIPTION OF CHANGES:

• Added logic to exrrfs_ioda_bufr.sh to run an offline tool to process IODA files. The offline tool adds MetaData/longitude_latitude_pressure to the IODA file which is used as the group category for duplicate checking based on RDASApp#259
• Also cleans up some a few things in the edited file including extra white space.

TESTS CONDUCTED:

Tested on Hera conus 12km retro runs.

ISSUE:

• Duplicate Checking in RDAS RDASApp#258

Dependencies:

• add the offline tool to add lon_lat_ps to IODA and updated each jedi yaml to use that as category variable RDASApp#259
• The above PR needs merged and the submodule hash to RDASApp updated in this repo in order to have access to the offline tool.

[MatthewPyle-NOAA]

Looks good to me, but will wait to hear from others before merging...

[guoqing-noaa]

I think NCO wants the script (including *.py) to run from the command line directly without the leading "python".

[guoqing-noaa]

Also, has the Python environment been correctly loaded for this tool?

[delippi]

@guoqing-noaa, thanks for the comments. The tool already had #!/usr/bin/env python at the top. Its just habit to run it with python anyway. I just removed it in my most recent commit and it still works. As for the python environment it seems everything that is needed is already loaded so I didn't have to load anything else.

[MatthewPyle-NOAA]

@delippi Did you test within the leading python to make sure it all still works? If so, think we can merge it.

[delippi]

@MatthewPyle-NOAA hold off on merging though. There are some dependencies.

Edit: The dependencies are documented in the initial comment. Please don't merge until those are satisfied, otherwise the python tool won't be available. I need some people to review that NOAA-EMC/RDASApp#259.

[delippi]

I also just noticed that this exrrfs_ioda_bufr.sh is not using the most up to date version of the bufr2ioda yamls. I actually have those saved in RDASApp. I think we ought to have those in a single location, but which repo? @hu5970 @MatthewPyle-NOAA @ShunLiu-NOAA

[guoqing-noaa]

Sorry for another comment, could you add ./ before offline_add_var_to_ioda.py?
Not all environments put the current directory in the $PATH variable.
(for example, our Jet role account cron jobs set $PATH to empty)

[guoqing-noaa]

I also just noticed that this exrrfs_ioda_bufr.sh is not using the most up to date version of the bufr2ioda yamls. I actually have those saved in RDASApp. I think we ought to have those in a single location, but which repo? @hu5970 @MatthewPyle-NOAA @ShunLiu-NOAA

@delippi Thanks for noticing this! Yes, we will need to update the bufr2ioda yaml files.

For all PARM files, rrfs-workflow will be self-contained. That means, RDASApp manages its parm/test files while rrfs-workflow adapts a given version from RDASApp. This adheres to the NCO standard generating PARM files on the fly from the PARMrrfs directory and has the benefit that RDASApp can move forward with any new experimental changes without worrying about affecting rrfs-workflow (similar to our previous practice that the gsiparm.* files are in both the GSI repo and the rrfsv1 repo).

[guoqing-noaa]

I would suggest we hold this PR until the discussion in NOAA-EMC/RDASApp#259 is resolved.