rainfallqc.checks.comparison_checks

Quality control checks for sub-hourly data.

From Sub-hourly QC developed by Roberto Villalobos and described in Villalobos-Herrera et al. (2022).

Code adapted from https://github.com/nclwater/SubHourlyQC/tree/main (GNU GPL v3.0) authored by Roberto Villalobos.

Classes and functions ordered by appearance in SubHourlyQC framework.

rainfallqc.checks.subhourlyqc_checks.check_daily_exceedance_of_UK_24hr_record(data, target_gauge_col)[source]

Check exceedance of UK 24-hour record when aggregating to 24 hours.

Flags: 0 == if doesn’t exceed threshold. Then seperate flags denote when the daily sums exceeds the 24-hour record by: 1 == < 20% 2 == >= 20% 3 == >= 33% 4 == >= 50%

This is HQC_UK24hr_rolling from the SubHourlyQC framework.

Parameters:

  • data (DataFrame) – Rainfall data (15-min or 1-min)

  • target_gauge_col (str) – Column with rainfall data

Return type:

DataFrame

Returns:

:
data_w_flags:

Rainfall data with exceedance of UK 24hr rolling record

rainfallqc.checks.subhourlyqc_checks.check_exceedance_of_UK_1hr_record(data, target_gauge_col)[source]

Check exceedance of UK 1-hour record.

Flags: 0 == if doesn’t exceed threshold. Then seperate flags denote when the data exceeds the 1-hour record by: 1 == < 20% 2 == >= 20% 3 == >= 33% 4 == >= 50%

This is HQC_UK1hr from the SubHourlyQC framework.

Parameters:

  • data (DataFrame) – Rainfall data (15-min or 1-min)

  • target_gauge_col (str) – Column with rainfall data

Return type:

DataFrame

Returns:

:
data_w_flags:

Rainfall data with exceedance of UK 1hr Record

rainfallqc.checks.subhourlyqc_checks.check_exceedance_of_UK_24hr_record(data, target_gauge_col)[source]

Check exceedance of UK 24-hour record.

Flags: 0 == if doesn’t exceed threshold. Then seperate flags denote when the data exceeds the 24-hour record by: 1 == < 20% 2 == >= 20% 3 == >= 33% 4 == >= 50%

This is HQC_UK24hr from the SubHourlyQC framework.

Parameters:

  • data (DataFrame) – Rainfall data (15-min or 1-min)

  • target_gauge_col (str) – Column with rainfall data

Return type:

DataFrame

Returns:

:
data_w_flags:

Rainfall data with exceedance of UK 24hr Record

rainfallqc.checks.subhourlyqc_checks.check_freq_is_subhourly(data, target_gauge_col)[source]

Check frequency and resolution of data is subhourly.

Specifically, it will check monthly periods with frequencies >= 30 minutes, or where the resolution is 1 mm (usually an indicator of tip counts not tip amounts in the data), and replace them with NAN.

Flags: 1 == when month has suspect frequency or resolution.

This is freqResChecker (Function 1 of the SHQC process) from the SubHourlyQC framework.

Parameters:

  • data (DataFrame) – Rainfall data (15-min or 1-min)

  • target_gauge_col (str) – Column with rainfall data

Return type:

DataFrame

Returns:

:
data_w_flags_disag

Rainfall data with flags denoting months where frequency or resolution is suspect

rainfallqc.checks.subhourlyqc_checks.check_streaks_20mm(data, target_gauge_col, flag_col_name='streak_flag_20mm')[source]

Check streaks with fixed minimum hourly threshold of 20 mm.

Flags: 1 == when data has streak of 2 or more timesteps that are more than 20 mm.

This is HQC_streaks_20mm from the SubHourlyQC framework.

Parameters:

  • data (DataFrame) – Rainfall data (15-min or 1-min)

  • target_gauge_col (str) – Column with rainfall data

  • flag_col_name (str) – Name for flag column (default: ‘streak_flag_20mm’)

Return type:

DataFrame

Returns:

:
data_w_flags:

Rainfall data with flags denotting period of repeating streak above 20mm

rainfallqc.checks.subhourlyqc_checks.check_subhourly_thresholds(data, target_gauge_col)[source]

Test hourly, 15-min, 1-min rainfall totals agaisnt thresholds agaisnt values set for each month.

Flags: 1 == when data is suspect.

This is subH_checkr (Function 2 of the SHQC process) from the SubHourlyQC framework.

Parameters:

  • data (DataFrame) – Rainfall data (15-min or 1-min)

  • target_gauge_col (str) – Column with rainfall data

Return type:

DataFrame

Returns:

:
data_w_flags_disag

Rainfall data with flags denoting

rainfallqc.checks.subhourlyqc_checks.flag_data_based_on_threshold(data, target_gauge_col, threshold_dict, threshold_col_name)[source]

Flag data based on thresholds.

Built for SubHourlyQC framework and to be used with inbuilt UK_MONTHLY_THRESHOLDS_n.

Parameters:

  • data (DataFrame) – Rainfall data (15-min or 1-min)

  • target_gauge_col (str) – Column with rainfall data

  • threshold_dict (dict) – given rainfall threshold in mm for each month (e.g. Jan: 10, Feb: 10, Mar: 12)

  • threshold_col_name (str) – Name to use for threshold column and threshold_flag column.

Return type:

DataFrame

Returns:

:
data_w_flags:

Rainfall data with exceedance of given month rainfall threshold

rainfallqc.checks.subhourlyqc_checks.get_subhourly_exceedance_of_given_record(data, target_gauge_col, record_rainfall_amount, flag_col_name)[source]

Check sub-houlry exceedance of given record.

Used in QCX and QCX of the SubHourlyQC framework.

Parameters:

  • data (DataFrame) – Rainfall data (15-min or 1-min)

  • target_gauge_col (str) – Column with rainfall data

  • record_rainfall_amount ([int | float]) – given record rainfall amount

  • flag_col_name (str) – Name for flag column

Return type:

DataFrame

Returns:

:
data_w_flags:

Rainfall data with exceedance of given rainfall record (see flag_exceedance_of_ref_val_as_col function)

Functions

check_exceedance_of_UK_1hr_record(data, ...)

Check exceedance of UK 1-hour record.

check_exceedance_of_UK_24hr_record(data, ...)

Check exceedance of UK 24-hour record.

check_daily_exceedance_of_UK_24hr_record(...)

Check exceedance of UK 24-hour record when aggregating to 24 hours.

check_streaks_20mm(data, target_gauge_col[, ...])

Check streaks with fixed minimum hourly threshold of 20 mm.

check_freq_is_subhourly(data, target_gauge_col)

Check frequency and resolution of data is subhourly.

check_subhourly_thresholds(data, ...)

Test hourly, 15-min, 1-min rainfall totals agaisnt thresholds agaisnt values set for each month.

get_subhourly_exceedance_of_given_record(...)

Check sub-houlry exceedance of given record.

flag_data_based_on_threshold(data, ...)

Flag data based on thresholds.