ppcpy.interface

ppcpy.interface.picassoProc

class ppcpy.interface.picassoProc.PicassoProc(rawdata_dict: dict, polly_config_dict: dict, picasso_config_dict: dict)

Initialize the data_cube.

Parameters:

rawdata_dictdict

The dict returned by readPollyRawData.readPollyRawData(filename=rawfile)

polly_config_dictdict

The configuration specific to the specific polly loadConfigs.loadPollyConfig(polly_config_file_fullname, polly_default_config_file)

picasso_config_dictdict

The general picasso config loadConfigs.loadPicassoConfig(args.picasso_config_file,picasso_default_config_file)

Notes

• The polly_default_dict is not longer available as a separate variable, but is included into the polly_config_dict

counter = 0

mdate_filename()

Get the date from filename in YYYYMMDD.

gf(wavelength, meth, telescope)

Get flag shorthand.

i.e., the following two calls are equivalent ` data_cube.flag_532_total_FR data_cube.gf(532, 'total', 'FR') `

where the pattern {wavelength}_{total|cross|parallel|rr}_{NR|FR|DFOV} from PollyNET/Pollynet_Processing_Chain#303 is obeyed

Parameters:

wavelength

wavelength tag

meth

method

telescope

telescope

Returns:

array

with bool flag

mdate_infile()

First date in file as string.

check_for_correct_mshots()

Check if mshots are more than 1.1 * laser_prf * deltatime or smaller 0.

filter_or_correct_false_mshots()

Filter or correct the mshots (currently only logging).

Todo

that might be covered via the mcps conversion

mdate_consistency() → bool

Check mdate consistency.

reset_date_infile()

Correct the date in the file.

setChannelTags()

Set the channel tags.

they are stored as dictionary in ` data_cube.channel_dict `

and as list in ` data_cube.retrievals_highres['channel'] data_cube.polly_config_dict['channelTags'] `

as an array of boolean flags in ` data_cube.flags `

as flags per channel in ` data_cube.flag_355_total_FR `

Returns:

self

preprocessing(collect_debug: bool = False)

Preprocessing of Lidar data.

Including in the followin processes in order:

Dead time correction Background correction Range correction etc.

Parameters:

collect_debugbool

If true, collect debug information. Default is False.

SaturationDetect()

Saturation Detection.

polarizationCaliD90(db_path: str = None)

Calibration with the Delta-90 method.

Parameters:

db_pathstr

Path to database to read DC values from in case none are successfully retrieved. Default is None.

The stuff that starts here in the matlab version

https://github.com/PollyNET/Pollynet_Processing_Chain/blob/5efd7d35596c67ef8672f5948e47d1f9d46ab867/lib/interface/picassoProcV3.m#L442

cloudScreen(collect_debug: bool = False)

Basic cloud screenting.

PollyNET/Pollynet_Processing_Chain

cloudFreeSeg()

Cloud free profile segmentation.

PollyNET/Pollynet_Processing_Chain

data_cube.clFreGrps = [
    [35,300],
    [2500,2800]
]

aggregate_profiles(var: str = None)

Aggregate highres profiles over cloud free segments.

Parameters:

varstr

Name of variables to aggregate. Default is RCS, sigBGCor, and BG.

Notes

Todo

Decide on a consistent way for doing the aggregation, do not mix mean and sum

loadMeteo()

Load meteorological data.

loadAOD()

Load the AOD from a co-located fotometer

Todo

Not yet implemented!

calcMolecular()

Calculate the molecular scattering for the cloud free periods with the strategy of first averaging the met data and then calculating the rayleigh scattering.

rayleighFit(collect_debug: bool = False)

Perform the rayleigh fit procedure.

Parameters:

collect_debugbool

If true, collects debug information. Default is False.

Notes

• Direct translation from the matlab code. There might be noticeable numerical discrepancies (especially in the residual) seemed to work ok for 532, 1064, but with issues for 355.

• The Douglas Peucker algorithm works fine (gives the same result as the Matlab version). However, due the numerical discrepancies In the residuals of the fit algorithm sometimes different refH segments are chosen.

polarizationCaliMol()

Calibration with molecular signal in reference height.

transCor()

GHK-Transmission correction

Todo

flagTransCor = True:

Fix the GHK - Transmission correction

flagTransCor = False:

Check if it is correct to use the BG corrected signal and find a better solution. It is a bit confusing to overwrite the signal as it is called sigTCor but actually is sigBGCor Like storing a dedicated signal dict to be used throuhot the processing, the dictionary could have elements like signal (sig), background (bg), and name. which we could overwrite each time a new correction is made. And by checking the name of the signal (TCor, BGCor) you can find out which signal it is.

retrievalKlett(oc=False, nr=False)

Apply Klett retrieval.

retrievalRaman(oc=False, nr=False, collect_debug=False)

Apply Raman retrieval (nighttime only).

overlapCalc(collect_debug: bool = False)

Estimate the overlap function.

Parameters:

collect_debugbool

If true, collects debug information. Default is False.

Notes

• Different to the matlab version, where an average over all cloud free periods is taken, it is done here per cloud free segment

Todo

The data structure of retrievals_profile[‘overlap’] differs from retrievals_profile[‘raman’]!

overlapFixLowestBins()

The lowest bins are affected by stange near range effects.

overlapCor()

Overlap correction

the overlap correction is implemented differently to the matlab version first a 2d (time, height) correction array is constructed then it is applied. In future this will allow for time varing overlap functions

calcDepol()

Calculate the volume depol and the particle depol.

estQualityMask()

Estimate the quality mask.

Angstroem()

Calculate the angstrom exponent.

LidarCalibration(db_path: str = None, collect_debug: bool = False)

calculate the lidar constant

Todo

Find out how we prioritise raman, klett, and database retrieved LC…

attBsc_volDepol()

highres attBsc and voldepol in 2D.

molecularHighres()

calculate the molecular signal for the 2d high resolution.

quasiV1()

QuasiV1 retrivals and target categorisation.

quasiV2()

QuasiV2 retrivals and target categorisation.

write_2_sql_db(parameter: str, db_path: str | None = None, method: str | None = None)

Write LC or eta to sqlite db table.

Parameters:

parameterstr

can be LC (Lidar-calibration-constant) or DC (Depol-calibration-constant)

methodstr

‘raman’ or ‘klett’

db_pathstr

location of the sqlite db-file

Notes

• The unique columns are needed that new entries overwrite old ones, otherwise they are just added to the table with same timestamps.

read_calibration_db(db_path: str | None = None)

Read the calibration constants from database.

Parameters:

db_pathstr

path to database of calibration values.

Time interval includes 24h before and after the actual date

adding_retrieving_infos_2_polly_config_dict()

Some infos from the polly_config_dict should have there own keys, e.g. reference_search_range.