ppcpy.interface.picassoProc.PicassoProc

class ppcpy.interface.picassoProc.PicassoProc(rawdata_dict, polly_config_dict, picasso_config_dict)

Bases: object

initialize the data_cube

Parameters:

• rawdata_dict – the dict returned by readPollyRawData.readPollyRawData(filename=rawfile)

• polly_config_dict – the configuration specific to the specific polly loadConfigs.loadPollyConfig(polly_config_file_fullname, polly_default_config_file)

• picasso_config_dict – the general picasso config loadConfigs.loadPicassoConfig(args.picasso_config_file,picasso_default_config_file)

Notes

The polly_default_dict is not longer available as a separate variable, but is included into the polly_config_dict

Methods

Angstroem
LidarCalibration	TODO: Add option to read constants from database.
SaturationDetect	Saturation Detection ...
__init__	initialize the data_cube
aggregate_profiles	Aggregate highres profiles over cloud free segments
attBsc_volDepol	highres attBsc and voldepol in 2d
calcDepol
calcMolecular	calculate the molecular scattering for the cloud free periods
check_for_correct_mshots
cloudFreeSeg	PollyNET/Pollynet_Processing_Chain
cloudScreen	PollyNET/Pollynet_Processing_Chain
estQualityMask
filter_or_correct_false_mshots
gf	get flag shorthand
loadAOD	TODO: Not yet implemented!
loadMeteo	Load meteorological data
mdate_consistency
mdate_filename
mdate_infile
molecularHighres
overlapCalc	estimate the overlap function
overlapCor	the overlap correction is implemented differently to the matlab version first a 2d (time, height) correction array is constructed then it is applied.
overlapFixLowestBins	the lowest bins are affected by stange near range effects
polarizationCaliD90	The stuff that starts here in the matlab version PollyNET/Pollynet_Processing_Chain
polarizationCaliMol
preprocessing	Preprocessing of Lidar data. Including in the followin processes in order:
quasiV1	quasiV1 retrivals and target categorisation.
quasiV2	quasiV2 retrivals and target categorisation.
rayleighFit	do the rayleigh fit
reset_date_infile
retrievalKlett
retrievalRaman
setChannelTags	set the channel tags
transCor	TODO: flagTransCor = True: Fix the GHK - Transmission correction flagTransCor = False: Check if it is correct to use the BG corrected signal and find a better solution. It is a bit confusing to overwrite the signal as it is called sigTCor but actually is sigBGCor Like storing a dedicated signal dict to be used throuhot the processing, the dictionary could have elements like signal (sig), background (bg), and name. which we could overwrite each time a new correction is made. And by checking the name of the signal (TCor, BGCor) you can find out which signal it is.
write_2_sql_db	write LC or eta to sqlite db table parameters: - parameter (str): can be LC (Lidar-calibration-constant) or DC (Depol-calibration-constant) - method (str): 'raman' or 'klett' - db_path (str): location of the sqlite db-file

counter = 0

mdate_filename()

gf(wavelength, meth, telescope)

get flag shorthand

i.e., the following two calls are equivalent ` data_cube.flag_532_total_FR data_cube.gf(532, 'total', 'FR') `

where the pattern {wavelength}_{total|cross|parallel|rr}_{NR|FR|DFOV} from PollyNET/Pollynet_Processing_Chain#303 is obeyed

Parameters:

• wavelength – wavelength tag

• meth – method

• telescope – telescope

Returns:

with bool flag

Return type:

array

mdate_infile()

check_for_correct_mshots()

filter_or_correct_false_mshots()

mdate_consistency() → bool

reset_date_infile()

setChannelTags()

set the channel tags

they are stored as dictionary in ` data_cube.channel_dict `

and as list in ` data_cube.retrievals_highres['channel'] data_cube.polly_config_dict['channelTags'] `

as an array of boolean flags in ` data_cube.flags `

as flags per channel in ` data_cube.flag_355_total_FR `

Return type:

self

preprocessing(collect_debug: bool = False)

Preprocessing of Lidar data. Including in the followin processes in order:

Dead time correction Background correction Range correction etc.

Returns:

• Background corrected signal including the background per channel.

• Range corrected signal.

• etc.

SaturationDetect()

Saturation Detection …

polarizationCaliD90()

The stuff that starts here in the matlab version PollyNET/Pollynet_Processing_Chain

cloudScreen()

PollyNET/Pollynet_Processing_Chain

cloudFreeSeg()

PollyNET/Pollynet_Processing_Chain

data_cube.clFreGrps = [
    [35,300],
    [2500,2800]
]

aggregate_profiles(var=None)

Aggregate highres profiles over cloud free segments

TODO: Decide on a consistent way for doing the aggregation, do not mix mean and sum

loadMeteo()

Load meteorological data

loadAOD()

TODO: Not yet implemented!

calcMolecular()

calculate the molecular scattering for the cloud free periods

with the strategy of first averaging the met data and then calculating the rayleigh scattering

rayleighFit()

do the rayleigh fit

direct translation from the matlab code. There might be noticeable numerical discrepancies (especially in the residual) seemed to work ok for 532, 1064, but with issues for 355

polarizationCaliMol()

transCor()

TODO: flagTransCor = True:

Fix the GHK - Transmission correction

flagTransCor = False:

Check if it is correct to use the BG corrected signal and find a better solution. It is a bit confusing to overwrite the signal as it is called sigTCor but actually is sigBGCor Like storing a dedicated signal dict to be used throuhot the processing, the dictionary could have elements like signal (sig), background (bg), and name. which we could overwrite each time a new correction is made. And by checking the name of the signal (TCor, BGCor) you can find out which signal it is.

retrievalKlett(oc=False, nr=False)

retrievalRaman(oc=False, nr=False, collect_debug=False)

overlapCalc(collect_debug=False)

estimate the overlap function

different to the matlab version, where an average over all cloud free periods is taken, it is done here per cloud free segment

overlapFixLowestBins()

the lowest bins are affected by stange near range effects

overlapCor()

the overlap correction is implemented differently to the matlab version first a 2d (time, height) correction array is constructed then it is applied. In future this will allow for time varing overlap functions

calcDepol()

estQualityMask()

Angstroem()

LidarCalibration()

TODO: Add option to read constants from database. TODO: Find out how we prioritise raman, klett, and database retrieved LC…

attBsc_volDepol()

highres attBsc and voldepol in 2d

molecularHighres()

quasiV1()

quasiV1 retrivals and target categorisation.

quasiV2()

quasiV2 retrivals and target categorisation.

write_2_sql_db(db_path: str, parameter: str, method: str | None = None)

write LC or eta to sqlite db table parameters: - parameter (str): can be LC (Lidar-calibration-constant) or DC (Depol-calibration-constant) - method (str): ‘raman’ or ‘klett’ - db_path (str): location of the sqlite db-file