ppcpy.calibration

ppcpy.calibration.lidarconstant

Testtext for documentation

contains get_best_LC()

ppcpy.calibration.lidarconstant.lc_for_cldFreeGrps(data_cube, retrieval: str, collect_debug: bool = False) → list

Estimate the lidar constant from the optical profiles.

Parameters:

data_cubeobject

Main PicassoProc object.

retrievalstr

Retrieval type. ‘klett’ or ‘raman’.

collect_debugbool

If true, collects debug information.

Returns:

LCslist

Lidar constant for retrieval type per channel per cloud free period.

Notes

• For NR, done directly form the optical profiles, whereas in the matlab version, the LC*olAttri387.sigRatio is taken.

• Through the config variable ‘flagUseRetrievedExt4LCCalc’, the extinction used to calculate the LCs can be specified. if ‘flagUseRetrievedExt4LCCalc’ is True the retrieved extinction will be used otherwise the extinction approximated by the backscatter times the assumed lidar constant will be used.

• Missing Rotational Raman and Aeronet LC retrieval.

Todo

Check if LC’s are normalized with respect to the mean of the profiles.

Todo

Add option for Aeronet and rotational Raman retrieved LC.

History

xxxx-xx-xx: First edition by … 2026-03-18: Changed beta_mol for inelastic wavelengths and added the ‘flagUseRetrievedExt4LCCalc’ variable.

ppcpy.calibration.polarization

ppcpy.calibration.polarization.onemx_onepx(x: float | ndarray) → float | ndarray

Calculate the fraction of (1-x)/(1+x)

ppcpy.calibration.polarization.smooth_signal(signal: ndarray, window_len: int) → ndarray

Uniformly smooth the input signal

Parameters:

singalndarray

Signal to be smooth

window_lenint

Width of the applied uniform filter

Returns:

ndarray

Smoothed signal

Notes

History

• 2026-02-04: Changed from scipy.ndimage.uniform_filter1d to ppcpy.misc.helper.uniform_filter

ppcpy.calibration.polarization.loadGHK(data_cube)

Prepare the GHK parameters, especially if given in TR convert them into GHK.

Parameters:

data_cubeobject

Main PicassoProc object.

ppcpy.calibration.polarization.calibrateGHK(data_cube) → dict

Estimate the polarization calibration from the delta 90 Method [1]

Parameters:

data_cube

the input data cube

Returns:

pol_calidict

polarization factors from delta 90 for each wavelength containing sub-dicts with ‘eta’, ‘eta_std’, ‘time_start’, ‘time_end’, ‘status’

Notes

History

Function is called here PollyNET/Pollynet_Processing_Chain The two most relevant functions here are PollyNET/Pollynet_Processing_Chain which also calls PollyNET/Pollynet_Processing_Chain

References

[1]

Freudenthaler 2016

ppcpy.calibration.polarization.depol_cali_ghk(signal_t: ndarray, bg_t: ndarray, signal_x: ndarray, bg_x: ndarray, time: ndarray, pol_cali_pang_start_time: ndarray, pol_cali_pang_stop_time: ndarray, pol_cali_nang_start_time: ndarray, pol_cali_nang_stop_time: ndarray, K: float, cali_h_indx_range: list | tuple, SNRmin: list, sig_max: list, rel_std_dplus: float, rel_std_dminus: float, segment_len: int, smooth_win: int, collect_debug: bool = False) → dict

Polarization calibration for PollyXT lidar system.

Parameters:

signal_tndarray

Background-removed photon count signal at the total channel. Shape: (n_bins, n_profiles)

bg_tndarray

Background at the total channel. Shape: (n_bins, n_profiles)

signal_xndarray

Background-removed photon count signal at the cross channel. Shape: (n_bins, n_profiles)

bg_xndarray

Background at the cross channel. Shape: (n_bins, n_profiles)

timendarray

Datetime array representing the measurement time of each profile.

pol_cali_pang_start_time, pol_cali_pang_stop_timendarray

Start and stop times when the polarizer rotates to the positive angle.

pol_cali_nang_start_time, pol_cali_nang_stop_timendarray

Start and stop times when the polarizer rotates to the negative angle.

Kfloat

Parameter from GHK to correct the calibration.

cali_h_indx_rangelist or tuple

Range of height indexes to use for polarization calibration.

SNRminlist

Minimum SNR for calibration. Length: 4

sig_maxlist

Maximum signal allowed for calibration to prevent pulse pileup.

rel_std_dplus, rel_std_dminusfloat

Maximum relative uncertainty of dplus and dminus allowed.

segment_lenint

Segment length for testing the variability of calibration results.

smooth_winint

Width of the sliding window for smoothing the signal.

collect_debugbool, default=False

store and return the intermediate results

Returns:

pol_cali_etalist

Eta values from polarization calibration.

pol_cali_eta_stdlist

Uncertainty of eta values from calibration.

pol_cali_start_time, pol_cali_stop_timelist

Start and stop times for successful calibration.

cali_statusint

1 if calibration is successful, 0 otherwise.

global_attridict, optional

Information about the depolarization calibration.

ppcpy.calibration.polarization.analyze_segments(dplus: ndarray, dminus: ndarray, segment_len: int, rel_std_dplus: float, rel_std_dminus: float) → ndarray

…

Parameters:

dplusndarray

…

dminusndarray

…

segment_lenint

Segment length for testing the variability of calibration results.

rel_std_dplus, rel_std_dminusfloat

Maximum relative uncertainty of dplus and dminus allowed.

Returns:

ndarray

…

Notes

Todo

Finish docstring.

ppcpy.calibration.polarization.calibrateMol(data_cube) → dict

Calibrate the polarization with the molecular signal.

Parameters:

data_cubeobject

Main PicassoProc object.

Returns:

dict

…

Notes

• Converted from the matlab code to the best knowledge, but not cross-validated yet

Todo

had to calculate TR_t, TR_c again when calling depol_cali_mol()

Todo

Finish docstring.

ppcpy.calibration.polarization.depol_cali_mol(signal_t: ndarray, background_t: ndarray, signal_c: ndarray, background_c: ndarray, TR_t: float, TR_t_std: float, TR_c: float, TR_c_std: float, minSNR: float, mdr: float, mdrStd: float) → dict

Molecular polarization calibration.

Parameters:

signal_t: numeric

Total signal (photon count).

background_t: numeric

Background at total channel (photon count).

signal_c: numeric

Cross signal (photon count).

background_c: numeric

Background at cross channel (photon count).

TR_t: scalar

Transmission ratio at total channel.

TR_t_std: scalar

Uncertainty of the transmission ratio at total channel.

TR_c: scalar

Transmission ratio at cross channel.

TR_c_std: scalar

Uncertainty of the transmission ratio at cross channel.

minSNR: float

The SNR constraint for the signal strength at reference height.

mdr: float

Default molecular depolarization ratio.

mdrStd: float

Default standard deviation of molecular depolarization ratio.

Returns:

polCaliEta: array

Polarization calibration eta.

polCaliEtaStd: array

Uncertainty of polarization calibration eta.

polCaliFac: array

Polarization calibration factor.

polCaliFacStd: array

Uncertainty of polarization calibration factor.

Notes

History

• 2021-07-06: First edition by Zhenping

• 2024-12-23: converted to python

References

Baars, H., et al., Aerosol profiling with lidar in the Amazon Basin during the wet and dry season, J Geophys Res-Atmos, 117, 10.1029/2012jd018338, 2012.

ppcpy.calibration.rayleighfit

ppcpy.calibration.rayleighfit.getVal(array: ndarray, indices: list) → list

Get values of array for a set of indices, including robustness to NaN indices.

Parameters:

heightndarray

Array to be searched.

indicesarry_like

Indeces of the array, may include NaN values.

Returns:

outlist

Values of the array at given indices. If indx is NaN value is also NaN.

ppcpy.calibration.rayleighfit.rayleighfit(data_cube, collect_debug: bool = False) → list

Perform Rayleighfit procedure using Douglas Peucker algorithm for each channel and cloud free period.

Parameters:

data_cubeobject

Main PicassoProc object.

collect_debugbool

If true, collects debug information.

Returns:

refHlist of dicts

Reference heights per channel per cloud free period.

Notes

• This function uses the Douglas Peucker algorithm to segmentize the signal inside a search range specified by the config variables ‘heightFullOverlap’ and ‘maxDecomHeight’. These segments are considered as potential reference height. The final reference height is chosen as the segment with the best fit to the molecular signal.

• Currently, only the FR reference heights are calculated, while the NR reference heights are taken from the config files.

ppcpy.calibration.rayleighfit.smooth_signal(signal: ndarray, window_len: int) → ndarray

Uniformly smooth the input signal

Parameters:

singalndarray

Signal to be smooth

window_lenint

Width of the applied uniform filter

Returns:

ndarray

Smoothed signal

Notes

History

• 2026-02-04: Changed from scipy.ndimage.uniform_filter1d to ppcpy.misc.helper.uniform_filter

• 2026-04-17: Changed to ppcpy.misc.helper.moving_average to get values at edges.

ppcpy.calibration.rayleighfit.DouglasPeucker(signal: ndarray, height: ndarray, epsilon: float, heightBase: float, heightTop: float, maxHThick: float, window_size: int = 1, showDetails: bool = False) → ndarray

Simplify signal according to Douglas-Peucker algorithm.

Parameters:

signalndarray

Molecule corrected signal. [MHz]

heightndarray

Height. [m]

epsilonfloat

Maximum distance.

heightBasefloat

Minimum height for the algorithm. [m]

heightTopfloat

Maximum height for the algorithm. [m]

maxHThickfloat

Maximum spatial thickness of each segment. [m]

window_sizeint

Size of the average smooth window. Default is 1.

showDetailsbool

If true, print debug information. Default is False.

Returns:

sigIndxarray

Index of the signal that stands for different segments of the signal.

Notes

History

• 2017-12-29: First edition by Zhenping.

• 2018-07-29: Add the height range for the searching instead of SNR restriction.

• 2018-07-31: Add the maxHThick argument to control the maximum thickness of each output segment.

• 2024-12-20: Direct translated from matlab with ai

References

https://en.wikipedia.org/wiki/Ramer%E2%80%93Douglas%E2%80%93Peucker_algorithm

ppcpy.calibration.rayleighfit.DP_algorithm(pointList: list, epsilon: float, maxHThick: float, recursive_depth: int = 0, showDetails: bool = False) → list

Recursive implementation of the Douglas-Peucker algorithm.

Parameters:

pointListlist

List of points [[x1, y1], [x2, y2], …].

epsilonfloat

Maximum distance.

maxHThickfloat

Maximum thickness for each segment.

showDetailsbool

If True, print debug information. Default is False.

Returns:

sigIndxlist

Indices of simplified points.

ppcpy.calibration.rayleighfit.my_dist(pointM: list, pointS: list, pointE: list, full: bool = True) → float

Calculate the perpendicular distance between pointM and the line connecting pointS and pointE.

Parameters:

pointMlist

Middle point [x, y].

pointSlist

Start point [x, y].

pointElist

End point [x, y].

fullbool

If true, calculate the full distance, othrewise calculate only the numerator. Default is True.

Returns:

dfloat

Distance.

Notes

• If pointS and pointE are constant only the numerator is neede to calculate the extreme points with respect to pointM.

ppcpy.calibration.rayleighfit.chi2fit(x: ndarray, y: ndarray, measure_error: ndarray) → tuple

CHI2FIT Chi-2 fitting. All the code are translated from the exemplified code in Numerical Recipes in C (2nd Edition). Great help comes from Birgit Heese.

Parameters:

xndarray

The length of x should be larger than 1.

yndarray

The measured signal.

measure_errorndarray

Measurement errors for the y values.

Returns:

afloat

Intercept of the linear regression.

bfloat

Slope of the linear regression.

sigmaAfloat

Uncertainty of the intercept.

sigmaBfloat

Uncertainty of the slope.

chi2float

Chi-square value.

Qfloat

Goodness of fit.

Notes

History

• 2018-08-03: First edition by Zhenping.

Examples

a, b, sigmaA, sigmaB, chi2, Q = chi2fit(x, y, measure_error)

ppcpy.calibration.rayleighfit.fit_profile(height: ndarray, sig_aer: ndarray, pc: ndarray, bg: ndarray, sig_mol: ndarray, dpIndx: ndarray, layerThickConstrain: float, slopeConstrain: float, SNRConstrain: float, flagShowDetail: bool = False, flagPicassoComparison: bool = False) → tuple

Search the clean region with rayleigh fit algorithm.

Parameters:

heightarray_like

height [m]

sig_aerarray_like

range corrected signal

pcarray_like

photon count signal

bgarray_like

background

sig_molarray_like

range corrected molecular signal

dpIndxarray_like

index of the region calculated by Douglas-Peucker algorithm

layerThickConstrainfloat

constrain for the reference layer thickness [m]

slopeConstrainfloat

constrain for the uncertainty of the regressed extinction coefficient

SNRConstrainfloat

minimum SNR for the signal at the reference height

flagShowDetailbool, optional

If true, calculation information will be printed. Default is False.

flagPicassoComparisonbool, optional

If true, use Picasso values and logic. Default is False.

Returns:

tuple

(hBIndx, hTIndx) - indices of bottom and top of searched region Returns (nan, nan) if region not found

Notes

• There are known numerical discrepancies between this and the Picasso versions especially in the residual calculations due to the subtraction and mean of very small numbers [-1e-24, 1e-24]. these numerical discrepancies may cause a different reference height to be chosen compared to Picasso.

ppcpy.calibration.select

ppcpy.calibration.select.single_best(d: dict, name_val: str, name_min: str, relative: bool = False) → dict

Select the best calibration constant

generalization of lidarconstant.get_best_LC

Parameters:

ddict

dict per channel of list of calibration constants

name_valstr

designator for the actual value

name_minstr

designator of the minimum

relativebool

If true, choose calibration constant based on the relative error.

Returns:

bestdict

Lidar constants/Etas with lowest standard deviation per channel.

Notes

Since LC = LC_stable and LCStd = LC_stable * LC_Std so will any negative LC also have a negative LCStd, and thus be chosen as the best LC.

History

• 2026-02-16: Added additional checks to hinder negative LCs to be chosen.

• 2026-03-27: generalized to also hold for depolarization calibration

ppcpy.calibration.select.plot_cals(d, param, used=None)

plot the calibration constants

Parameters:

ddict

the dict as in data_cube

paramstr

the parameter to extract

useddict

the LCused or etaused (will produce a dashed line in the plot)

Examples

>>> plot_cals(data_cube.pol_cali, 'eta', used=data_cube.etaused)
>>> plot_cals(data_cube.LC, 'LC', used=data_cube.LCused)