ppcpy.misc#
ppcpy.misc.helper#
- ppcpy.misc.helper.detect_path_type(fullpath)[source]#
Detect the type of path (Windows or Linux) based on the input.
- ppcpy.misc.helper.get_pollyxt_files(timestamp, device, raw_folder, output_path)[source]#
This function locates multiple pollyxt level0 nc-zip files from one day measurements, unzipps the files to output_path and returns a list of files to be merged and the title of the new merged nc-file
- ppcpy.misc.helper.get_pollyxt_logbook_files(timestamp, device, raw_folder, output_path)[source]#
This function locates multiple pollyxt logbook-zip files from one day measurements, unzipps the files to output_path and merge them to one file
- ppcpy.misc.helper.checking_attr(timestamp, device, raw_folder, output_path)[source]#
…
- Parameters:
- timestamp…
…
- device…
…
- raw_folder…
…
- output_path…
…
- Returns:
- …
Todo
Variables ‘force’ and ‘polly_files_list’ are not defined anywhere ..
- ppcpy.misc.helper.remove_whitespaces_and_replace_dash_with_underscore(string: str) str[source]#
remove whitespaces and replace dashes with underscores
- Parameters:
- stringstr
String to be modified
- Returns:
- new_stringstr
Modified string
- ppcpy.misc.helper.find_matching_dimension(array, reference_list)[source]#
Finds the dimension of a 3D array that matches the length of the reference list.
- Parameters:
- array (np.ndarray): The 3D NumPy array to check.
- reference_list (list): The list to compare the dimension lengths with. This can also be a dict.
- Returns:
- int: The index of the matching dimension, or -1 if no match is found.
- ppcpy.misc.helper.uniform_filter(x: ndarray, win: int, fill_val: float = nan) ndarray[source]#
Uniform smoothing filter.
The smoothing is applied without padding and the original dimension of the input is recreated by filling the reduced edges with a fill value.
- Parameters:
- x(N,) ndarray
One dimensional input signal.
- winint
(M,) Width of the filter.
- fill_valfloat, optional
Value to be used for filling edges, in order to recreate the input dimension. Default is np.nan.
- Returns:
- outndarray
Smoothed signal.
Notes
Currently only support smoothing of 1D-arrays with a constant window size.
History
2026-02-02: First edition by Buholdt
- ppcpy.misc.helper.mean_stable(x: ndarray, win: int, minBin: int = None, maxBin: int = None, minRelStd: float = None) tuple[source]#
Calculate the mean value of x based on the least fluctuated segment of x. The searching is based on the std inside each window of x.
- Parameters:
- xndarray
Signal array.
- winint
Window width for calculating the relative standard deviation.
- minBinint, optional
The start index for the mean calculation (default: 1).
- maxBinint, optional
The end index for the mean calculation (default: length of x).
- minRelStdfloat, optional
Minimum relative standard deviation threshold.
- Returns:
- xStablefloat
Stable mean value.
- xIndxndarray
Index of the elements used to calculate the mean value.
- xRelStdfloat
Relative uncertainty of the sequences used to calculate the mean values.
Notes
History
2021-05-30: First edition by Zhenping
2026-02-04: Changed from scipy.ndimage.uniform_filter1d to uniform_filter
- ppcpy.misc.helper.smooth2a(matrix_in: ndarray, Nr: int, Nc: int = None) ndarray[source]#
Smooths 2D array data while ignoring NaNs.
This function smooths the data in
matrix_inusing a mean filter over a rectangle of size(2*Nr+1)-by-(2*Nc+1). Each element is replaced by the mean of the surrounding rectangle, ignoring NaN elements. If an element is NaN, it remains NaN in the output. At the edges, as much of the rectangle as fits is used.- Parameters:
- matrix_inndarray
Original matrix to be smoothed.
- Nrint
Number of points used to smooth rows.
- Ncint, optional
Number of points used to smooth columns. If not specified, Nc = Nr.
- Returns:
- matrix_outndarray
Smoothed version of the input matrix.
References
Written by Greg Reeves, March 2009, Division of Biology, Caltech.
Inspired by “smooth2” by Kelly Hilands, October 2004, Applied Research Laboratory, Penn State University.
Developed from code by Olof Liungman, 1997, Dept. of Oceanography, Earth Sciences Centre, Göteborg University.
- ppcpy.misc.helper.get_wv_pol_telescope_from_dictkeyname(keyname: str) tuple[source]#
translate {wavelength}_{total|cross|parallel|rr}_{NR|FR|DFOV} to wavelength, polarisation, telescope separataly.
- Parameters:
- keynamestr
e.g. 532_total_FR.
- Returns:
- wavelengthstr
eg. ‘532’
- polarisationstr
eg. ‘total’
- telescopestr
eg. ‘FR’
- ppcpy.misc.helper.idx2time(cldFreeIdx: ndarray[int, int], nIdx: int, nHour: int) str[source]#
Convert cloud free indecis to cloud free times.
- Parameters:
- cldFreeIdxnp.ndarray
Cloud free group (start idx, end idx).
- nIdxint
Number of idx / length of time dimension.
- nHourint
Number of hours represented in the time dimension.
- Returns:
- outstr
Time stamps of cldFreeIdx as a string (eg. 0920_1020)
ppcpy.misc.json2nc_mapping#
- ppcpy.misc.json2nc_mapping.read_json_to_dict(file_path)[source]#
Reads in an existing json-file and outputs a dict-structure
- ppcpy.misc.json2nc_mapping.create_netcdf_from_dict(nc_file_path: str, data_cube, data_dict: dict, compression_level: int, prod: str, cldFreeIndx: int = None)[source]#
Creates a NetCDF file from a structured dictionary.
- Parameters:
- nc_file_pathstr
Path to the NetCDF file to create.
- data_cubeobject
the data-object of class PicassoProc.
- data_dictdict
Dictionary with keys ‘global_attributes’, ‘dimensions’, and ‘variables’.
- compression_levelint
a nc-compressionlevel of 1 is a good reference.
- prodstr
e.g. profile, OC_profile, NR_profile, …
- cldFreeIndxint, optional
If the product to be saved are optical profiles; index of the cloud free region for the saved profiles. Else; None. Default is None.
Examples
Example of data_dict structure:
{ "global_attributes": { "title": "Example NetCDF File", "institution": "My Organization" }, "dimensions": { "time": None, // Unlimited dimension "lat": 10, "lon": 20 }, "variables": { "temperature": { "dimensions": ("time", "lat", "lon"), "dtype": "float32", "attributes": { "units": "K", "long_name": "Surface temperature" }, "data": np.random.rand(5, 10, 20) // Example data }, "pressure": { "dimensions": ("time", "lat", "lon"), "dtype": "float32", "attributes": { "units": "Pa", "long_name": "Surface pressure" }, "data": np.random.rand(5, 10, 20) // Example data } } }
- ppcpy.misc.json2nc_mapping.add_variable_2_json_dict_mapper(data_dict, new_key, reference_key, new_data=None, new_attributes=None)[source]#
Adds a new variable to the ‘variables’ section of the given dictionary.
- Parameters:
data_dict (dict): The original dictionary structure. reference_key (str): The name of the existing variable to reference to (template for new key/variable) new_key (str): The name of the new variable to add. new_data (np.ndarray, optional): The data for the new variable. new_attributes (dict, optional): Additional or updated attributes for the new variable.
- ppcpy.misc.json2nc_mapping.remove_variable_from_json_dict_mapper(data_dict, key_to_remove)[source]#
Removes a specific variable from the ‘variables’ section of the given dictionary.
- Parameters:
data_dict (dict): The original dictionary structure. key_to_remove (str): The name of the variable to remove.
- ppcpy.misc.json2nc_mapping.update_variable_attribute_of_json_dict_mapper(data_dict, variable_key, attribute_key, new_value)[source]#
Updates the value of a specific attribute for a specified variable in the data dictionary.
- Parameters:
data_dict (dict): The dictionary containing the variables and attributes. variable_key (str): The key of the variable to update. attribute_key (str): The key of the attribute to update. new_value (any): The new value to set for the attribute.
ppcpy.misc.molecular#
- ppcpy.misc.molecular.air_refractive_index(wavelength, pressure, temperature, C, relative_humidity)[source]#
Calculate the refractive index of air.
- Parameters:
- wavelength: Wavelength [nm]
- pressure: Atmospheric pressure [hPa]
- temperature: Atmospheric temperature [K]
- C: CO2 concentration [ppmv]
- relative_humidity: Relative humidity [%]
- Returns: Refractive index of air.
- ppcpy.misc.molecular.moist_air_density(pressure, temperature, C, Xw)[source]#
Calculate the density of moist air.
- Parameters:
- pressure: Total pressure [hPa]
- temperature: Temperature [K]
- C: CO2 concentration [ppmv]
- Xw: Molar fraction of water vapor
- ppcpy.misc.molecular.molar_mass_dry_air(C)[source]#
Molar mass of dry air as a function of CO2 concentration.
- Parameters:
- C: CO2 concentration [ppmv]
- Returns:
- Molar mass of dry air [kg/mol]
- ppcpy.misc.molecular.compressibility_of_moist_air(pressure, temperature, molar_fraction)[source]#
Compressibility of moist air.
- Parameters:
- pressure: Total pressure [hPa]
- temperature: Temperature [K]
- molar_fraction: Molar fraction of water vapor
- ppcpy.misc.molecular.n_standard_air(wavelength)[source]#
Calculate the refractive index of standard air at a given wavelength.
- Parameters:
- wavelength (float): Wavelength [nm].
- Returns:
- float: Refractive index of standard air.
- ppcpy.misc.molecular.n_standard_air_with_CO2(wavelength, C)[source]#
Calculate the refractive index of air at a specific wavelength with CO2 concentration.
- Parameters:
- wavelength (float): Wavelength [nm].
- C (float): CO2 concentration [ppmv].
- Returns:
- float: Refractive index of air for the given CO2 concentration.
- ppcpy.misc.molecular.n_water_vapor(wavelength)[source]#
Calculate the refractive index of water vapor.
- Parameters:
- wavelength (float): Wavelength [nm].
- Returns:
- float: Refractive index of water vapor.
- ppcpy.misc.molecular.alpha_rayleigh(wavelength, pressure, temperature, C, rh)[source]#
Cacluate the extinction coefficient for Rayleigh scattering.
- Parameters:
- wavelengthfloat or array of floats
Wavelegnth [nm]
- pressurefloat or array of floats
Atmospheric pressure [hPa]
- temperaturefloat
Atmospheric temperature [K]
- Cfloat
CO2 concentration [ppmv].
- rhfloat
Relative humidity from 0 to 100 [%]
- Returns:
- alpha: float
The molecular scattering coefficient [m-1]
- ppcpy.misc.molecular.beta_pi_rayleigh(wavelength, pressure, temperature, C, rh)[source]#
Calculates the total Rayleigh backscatter coefficient.
- Parameters:
- wavelength: float
Wavelength [nm]
- pressure: float
The atmospheric pressure [hPa]
- temperature: float
The atmospheric temperature [K]
- C: float
CO2 concentration [ppmv].
- rh: float
Relative humidity from 0 to 100 [%]
- Returns:
- beta_pi: array
molecule backscatter coefficient. [m^{-1}Sr^{-1}]
- ppcpy.misc.molecular.dsigma_phi_rayleigh(theta, wavelength, pressure, temperature, C, rh)[source]#
Calculates the angular rayleigh scattering cross section per molecule.
- Parameters:
- theta: float
Scattering angle [rads]
- wavelength: float
Wavelength [nm]
- pressure: float
The atmospheric pressure [hPa]
- temperature: float
The atmospheric temperature [K]
- C: float
CO2 concentration [ppmv].
- rh: float
Relative humidity from 0 to 100 [%]
- Returns:
- dsigma: float
rayleigh-scattering cross section [m2sr-1]
- ppcpy.misc.molecular.enhancement_factor_f(pressure, temperature)[source]#
Enhancement factor.
- Parameters:
- pressure: float
Atmospheric pressure [hPa]
- temperature: float
Atmospehric temperature [K]
- ppcpy.misc.molecular.kings_factor_atmosphere(wavelength, C, p_e, p_t)[source]#
calculate the king factor.
- Parameters:
- wavelength: float
Unit: nm
- C: float
CO2 concentration in ppmv
- p_e: float
water vapor pressure in hPa
- p_t: float
total air pressure in hPa
- Returns:
- k: float
total atmospheric King’s factor
References
https://bitbucket.org/iannis_b/lidar_molecular
Examples
k = king_factor_atmosphere(wavelength, C, p_e, p_t)
- ppcpy.misc.molecular.kings_factor_N2(wavenumber)[source]#
- approximates the King’s correction factor for a specific wavenumber.
According to Bates, the agreement with experimental values is “rather better than 1 per cent.”
- Parameters:
- wavenumberfloat
- Wavenumber (defined as 1/lamda) in cm-1
- Returns:
- Fkfloat
- Kings factor for N2
Notes
The King’s factor is estimated as
\[F_{N_2} = 1.034 + 3.17 \cdot 10^{-4} \cdot \lambda^{-2}\]where \(\lambda\) is the wavelength in micrometers.
References
Tomasi, C., Vitale, V., Petkov, B., Lupi, A. & Cacciari, A. Improved algorithm for calculations of Rayleigh-scattering optical depth in standard atmospheres. Applied Optics 44, 3320 (2005). Bates, D. R.: Rayleigh scattering by air, Planetary and Space Science, 32(6), 785-790, doi:10.1016/0032-0633(84)90102-8, 1984.
- ppcpy.misc.molecular.molar_fraction_water_vapour(pressure, temperature, relative_humidity)[source]#
Molar fraction of water vapor.
- Parameters:
- pressure: float
Total pressure [hPa]
- temperature: float
Atmospehric temperature [K]
- relative_humidity:
Relative humidity from 0 to 100 [%]
- ppcpy.misc.molecular.number_density_at_pt(pressure, temperature, relative_humidity, ideal)[source]#
Calculate the number density for a given temperature and pressure, taking into account the compressibility of air.
- Parameters:
- pressure: float or array
Pressure in hPa
- temperature: float or array
Temperature in K
- relative_humidity: float or array (?)
? The relative humidity of air (Check)
- ideal: boolean
If False, the compressibility of air is considered. If True, the compressibility is set to 1.
- Returns:
- n: array or array
Number density of the atmosphere [m^{-3}]
- ppcpy.misc.molecular.phase_function(theta, wavelength, pressure, temperature, C, rh)[source]#
Calculates the phase function at an angle theta for a specific wavelegth.
- Parameters:
- theta: float
Scattering angle [rads]
- wavelength: float
Wavelength [nm]
- pressure: float
The atmospheric pressure [hPa]
- temperature: float
The atmospheric temperature [K]
- C: float
CO2 concentration [ppmv].
- rh: float
Relative humidity from 0 to 100 [%]
- Returns:
- p: float
Scattering phase function
Notes
The formula is derived from Bucholtz (1995). A different formula is given in Miles (2001).
The use of this formula insetad of the wavelenght independent 3/4(1+cos(th)**2) improves the results for back and forward scatterring by ~1.5%
Anthony Bucholtz, “Rayleigh-scattering calculations for the terrestrial atmosphere”, Applied Optics 34, no. 15 (May 20, 1995): 2765-2773.
R. B Miles, W. R Lempert, and J. N Forkey, “Laser Rayleigh scattering”, Measurement Science and Technology 12 (2001): R33-R51
- ppcpy.misc.molecular.pressure_to_rh(partial_pressure, temperature)[source]#
Convert water vapour partial pressure to relative humidity.
- Parameters:
- partial_pressure (float): Water vapour partial pressure [hPa]
- temperature (float): Temperature [K]
- Returns:
- float: Relative humidity from 0 to 100 [%]
- ppcpy.misc.molecular.rh_to_pressure(rh, temperature)[source]#
Convert relative humidity to water vapour partial pressure.
- Parameters:
- rh (float): Relative humidity from 0 to 100 [%]
- temperature (float): Temperature [K]
- Returns:
- float: Water vapour pressure [hPa]
- ppcpy.misc.molecular.rho_atmosphere(wavelength, C, p_e, p_t)[source]#
Calculate the depolarization factor of the atmosphere.
- Parameters:
- wavelength (float or array): Wavelength in nm
- C (float): CO2 concentration in ppmv
- p_e (float): water-vapor pressure [hPa]
- p_t (float): total air pressure [hPa]
- Returns:
- float or array: Depolarization factor
- ppcpy.misc.molecular.saturation_vapor_pressure(temperature)[source]#
Saturation vapor pressure of water of moist air.
- Parameters:
- temperature (float): Atmospheric temperature [K]
- Returns:
- float: Saturation vapor pressure [hPa]
- ppcpy.misc.molecular.sigma_rayleigh(wavelength, pressure, temperature, C, rh)[source]#
Calculates the Rayleigh-scattering cross section per molecule.
- Parameters:
- wavelength (float): Wavelength [nm]
- pressure (float): The atmospheric pressure [hPa]
- temperature (float): The atmospheric temperature [K]
- C (float): CO2 concentration [ppmv]
- rh (float): Relative humidity from 0 to 100 [%]
- Returns:
- float: Rayleigh-scattering cross section [m2]
- ppcpy.misc.molecular.rayleigh_scattering(wavelength, pressure, temperature, C, rh)[source]#
Calculate the molecular volume backscatter coefficient and extinction coefficient.
- Parameters:
- wavelengthfloat
Wavelength in nanometers [nm].
- pressurefloat
Atmospheric pressure [hPa].
- temperaturefloat
Atmospheric temperature [K].
- Cfloat
CO2 concentration [ppmv].
- rhfloat
Relative humidity as a percentage (0 to 100).
- Returns:
- beta_molfloat
Molecular backscatter coefficient [m^{-1}*sr^{-1}].
- alpha_molfloat
Molecular extinction coefficient [m^{-1}].
Notes
History
First edition by Zhenping, 2017-12-16.
Based on the Python source code of Ioannis Binietoglou’s [repo](https://bitbucket.org/iannis_b/lidar_molecular).
AI-Translated, 2024-12-03
References
Bucholtz, A.: Rayleigh-scattering calculations for the terrestrial atmosphere, Appl. Opt. 34, 2765-2773 (1995). A. Behrendt and T. Nakamura, “Calculation of the calibration constant of polarization lidar and its dependency on atmospheric temperature,” Opt. Express, vol. 10, no. 16, pp. 805-817, 2002.