ppcpy.io#

ppcpy.io.loadConfigs#

ppcpy.io.loadConfigs.loadPicassoConfig(picasso_config_file: str, picasso_default_config_file: str) → dict[source]#

Load the general Picasso config file.

Parameters:
picasso_config_filestr or path

The specific config file.

picasso_default_config_filstr or path

The default (template) file.

Returns:
picasso_config_dictdict

Mereged config file containing all config variables.

ppcpy.io.loadConfigs.readPollyNetConfigLinkTable(polly_config_table_file: str, timestamp: str, device: str) → DataFrame[source]#

Read PollyNet processing chain config link table.

Parameters:
polly_config_table_filestr

Path to the link tabele file.

timestampstr

Date of measurement.

devicestr

Name of polly device.

Returns:
config_arraypd.DataFrame

Row of intrest in the PollyNet processing chain config link table.

ppcpy.io.loadConfigs.fix_indexing(config_dict: dict, keys: list = ['first_range_gate_indx', 'bgCorRangeIndx', 'bgCorRangeIndxLow', 'bgCorRangeIndxHigh', 'LCMeanMinIndx', 'LCMeanMaxIndx', 'depol_cal_minbin_355', 'depol_cal_minbin_532', 'depol_cal_minbin_1064']) → dict[source]#

Fix indexing convention of config variables to 0based.

Parameters:
config_dictdict

A dictionary containing all config variables.

keyslist

Key(s) of config_dict variables to fix indexing convention.

Returns:
config_dictdict

config_dict with 0based indexing.

ppcpy.io.loadConfigs.getPollyConfigfromArray(polly_config_array: DataFrame, picasso_config_dict: dict) → dict[source]#

Function to load the config for the time identified.

Aim is to declutter the runscript

Parameters:
polly_config_arraypandas dataframe

Selected line form the links.xlsx.

picasso_config_dictdict

General picasso config.

Returns:
polly_config_dictdict

A dictionary with the polly config / default parameters.

ppcpy.io.loadConfigs.loadPollyConfig(polly_config_file: str, polly_default_config_file: str) → dict[source]#

Load the spesific and default config files and combine them into one dictionary.

Parameters:
polly_config_filestr

Path to polly config file.

polly_default_config_filestr

path to polly default file.

Returns:
dict

A dictionary with the polly config / default parameters.

Notes

Todo

  • Maybe change polly_default_config_file to polly_global_config_file as the defualt one is no longer used.

ppcpy.io.loadConfigs.checkPollyConfigDict(polly_config_dict: dict) → dict[source]#

Check and potentially modify polly config dict.

Parameters:
polly_config_dictdict

Polly config dict to be checked.

Returns:
new_polly_config_dictdict

Checked (and modified) polly config dict.

ppcpy.io.readMeteo#

class ppcpy.io.readMeteo.Meteo(meteorDataSource: str, meteo_folder: str, meteo_file: str)[source]#

Initialise Meteo object.

Parameters:
meteorDataSourcestr

Meteorological data source name eg. “nc_cloudnet”.

meteo_folderstr

Path to meteorological data folder.

meteo_filestr

Meteorological data file name.

Notes

** History ** - 2021-05-22: First edition by Zhenping.

load(times: float, heights: ndarray, asl: float, flagPicassoComparison: bool = False)[source]#

Load the data and resample to 15 minute intervals.

Parameters:
timesfloat

Date of measurement [Unix time].

heightsnp.ndarray

Height above ground [m].

aslfloat

Altitude of measurement site (station) [m.a.s.l.].

flagPicassoComparisonbool

If true, use Picasso values and logic.

get_mean_profiles(time_slice: list) → list[source]#

Get the mean meteorological profiles.

Parameters:
time_slicelist

Nested list of time slices in np.datetime [[start time, end time], ..]

Returns:
mean_profileslist

Mean meteorological profiles for each time slice.

class ppcpy.io.readMeteo.MeteoNcCloudnet(basepath, filepattern)[source]#

TODO for now only one filename define preferred model

Initialize MeteoNcCloudnet object.

Parameters:
basepathstr

Path to meteorological data folder.

filepatternstr

Meteorological data file name.

Notes

** History ** - xxxx-xx-xx: First edition by …

radiusEarth: float = 6371200.0#
acelerationEarth: float = 9.80665#
find_path_for_time(time: float) → str[source]#

Find the files for a given time.

Returns:
str

Path to the file for the given time.

load(time: float, height_grid: ndarray, station_altitude: float, flagPicassoComparison: bool = False) → Dataset[source]#

Load the data.

Parameters:
timefloat

Date of measurement [Unix time].

height_gridfloat

Height above ground [m].

station_altitudefloat

Altitude of measurement site (station) [m.a.s.l.].

flagPicassoComparisonbool

If True, use Picasso values and logic.

Returns:
ds_newDataset

Dataset of meteorology profiles (temperature, pressure, relative humidity, specific humidity).

Notes

not quite sure on the interface yet ` met.load(data_cube.retrievals_highres['time'][0]) met.load(datetime.datetime.timestamp(datetime.datetime.strptime(data_cube.date, '%Y%m%d'))) `

Recipie:
  • load

  • select variables?

  • rename?

  • regrid from (time, level) to (time, lidar heights)

Todo

clarify the above ground above sea level issues.

Todo

Check how the interpolation handles negative grid points

Todo

Check if we can interpolate several variables at once

ppcpy.io.readPollyRawData#

ppcpy.io.readPollyRawData.readPollyRawData(filename: str) → dict[source]#

Read Polly raw data.

Parameters:
filenamestr

Absolute path of the polly data.

Returns:
data_dictdict
rawSignalndarray (channel x height x time)

backscatter signal [Photon Count].

mShotsndarray

number of the laser shots for each profile.

flagValidProfilendarray

flag to represent the validity of each signal profile.

mTimendarray

datetime array for the measurement time of each profile.

depCalAngndarray

angle of the polarizer in the receiving channel. (>0 means calibration process starts) [degree].

zenithAngnumeric???

zenith angle of the laser beam [degree].

repRatefloat

laser pulse repetition rate [s^-1].

hResfloat

spatial resolution [m].

mSitestr

measurement site.

deadtimendarray (channel x polynomial_orders)

deadtime correction parameters.

latfloat

latitude of measurement site [degree].

lonfloat

longitude of measurement site [degree].

altfloat

altitude of measurement site [degree].

filenameStartTimedatenum??????

start time extracted from filename.

Notes

Todo

  • Go over the Returns section in the docsting. Are these variables actuly returned by the function?

  • Deleat the matlab code underneath.S

** HISTORY **

  • 2024-03-21: First edition by Andi Klamt.

  • xxxx-xx-xx: Translated to python.

ppcpy.io.write2nc#

ppcpy.io.write2nc.get_git_info(path: str = '.') → tuple[source]#

…

Parameters:
pathstr

Path to …

Returns:
branch… or None

…

commit… or None

…

Notes

Todo

Finish docstring!

ppcpy.io.write2nc.date_splitting(timestamp: str) → tuple[source]#

Split date into YYYY, MM, DD.

Parameters:
timestampstr

Date in format ‘YYYYMMDD’.

Returns:
YYYYstr

Year.

MMstr

Month.

DDstr

Day.

ppcpy.io.write2nc.adding_fixed_vars(data_cube, json_nc_mapping_dict: dict)[source]#

Add fixed variables to mapping dict.

Parameters:
data_cubeobject

Main PicassoProc object.

json_nc_mapping_dictdict

…

Notes

Todo

Finish docstring!

ppcpy.io.write2nc.adding_global_attr(data_cube, json_nc_mapping_dict: dict)[source]#

Add global attributes to mapping dict.

Parameters:
data_cubeobject

Main PicassoProc object.

json_nc_mapping_dictdict

…

Notes

Todo

Finish docstring!

ppcpy.io.write2nc.write_channelwise_2_nc_file(data_cube, root_dir: str = PosixPath('/mnt/c/Users/radenz/dev/PicassoPy/PicassoPy'), prod_ls: list = [])[source]#

…

Parameters:
data_cubeobject

Main PicassoProc object.

root_dirstr

Path to root directory.

prod_lslist of str

List of product names.

Notes

Todo

Finish docstring!

ppcpy.io.write2nc.write2nc_file(data_cube, root_dir: str = PosixPath('/mnt/c/Users/radenz/dev/PicassoPy/PicassoPy'), prod_ls: list = [])[source]#

…

Parameters:
data_cubeobject

Main PicassoProc object.

root_dirstr

Path to root directory.

prod_lslist of str

List of product names.

Notes

Todo

Finish docstring!

ppcpy.io.write2nc.write_profile2nc_file(data_cube, root_dir: str = PosixPath('/mnt/c/Users/radenz/dev/PicassoPy/PicassoPy'), prod_ls: list = [], collect_debug: bool = False)[source]#

Saving profile data to NetCDF4 files.

Parameters:
data_cubeobject

Main PicassoProc object.

root_dirstr

Path to root directory.

prod_lslist

List of product names.

Notes

Todo

  • Missing comment in variable attributes.

  • Not all retrievals / information needed for the profiles are in data_cube.retrivals_highres…

  • write docstring

ppcpy.io.write2nc.adding_mol_profiles(data_cube, json_nc_mapping_dict: dict, cldFreeGrp: int) → dict[source]#

Temporarily quick fix for adding molecular profiles as variables to the NetCDF profile outputs.

Parameters:
data_cubeobject

Main PicassoProc object.

json_nc_mapping_dictdict

…

cldFreeGrpint

Index of floud free segment.

Notes

Todo

Finish docstring.

ppcpy.io.sql_interaction#

ppcpy.io.sql_interaction.string_to_ts(s)[source]#

String of format %Y-%m-%d %H:%M:%S to timestamp (timezone-aware)

ppcpy.io.sql_interaction.get_from_sql_db(db_path: str, table_name: str, ts_interval: list[str], ts_delta: float = 24) → dict[source]#

Read lidar calibration constant or depol calibration from database.

Parameters:
db_pathstr

Name of the specific sqlite db file.

table_namestr

Default ‘lidar_calibration_constant’.

ts_intervalstr

The date or timestamp to look for.

Returns:
dict

in calibration storage format

History
  • 2026-03-28: First edition by Radenz.
  • 2026-05-08: Added check for table existence. Added retrieval methods.
  • 2026-11-01: Gently tried to make it more pythonic
ppcpy.io.sql_interaction.prepare_for_sql_db_writing(data_cube, parameter: str, method: str) → list[tuple][source]#

Collect all necessary variable and save it to a list of tuples for inserting into a SQLite table.

Parameters:
data_cubeobject

Main PicassoProc object

parameter :str

Name of parameter to stoere eg. ‘LC’ or ‘DC’.

methodstr

Name of retrieval method used to retrieve the parameter (see METHOD_DB_NAMES) eg. ‘klett’ or ‘raman’.

Returns:
rows_to_insertlist of tuples

Rows to insert in the database.

** History **
  • xxxx-xx-xx: First edition by
  • xxxx-xx-xx: Translated to python.
  • 2026-11-01: Gently tried to make it more pythonic
ppcpy.io.sql_interaction.setup_empty(db_path: str, table_name: str, column_names: list[str], data_types: list[str], unique: str = '')[source]#

Create/Initialise an empty database.

Parameters:
db_pathstr

Path to the SQLite database file.

table_namestr

Name of the target table.

column_nameslist of str

List of column names to insert values into (e.g. [‘col1’, ‘col2’]).

data_typeslist of str

List of SQLite data types for each respective columns (e.g. [‘text’, ‘real’])

ppcpy.io.sql_interaction.write_rows_to_sql_db(db_path: str, table_name: str, column_names: list[str], rows_to_insert: list[str])[source]#

Insert multiple rows into a SQLite table.

Parameters:
db_pathstr

Path to the SQLite database file.

table_namestr

Name of the target table.

column_nameslist of str

List of column names to insert values into (e.g. [‘col1’, ‘col2’]).

rows_to_insertlist of tuples

Data to insert, e.g. [(‘a’, ‘b’), (‘c’, ‘d’)].

Notes

  • The IGNORE syntax somehow did not work. With the UNIQUE colums defined and INSERT OR REPLACE at least the new values are updated. Though they are given a new ID.