ppcpy.io#
ppcpy.io.loadConfigs#
- ppcpy.io.loadConfigs.loadPicassoConfig(picasso_config_file: str, picasso_default_config_file: str) dict[source]#
Load the general Picasso config file.
- Parameters:
- picasso_config_filestr or path
The specific config file.
- picasso_default_config_filstr or path
The default (template) file.
- Returns:
- picasso_config_dictdict
Mereged config file containing all config variables.
- ppcpy.io.loadConfigs.readPollyNetConfigLinkTable(polly_config_table_file: str, timestamp: str, device: str) DataFrame[source]#
Read PollyNet processing chain config link table.
- Parameters:
- polly_config_table_filestr
Path to the link tabele file.
- timestampstr
Date of measurement.
- devicestr
Name of polly device.
- Returns:
- config_arraypd.DataFrame
Row of intrest in the PollyNet processing chain config link table.
- ppcpy.io.loadConfigs.fix_indexing(config_dict: dict, keys: list = ['first_range_gate_indx', 'bgCorRangeIndx', 'bgCorRangeIndxLow', 'bgCorRangeIndxHigh', 'LCMeanMinIndx', 'LCMeanMaxIndx', 'depol_cal_minbin_355', 'depol_cal_minbin_532', 'depol_cal_minbin_1064']) dict[source]#
Fix indexing convention of config variables to 0based.
- Parameters:
- config_dictdict
A dictionary containing all config variables.
- keyslist
Key(s) of config_dict variables to fix indexing convention.
- Returns:
- config_dictdict
config_dict with 0based indexing.
- ppcpy.io.loadConfigs.getPollyConfigfromArray(polly_config_array: DataFrame, picasso_config_dict: dict) dict[source]#
Function to load the config for the time identified.
Aim is to declutter the runscript
- Parameters:
- polly_config_arraypandas dataframe
Selected line form the links.xlsx.
- picasso_config_dictdict
General picasso config.
- Returns:
- polly_config_dictdict
A dictionary with the polly config / default parameters.
- ppcpy.io.loadConfigs.loadPollyConfig(polly_config_file: str, polly_default_config_file: str) dict[source]#
Load the spesific and default config files and combine them into one dictionary.
- Parameters:
- polly_config_filestr
Path to polly config file.
- polly_default_config_filestr
path to polly default file.
- Returns:
- dict
A dictionary with the polly config / default parameters.
Notes
Todo
Maybe change polly_default_config_file to polly_global_config_file as the defualt one is no longer used.
ppcpy.io.readMeteo#
- class ppcpy.io.readMeteo.Meteo(meteorDataSource: str, meteo_folder: str, meteo_file: str)[source]#
Initialise Meteo object.
- Parameters:
- meteorDataSourcestr
Meteorological data source name eg. “nc_cloudnet”.
- meteo_folderstr
Path to meteorological data folder.
- meteo_filestr
Meteorological data file name.
Notes
** History ** - 2021-05-22: First edition by Zhenping.
- load(times: float, heights: ndarray, asl: float, flagPicassoComparison: bool = False)[source]#
Load the data and resample to 15 minute intervals.
- Parameters:
- timesfloat
Date of measurement [Unix time].
- heightsnp.ndarray
Height above ground [m].
- aslfloat
Altitude of measurement site (station) [m.a.s.l.].
- flagPicassoComparisonbool
If true, use Picasso values and logic.
- class ppcpy.io.readMeteo.MeteoNcCloudnet(basepath, filepattern)[source]#
TODO for now only one filename define preferred model
Initialize MeteoNcCloudnet object.
- Parameters:
- basepathstr
Path to meteorological data folder.
- filepatternstr
Meteorological data file name.
Notes
** History ** - xxxx-xx-xx: First edition by …
- radiusEarth: float = 6371200.0#
- acelerationEarth: float = 9.80665#
- find_path_for_time(time: float) str[source]#
Find the files for a given time.
- Returns:
- str
Path to the file for the given time.
- load(time: float, height_grid: ndarray, station_altitude: float, flagPicassoComparison: bool = False) Dataset[source]#
Load the data.
- Parameters:
- timefloat
Date of measurement [Unix time].
- height_gridfloat
Height above ground [m].
- station_altitudefloat
Altitude of measurement site (station) [m.a.s.l.].
- flagPicassoComparisonbool
If True, use Picasso values and logic.
- Returns:
- ds_newDataset
Dataset of meteorology profiles (temperature, pressure, relative humidity, specific humidity).
Notes
not quite sure on the interface yet
` met.load(data_cube.retrievals_highres['time'][0]) met.load(datetime.datetime.timestamp(datetime.datetime.strptime(data_cube.date, '%Y%m%d'))) `- Recipie:
load
select variables?
rename?
regrid from (time, level) to (time, lidar heights)
Todo
clarify the above ground above sea level issues.
Todo
Check how the interpolation handles negative grid points
Todo
Check if we can interpolate several variables at once
ppcpy.io.readPollyRawData#
- ppcpy.io.readPollyRawData.readPollyRawData(filename: str) dict[source]#
Read Polly raw data.
- Parameters:
- filenamestr
Absolute path of the polly data.
- Returns:
- data_dictdict
- rawSignalndarray (channel x height x time)
backscatter signal [Photon Count].
- mShotsndarray
number of the laser shots for each profile.
- flagValidProfilendarray
flag to represent the validity of each signal profile.
- mTimendarray
datetime array for the measurement time of each profile.
- depCalAngndarray
angle of the polarizer in the receiving channel. (>0 means calibration process starts) [degree].
- zenithAngnumeric???
zenith angle of the laser beam [degree].
- repRatefloat
laser pulse repetition rate [s^-1].
- hResfloat
spatial resolution [m].
- mSitestr
measurement site.
- deadtimendarray (channel x polynomial_orders)
deadtime correction parameters.
- latfloat
latitude of measurement site [degree].
- lonfloat
longitude of measurement site [degree].
- altfloat
altitude of measurement site [degree].
- filenameStartTimedatenum??????
start time extracted from filename.
Notes
Todo
Go over the Returns section in the docsting. Are these variables actuly returned by the function?
Deleat the matlab code underneath.S
** HISTORY **
2024-03-21: First edition by Andi Klamt.
xxxx-xx-xx: Translated to python.
ppcpy.io.write2nc#
- ppcpy.io.write2nc.get_git_info(path: str = '.') tuple[source]#
…
- Parameters:
- pathstr
Path to …
- Returns:
- branch… or None
…
- commit… or None
…
Notes
Todo
Finish docstring!
- ppcpy.io.write2nc.date_splitting(timestamp: str) tuple[source]#
Split date into YYYY, MM, DD.
- Parameters:
- timestampstr
Date in format ‘YYYYMMDD’.
- Returns:
- YYYYstr
Year.
- MMstr
Month.
- DDstr
Day.
- ppcpy.io.write2nc.adding_fixed_vars(data_cube, json_nc_mapping_dict: dict)[source]#
Add fixed variables to mapping dict.
- Parameters:
- data_cubeobject
Main PicassoProc object.
- json_nc_mapping_dictdict
…
Notes
Todo
Finish docstring!
- ppcpy.io.write2nc.adding_global_attr(data_cube, json_nc_mapping_dict: dict)[source]#
Add global attributes to mapping dict.
- Parameters:
- data_cubeobject
Main PicassoProc object.
- json_nc_mapping_dictdict
…
Notes
Todo
Finish docstring!
- ppcpy.io.write2nc.write_channelwise_2_nc_file(data_cube, root_dir: str = PosixPath('/mnt/c/Users/radenz/dev/PicassoPy/PicassoPy'), prod_ls: list = [])[source]#
…
- Parameters:
- data_cubeobject
Main PicassoProc object.
- root_dirstr
Path to root directory.
- prod_lslist of str
List of product names.
Notes
Todo
Finish docstring!
- ppcpy.io.write2nc.write2nc_file(data_cube, root_dir: str = PosixPath('/mnt/c/Users/radenz/dev/PicassoPy/PicassoPy'), prod_ls: list = [])[source]#
…
- Parameters:
- data_cubeobject
Main PicassoProc object.
- root_dirstr
Path to root directory.
- prod_lslist of str
List of product names.
Notes
Todo
Finish docstring!
- ppcpy.io.write2nc.write_profile2nc_file(data_cube, root_dir: str = PosixPath('/mnt/c/Users/radenz/dev/PicassoPy/PicassoPy'), prod_ls: list = [], collect_debug: bool = False)[source]#
Saving profile data to NetCDF4 files.
- Parameters:
- data_cubeobject
Main PicassoProc object.
- root_dirstr
Path to root directory.
- prod_lslist
List of product names.
Notes
Todo
Missing comment in variable attributes.
Not all retrievals / information needed for the profiles are in data_cube.retrivals_highres…
write docstring
- ppcpy.io.write2nc.adding_mol_profiles(data_cube, json_nc_mapping_dict: dict, cldFreeGrp: int) dict[source]#
Temporarily quick fix for adding molecular profiles as variables to the NetCDF profile outputs.
- Parameters:
- data_cubeobject
Main PicassoProc object.
- json_nc_mapping_dictdict
…
- cldFreeGrpint
Index of floud free segment.
Notes
Todo
Finish docstring.
ppcpy.io.sql_interaction#
- ppcpy.io.sql_interaction.string_to_ts(s)[source]#
String of format %Y-%m-%d %H:%M:%S to timestamp (timezone-aware)
- ppcpy.io.sql_interaction.get_from_sql_db(db_path: str, table_name: str, ts_interval: list[str], ts_delta: float = 24) dict[source]#
Read lidar calibration constant or depol calibration from database.
- Parameters:
- db_pathstr
Name of the specific sqlite db file.
- table_namestr
Default ‘lidar_calibration_constant’.
- ts_intervalstr
The date or timestamp to look for.
- Returns:
- dict
in calibration storage format
- History
- 2026-03-28: First edition by Radenz.
- 2026-05-08: Added check for table existence. Added retrieval methods.
- 2026-11-01: Gently tried to make it more pythonic
- ppcpy.io.sql_interaction.prepare_for_sql_db_writing(data_cube, parameter: str, method: str) list[tuple][source]#
Collect all necessary variable and save it to a list of tuples for inserting into a SQLite table.
- Parameters:
- data_cubeobject
Main PicassoProc object
- parameter :str
Name of parameter to stoere eg. ‘LC’ or ‘DC’.
- methodstr
Name of retrieval method used to retrieve the parameter (see METHOD_DB_NAMES) eg. ‘klett’ or ‘raman’.
- Returns:
- rows_to_insertlist of tuples
Rows to insert in the database.
- ** History **
- xxxx-xx-xx: First edition by
- xxxx-xx-xx: Translated to python.
- 2026-11-01: Gently tried to make it more pythonic
- ppcpy.io.sql_interaction.setup_empty(db_path: str, table_name: str, column_names: list[str], data_types: list[str], unique: str = '')[source]#
Create/Initialise an empty database.
- Parameters:
- db_pathstr
Path to the SQLite database file.
- table_namestr
Name of the target table.
- column_nameslist of str
List of column names to insert values into (e.g. [‘col1’, ‘col2’]).
- data_typeslist of str
List of SQLite data types for each respective columns (e.g. [‘text’, ‘real’])
- ppcpy.io.sql_interaction.write_rows_to_sql_db(db_path: str, table_name: str, column_names: list[str], rows_to_insert: list[str])[source]#
Insert multiple rows into a SQLite table.
- Parameters:
- db_pathstr
Path to the SQLite database file.
- table_namestr
Name of the target table.
- column_nameslist of str
List of column names to insert values into (e.g. [‘col1’, ‘col2’]).
- rows_to_insertlist of tuples
Data to insert, e.g. [(‘a’, ‘b’), (‘c’, ‘d’)].
Notes
The IGNORE syntax somehow did not work. With the UNIQUE colums defined and INSERT OR REPLACE at least the new values are updated. Though they are given a new ID.