synthesis_workflow.validation

Functions for validation of synthesis to be used by luigi tasks.

Attributes

VacuumCircuit

AtlasCircuit

SYNTH_MORPHOLOGY_PATH

Functions

convert_circuit_to_morphs_df(circuit_path, ...[, ext])

Convert the list of morphologies from circuit to morphology dataframe.

get_features_df(morphologies_mtypes, features_config)

Create a feature dataframe from a dictionary of morphology_folders per mtypes.

_get_features_df_all_mtypes(morphs_df, ...[, n_workers])

Wrapper for morph-validator functions.

get_feature_configs([config_types])

Getter function of default features configs.

_expand_lists(data)

Convert list element of dataframe to duplicated rows with float values.

_normalize(data)

Normalize data with mean and std.

plot_violin_features(features, neurite_types, ...[, ...])

Create violin plots from features dataframe.

plot_morphometrics(base_morphs_df, comp_morphs_df, ...)

Plot morphometrics.

iter_positions(morph, sample_distance[, neurite_filter])

Iterator for positions in space of points every <sample_distance> um.

sample_morph_voxel_values(morphology, sample_distance, ...)

Sample the values of the neurites in the given voxeldata.

_get_depths_df(circuit, mtype, sample, voxeldata, ...)

Create dataframe with depths data for violin plots.

_get_vacuum_depths_df(circuit, mtype)

Create dataframe with depths data for violin plots.

_plot_layers(x_pos, atlas, ax)

Plot the layers at position x.

_plot_density_profile(mtype[, circuit, x_pos, sample, ...])

Plot density profile of an mtype.

_spherical_filter(radius)

relative_depth_volume(atlas[, top_layer, ...])

Return volumetric data of relative cortical depth at voxel centers.

plot_density_profiles(circuit, sample, region, ...[, ...])

Plot density profiles for all mtypes.

_generate_synthetic_random_population(dir_path, nb, ...)

Generate a synthetic population with random projections.

_get_fit_population(mtype, files, outlier_percentage, ...)

Get projections and path lengths of a given and a synthetic population.

plot_path_distance_fits(tmd_parameters_path, ...[, ...])

Plot path-distance fits.

get_debug_data(log_file)

Parse log file and return a DataFrame with data.

plot_scale_statistics(mtypes, scale_data, cols[, ...])

Plot collage of an mtype and a list of planes.

mvs_score(data1, data2[, percentile])

Get the MED - MVS score.

get_scores(df1, df2[, percentile])

Return scores between two data sets.

compute_scores(ref, test, config)

Compute scores of a test population against a reference population.

plot_score_matrix(ref_morphs_df, test_morphs_df, ...)

Plot score matrix for a test population against a reference population.

extract_angle_data(df, morph_key[, pia])

Extract all pairwise angles between neurite_types from the morphologies in df.

_get_hist(data[, bins])

Return density histogram with bin centers.

trunk_validation(morphs_df, synth_morphs_df, ...)

Create plots to validate trunk angles.

Module Contents

VacuumCircuit
AtlasCircuit
SYNTH_MORPHOLOGY_PATH = 'synth_morphology_path'
convert_circuit_to_morphs_df(circuit_path, synth_output_path, ext='asc')

Convert the list of morphologies from circuit to morphology dataframe.

Parameters:

  • circuit_path (str) – path to circuit file

  • synth_output_path (str) – path to morphology files

  • ext (str) – extension of morphology files

Returns:

morphology dataframe

Return type:

DataFrame

get_features_df(morphologies_mtypes: Dict, features_config: Dict, n_workers: int = 1)

Create a feature dataframe from a dictionary of morphology_folders per mtypes.

Parameters:

  • morphologies_mtypes (dict) – dict of morphology_folders files per mtype

  • features_config (dict) – configuration dict for features extraction (see neurom.apps.morph_stats.extract_dataframe)

  • n_workers (int) – number of workers for feature extractions

_get_features_df_all_mtypes(morphs_df, features_config, morphology_path, n_workers=None)

Wrapper for morph-validator functions.

get_feature_configs(config_types='default')

Getter function of default features configs.

Currently available config_types:

  • default

  • repair

  • synthesis

Parameters:

config_types (list/str) – list of types of config files

_expand_lists(data)

Convert list element of dataframe to duplicated rows with float values.

_normalize(data)

Normalize data with mean and std.

plot_violin_features(features: pandas.DataFrame, neurite_types: List, output_dir: pathlib.Path, bw: float, normalize=True)

Create violin plots from features dataframe.

Parameters:

  • features (pandas.DataFrame) – features dataframe to plot

  • neurite_types (list) – list of neurite types to plot (one plot per neurite_type)

  • output_dir (Path) – path to folder for saving plots

  • bw (float) – resolution of violins

  • normalize (bool) – normalize feature values with mean/std

plot_morphometrics(base_morphs_df, comp_morphs_df, output_path, base_key='morphology_path', comp_key=SYNTH_MORPHOLOGY_PATH, base_label='base', comp_label='comp', normalize=False, config_features=None, n_workers=None)

Plot morphometrics.

Parameters:

  • base_morphs_df (DataFrame) – base morphologies

  • comp_morphs_df (DataFrame) – compared morphologies

  • output_path (str) – path to save figures

  • base_key (str) – column name in the DF

  • comp_key (str) – column name in the DF

  • base_label (str) – label for the base morphologies

  • comp_label (str) – label for the compared morphologies

  • normalize (bool) – normalize data if set to True

  • config_features (dict) – mapping of features to plot

  • n_workers (int) – the number of workers used to compute morphology features

iter_positions(morph, sample_distance, neurite_filter=None)

Iterator for positions in space of points every <sample_distance> um.

Assumption about segment linearity is that curvature between the start and end of segments is negligible.

Parameters:

  • morph (neurom.FstNeuron) – morphology

  • sample_distance (int) – points sampling distance (in um)

  • neurite_filter – filter neurites, see neurite_filter of neurom.iter_sections()

Yields:

sampled points for the neurites (each point is a (3,) numpy array).

sample_morph_voxel_values(morphology, sample_distance, voxel_data, out_of_bounds_value, neurite_types=None)

Sample the values of the neurites in the given voxeldata.

The value is out_of_bounds_value if the neurite is outside the voxeldata.

Parameters:

  • morphology (neurom.FstNeuron) – cell morphology

  • sample_distance (int in um) – sampling distance for neurite points

  • voxel_data (voxcell.VoxelData) – volumetric data to extract values from

  • out_of_bounds_value – value to assign to neurites outside of voxeldata

  • neurite_types (list) – list of neurite types, or None (will use basal and axon)

Returns:

dict mapping each neurite type of the morphology to the sampled values {(neurom.NeuriteType): np.array(…)}

_get_depths_df(circuit, mtype, sample, voxeldata, sample_distance)

Create dataframe with depths data for violin plots.

_get_vacuum_depths_df(circuit, mtype)

Create dataframe with depths data for violin plots.

_plot_layers(x_pos, atlas, ax)

Plot the layers at position x.

_plot_density_profile(mtype, circuit=None, x_pos=None, sample=None, voxeldata=None, sample_distance=None)

Plot density profile of an mtype.

_spherical_filter(radius)
relative_depth_volume(atlas, top_layer='1', bottom_layer='6', in_region='Isocortex', relative=True)

Return volumetric data of relative cortical depth at voxel centers.

The relative cortical depth is equal to <distance from pia> / <total_cortex_thickness>. Outside of the region in_region relative depth will be estimated, i.e. extrapolated from the internal relative depth. The in_region is the region within which to use the relative depth-values outside this region are estimated.

plot_density_profiles(circuit, sample, region, sample_distance, output_path, nb_jobs=-1)

Plot density profiles for all mtypes.

WIP function, waiting on complete atlas to update.

_generate_synthetic_random_population(dir_path, nb, proj_min, proj_max, tmd_parameters, tmd_distributions)

Generate a synthetic population with random projections.

_get_fit_population(mtype, files, outlier_percentage, tmd_parameters, tmd_distributions, neurite_type='apical_dendrite')

Get projections and path lengths of a given and a synthetic population.

plot_path_distance_fits(tmd_parameters_path, tmd_distributions_path, morphs_df_path, morphology_path, output_path, mtypes=None, region=None, outlier_percentage=90, nb_jobs=-1, neurite_type='apical_dendrite')

Plot path-distance fits.

get_debug_data(log_file)

Parse log file and return a DataFrame with data.

plot_scale_statistics(mtypes, scale_data, cols, output_dir='scales', dpi=100)

Plot collage of an mtype and a list of planes.

Parameters:

  • mtypes (list) – mtypes of cells to plot

  • scale_data (pd.DataFrame) – DataFrame with scale data

  • cols (list) – The column names that should be plotted from the data

  • output_dir (str) – result directory

  • dpi (int) – resolution of the output image

mvs_score(data1, data2, percentile=10)

Get the MED - MVS score.

The MED - MVS is equal to the absolute difference between the median of the population and the median of the neuron divided by the maximum visible spread.

Parameters:

  • data1 (list) – the first data set.

  • data2 (list) – the second data set.

  • percentile (int) – percentile to compute.

get_scores(df1, df2, percentile=5)

Return scores between two data sets.

Parameters:
Returns:

The list of feature scores.

compute_scores(ref, test, config)

Compute scores of a test population against a reference population.

Parameters:

  • ref (tuple(str, list)) – the reference data.

  • test (tuple(str, list)) – the test data.

  • config (dict) – the configuration used to compute the scores.

Returns:

The scores and the feature list.

plot_score_matrix(ref_morphs_df, test_morphs_df, output_path, config, mtypes=None, path_col='filepath', dpi=100, nb_jobs=-1)

Plot score matrix for a test population against a reference population.

extract_angle_data(df, morph_key, pia=None)

Extract all pairwise angles between neurite_types from the morphologies in df.

_get_hist(data, bins=50)

Return density histogram with bin centers.

trunk_validation(morphs_df, synth_morphs_df, output_dir, base_key, comp_key, base_label, comp_label, tmd_parameters_path, tmd_distributions_path, region)

Create plots to validate trunk angles.