openhsv.analysis package¶

Module contents¶

Submodules¶

openhsv.analysis.nn module¶

class openhsv.analysis.nn.Analysis(app=None)¶

Bases: PyQt5.QtWidgets.QWidget

Analysis widget that shows the segmentation process of the neural network.

Parameters:	QWidget (PyQt5.QtWidgets.QWidget) – Inherits from QWidget app (PyQt5.QtWidgets.QWidget, optional) – QApplication, needed to process events to avoid freezing of the GUI, defaults to None

segmentSequence(ims, normalize=True, reinit=True)¶

segments an image sequence, such as a video, frame by frame.

Parameters:	ims (list of numpy.ndarray, or numpy.ndarray) – collection of images normalize (bool, optional) – normalize 0..255 to -1..1, defaults to True reinit (bool, optional) – deletes any previous segmentation information, defaults to True

segment(im)¶

Segments an endoscopic image using a deep neural network

Parameters:	im – np.ndarray (HxWx3)
Returns:

setAudio(audio)¶

syncAudio(start_frame, end_frame, total_frames, debug=False)¶

computeParameters(dt_audio=1.25e-05, dt_video=0.00025, debug=False)¶

Compute parameters from GAW

Parameters:	dt_audio (float, optional) – audio sampling time in seconds, defaults to 1/80000 dt_video (float, optional) – video sampling time in seconds, defaults to 1/4000 debug (bool, optional) – shows debugging information and plots, defaults to False

get()¶

returns GAW and segmentation maps for video

Returns:	GAW and segmentations
Return type:	tuple(list, list(numpy.ndarray))

class openhsv.analysis.nn.ROIDialog(ims)¶: Bases: PyQt5.QtWidgets.QDialog

openhsv.analysis.parameters module¶

openhsv.analysis.parameters.movingAverage¶

computes moving average from x with a window of n

Parameters:	x (numpy.ndarray) – input array n (int, optional) – window, defaults to 3
Returns:	filtered array
Return type:	numpy.ndarray

openhsv.analysis.parameters.detectMaximaMinima(s, distance=5, rel_height=0.01, use_prominence=True, clean_f0=None)¶

Detect maxima and minima from a signal s.

Parameters:	s (numpy.ndarray) – signal distance (int, optional) – distance between two peaks in samples, defaults to 5 rel_height (float, optional) – minimum relative height of a found peak, defaults to .35 use_prominence (bool, optional) – uses peak prominence for peak detection, defaults to True
Returns:	tuple of maxima and minima locations
Return type:	tuple(numpy.ndarray, numpy.ndarray)

openhsv.analysis.parameters.detectOpeningAndClosingEvents(signal, p_max, t=0.02)¶

Detects glottis opening and closing events relative to maximum opening events.

Parameters:	signal (numpy.ndarray) – glottal area waveform p_max (list of indexes) – maxima of glottal area waveform t (float, optional) – threshold for finding signal bottom, defaults to 0.02
Returns:	opening and closing point for each maximum
Return type:	tuple(list(int), list(int))

openhsv.analysis.parameters.computeOpenAndClosedIntervals(t, opening, closed)¶

computes the opened and closed intervals during each cycle.

Parameters:	t (numpy.ndarray) – time opening (list(int)) – indices of opening points closed (list(int)) – indices of closing points
Returns:	duration of opening and closed phases
Return type:	tuple(list(float), list(float))

openhsv.analysis.parameters.computeOCandCOTransitions(t, opening, closed, p_max)¶

Computes Open->Closed (OC) and Closed-Open (CO) transitions.

Parameters:	t (numpy.ndarray) – time opening (list(int)) – indices of opening points closed (list(int)) – indices of closing points p_max (numpy.ndarray) – indices of cycle maxima
Returns:	CO and OC durations
Return type:	tuple(numpy.ndarray, numpy.ndarray)

openhsv.analysis.parameters.F0fromCycles(T, verbose=False, epsilon=1e-09)¶

determine fundamental frequency (F0) based on period lengths

Parameters:	T (numpy.ndarray) – periods verbose (bool, optional) – prints F0 mean and standard deviation, defaults to False
Returns:	mean of F0 and std of F0
Return type:	tuple(float, float)

openhsv.analysis.parameters.F0fromFFT(fft, freqs, freq_lower=75, freq_higher=500)¶

fundamental frequency from power spectrum determined assuming that fundamental frequency is equal to the dominant frequency. Frequency is determined in physiological range (75-500 Hz).

Parameters:	fft (numpy.ndarray) – fast fourier transformation of signal freqs (numpy.ndarray) – frequencies corresponding to FFT freq_lower (int, optional) – lower border for F0 prediction, defaults to 75 freq_higher (int, optional) – higher border for F0 prediction, defaults to 500
Returns:	fundamental frequency (F0)
Return type:	float

openhsv.analysis.parameters.F0fromAutocorrelation(signal, freq=40000)¶

Fundamental frequency using autocorrelation.

Steps:

rFFT of signal
Corresponding frequencies
Autocorrelation
Find peaks in autocorrelation
Remove first unrelevant peaks
Use first peak as fundamental frequency

Parameters:	signal (numpy.ndarray) – Signal (e.g. audio trace or GAW) freq (int) – sampling frequency, defaults to 40000 Hz
Returns:	fundamental frequency (F0)
Return type:	float

openhsv.analysis.parameters.asymmetryQuotient(CO, OC)¶

Asymmetry Quotient (AsyQ)

\[\text{AsyQ} = \frac{1}{N} \sum_{i=1}^{N} \frac{\frac{CO_i}{OC_i}}{1+\frac{CO_i}{OC_i}}\]

Parameters:	CO (numpy.ndarray) – Closed->Open transitions OC (numpy.ndarray) – Open->Closed transitions
Returns:	asymmetry quotient (AQ), a.u.
Return type:	float

openhsv.analysis.parameters.closingQuotient(CO, t_open)¶

Closing Quotient (CQ)

\[\text{CQ} = \frac{1}{N} \sum_{i=1}^{N} \frac{CO_i}{t_{open,i}}\]

Parameters:	CO (numpy.ndarray) – Closed->Open transitions t_open – Open interval
Returns:	closing quotient (CQ), a.u.
Return type:	float

openhsv.analysis.parameters.openQuotient(t_open, t_closed)¶

Open Quotient (OQ)

\[\text{OQ} = \frac{1}{N} \sum_{i=1}^{N} \frac{t_{open,i}}{t_{open,i}+t_{closed,i}}\]

Parameters:	t_open (numpy.ndarray) – Open intervals t_closed (numpy.ndarray) – Closed intervals
Returns:	open quotient (OQ), a.u.
Return type:	float

openhsv.analysis.parameters.rateQuotient(CO, OC, t_closed)¶

Rate Quotient (RQ)

\[\text{RQ} = \frac{1}{N} \sum_{i=1}^{N} \frac{t_{closed,i} - CO_i}{OC_i}\]

Parameters:	CO (numpy.ndarray) – Closed->Open transitions OC (numpy.ndarray) – Open->Closed transitions t_closed (numpy.ndarray) – closed intervals
Returns:	rate quotient (RQ), a.u.
Return type:	float

openhsv.analysis.parameters.speedIndex(CO, OC, t_open)¶

Speed Index (SI)

\[\text{SI} = \frac{1}{N} \sum_{i=1}^{N} \frac{CO_i - OC_i}{t_{open,i}}\]

Parameters:	CO (numpy.ndarray) – Closed->Open transitions OC (numpy.ndarray) – Open->Closed transitions t_open (numpy.ndarray) – open intervals
Returns:	speed index (SI), a.u.
Return type:	float

openhsv.analysis.parameters.speedQuotient(CO, OC)¶

Speed Quotient (SQ)

\[\text{SQ} = \frac{1}{N} \sum_{i=1}^{N} \frac{CO_i}{OC_i}\]

Parameters:	CO (numpy.ndarray) – Closed->Open transitions OC (numpy.ndarray) – Open->Closed transitions
Returns:	speed quotient (SQ), a.u.
Return type:	float

openhsv.analysis.parameters.meanJitter(T)¶

Calculating the mean jitter in ms from signal periods

\[\text{mean-Jitter} = \frac{\sum_{i=1}^{N-1}|T_i - T_{i-1}|}{N-1}\]

Parameters:	T (numpy.ndarray or list) – The signal periods
Returns:	mean jitter in ms
Return type:	float

openhsv.analysis.parameters.jitterPercent(T)¶

Calculating the jitter in percent from signal periods.

\[\text{Jitter[%]} = \frac{\frac{1}{N-1}\sum_{i=1}^{N-1}|T_i - T_{i-1}|}{\frac{1}{N}\sum_{i=0}^{N-1}T_i} \cdot 100.\]

Parameters:	T (numpy.ndarray or list) – The signal periods
Returns:	jitter in percent
Return type:	float

openhsv.analysis.parameters.meanShimmer(A, epsilon=1e-09)¶

Calculating the mean shimmer in dB from the signal amplitude maxima.

\[\text{mean-Shimmer [db]} = \frac{20}{N-1}\sum_{i=0}^{N-2}|\log_{10} \left[ \frac{A_i}{A_{i+1}} \right]|.\]

Parameters:	A (numpy.ndarray or list) – The signal amplitude maxima
Returns:	mean shimmer in db
Return type:	float

openhsv.analysis.parameters.shimmerPercent(A, e=1e-05)¶

Calculating the shimmer in percent from the signal amplitude maxima.

\[\text{Shimmer[%]} = \frac{\frac{20}{N-1}\sum_{i=0}^{N-2}|\log_{10} \left[ \frac{A_i}{A_{i+1}} \right]|}{\frac{20}{N}\sum_{i=0}^{N-1}|\log_{10} A_i|} \cdot 100.\]

Parameters:	A (numpy.ndarray or list) – The signal amplitude maxima
Returns:	shimmer in percent
Return type:	float

openhsv.analysis.parameters.periodPerturbationFactor(T)¶

Calculating the Period Perturbation Factor (PPF) in arbitrary units using the signal periods.

\[\text{PPF} = \frac{1}{N-1} \sum_{i=1}^{N-1}|\frac{T_i-T_{i-1}}{T_i}| \cdot 100\]

Parameters:	T (list) – periods
Returns:	PPF in percent
Return type:	float

openhsv.analysis.parameters.glottalGapIndex(signal, opening, epsilon=1e-09)¶

Glottal Gap Index (GGI) that computes the relation between minimum and maximum glottal area in each glottal cycle.

\[GGI = \frac{1}{N} \sum_i^N \frac{\min(a_i)}{\max(a_i)}\]

Parameters:	signal (numpy.ndarray) – glottal area waveform opening (list) – points where the glottis opens epsilon (float, optional) – numerical stability, defaults to 1e-9
Returns:	glottal gap index mean and std
Return type:	tuple(float, float)

openhsv.analysis.parameters.amplitudePerturbationFactor(A)¶

Amplitude perturbation factor.

\[APF = \frac{1}{N} \sum_{i=1}^{N-1}|\frac{A_i-A_{i-1}}{A_i}| \cdot 100\]

Parameters:	A (list or numpy.ndarray) – amplitudes in each cycle
Returns:	APF mean and std
Return type:	tuple(float, float)

openhsv.analysis.parameters.amplitudePerturbationQuotient(A, k=3)¶

Amplitude perturbation quotient

\[l = \frac{k-1}{2}\]

\[APQ_k = \frac{1}{N-k} \sum_{i=l}^{N-l-1}|1-\frac{k \cdot A_i}{\sum_{j=-l}^{l}A_{i+j}}| \cdot 100\]

Parameters:	A (list or numpy.ndarray) – amplitudes in each cycle k (int, optional) – range, defaults to 3
Returns:	APQ mean and std
Return type:	tuple(float, float)

openhsv.analysis.parameters.amplitudeQuotient(signal, opening)¶

Amplitude Quotient

\[AQ = \frac{A_i}{|\min_j \frac{d}{dj}f_i(j)|}\]

Parameters:	signal (numpy.ndarray) – audio or GAW signal opening (list) – indices where glottis starts to open
Returns:	AQ mean and std
Return type:	tuple(float, float)

openhsv.analysis.parameters.stiffness(signal, opening)¶

Stiffness

\[Stiffness = \frac{|\max_j |\frac{d}{dj}f_i(j)||}{A_i}\]

Parameters:	signal (numpy.ndarray) – audio or GAW signal opening (list) – indices where glottis starts to open
Returns:	stiffness mean and std
Return type:	tuple(float, float)

openhsv.analysis.parameters.harmonicNoiseRatio(signal, freq, freq_lower=50, freq_higher=450, filter_autocorrelation=False, epsilon=1e-09)¶

Computes Harmonic-Noise-Ratio (HNR) using autocorrelation approximation. First, it computes the fundamental frequency using the power spectrum of signal. Next, it computes the autocorrelation in Fourier space. Then, local maxima in the autocorrelation are found, the HNR computed and the maximum HNR and the corresponding frequency is returned.

\[R_{xx} = \frac{1}{N} \sum_{k=l}^{N-1} x[k]x[k-l]\]

\[HNR = \frac{R_{xx}[T_0]}{R_{xx}[0]-R_{xx}[T_0]}\]

Parameters:	signal (numpy.ndarray) – audio signal freq (int) – sampling rate/frequency, e.g. 44100 freq_lower (int, optional) – lower frequency cut-off, defaults to 50 freq_higher (int, optional) – higher frequency cut-off, defaults to 350
Returns:	HNR [dB], F0_FFT [Hz], F0_Autocorr [Hz]
Return type:	tuple(float, float, float)

openhsv.analysis.parameters.cepstralPeakProminence(signal, freq, freq_lower=70, freq_higher=350, plot=False)¶

Computes cepstral peak prominence from signal using Fourier transformations.

Steps:

Compute FFT from signal
Compute fundamental frequency from power spectrum
Compute cepstrum from FFT, filter with moving average (window = 3)
Find maximum peak in cepstrum
Find corresponding quefrency
Fit line to cepstrum
Compute distance from peak to line –> Cepstral Peak Prominence

Parameters:	signal (numpy.ndarray) – audio signal freq (int) – sampling rate/frequency, e.g. 44100 freq_lower (int, optional) – lower frequency cut-off, defaults to 70 freq_higher (int, optional) – higher frequency cut-off, defaults to 350 plot (bool, optional) – plots the cepstrum, the line and the peak prominence, defaults to False
Returns:	CPP [dB], F0_FFT [Hz], F0_Cepstrum [Hz]
Return type:	tuple(float, float, float)

openhsv.analysis.parameters.phaseAsymmetryIndex(left_gaw, right_gaw, opening)¶

Phase Asymmetry Index (PAI)

\[PAI = \frac{|argmax_j GA_i^L(j) - argmax_j GA_i^R(j)|}{N_i}\]

Note

We use here the maximum instead of the minimum, as it is more likely that there are multiple time points where the value is minimal or close to the minimum (i.e. when the glottis is closed for prolonged time).

Parameters:	left_gaw (numpy.ndarray) – GAW of left vocal fold right_gaw (numpy.ndarray) – GAW from right vocal fold opening (list) – indices where glottis starts to open (i.e. new cycle)
Returns:	PAI mean and std
Return type:	tuple(float, float)

openhsv.analysis.parameters.amplitudeSymmetryIndex(left_gaw, right_gaw, opening, epsilon=1e-05)¶

Amplitude Symmetry Index (ASI)

\[ASI = \frac{\min(\max_j(GA_i^L(j)), \max_j(GA_i^R(j)))}{\max(\max_j(GA_i^L(j)), \max_j(GA_i^R(j)))}\]

Parameters:	left_gaw (numpy.ndarray) – GAW of left vocal fold right_gaw (numpy.ndarray) – GAW from right vocal fold opening (list) – indices where glottis starts to open (i.e. new cycle) epsilon (float, optional) – numerical stability, defaults to 1e-5
Returns:	ASI mean and std
Return type:	tuple(float, float)

class openhsv.analysis.parameters.Signal(raw_signal, dt=0.00025, debug=True)¶

Bases: object

Inits the signal class with the raw signal, e.g. audio data or glottal area waveform.

Parameters:	raw_signal (numpy.ndarray) – Audio signal, GAW or alike

computeFFT(use_filtered_signal=True, use_hanning=True, lowpass_filter=20)¶

computeCepstrum()¶

filterSignal(cutoff_frequency=0.1)¶

detectCycles(method='peaks', peak='max', use_filtered_signal=True)¶

Detects cycles using different methods.

Parameters:	method (str, optional) – method to detect cycles, defaults to ‘peaks’ peaks Using a peak finding algorithm on the raw signal autocorrelation Detects cycles and period using autocorrelation use_filtered_signal (bool, optional) – uses filtered signal, if available.

detectPhases(use_filtered_signal=True)¶

Detects opening and close phase in each cycle.

Parameters:	use_filtered_signal (bool, optional) – Event detection on raw (False) or filtered (True) signal.

getPowerSpectrum()¶

Returns power spectrum from signal

Returns:	Frequencies and Amplitude
Return type:	tuple(np.ndarray, np.ndarray)

getCepsturm()¶

Returns cepstrum from signal

Returns:	quefrencies and cepstrum
Return type:	tuple(np.ndarray, np.ndarray)

class openhsv.analysis.parameters.Audio(raw_signal, dt=1.25e-05, use_filtered_signal=True, use_hanning=True, debug=False)¶

Bases: openhsv.analysis.parameters.Signal

The Audio class handles audio data to compute respective parameters.

Parameters:

raw_signal (numpy.ndarray) – the raw audio signal
dt (float, optional) – time between samples in seconds, defaults to 1/80000
use_filtered_signal (bool, optional) – use filtered signal for computations, defaults to True
use_hanning (bool, optional) – use hanning window for FFT, defaults to True
debug (bool, optional) – enable debugging mode, defaults to False

detectCycles(method='peaks', peak='max', use_filtered_signal=True)¶

Detects cycles using different methods.

Parameters:	method (str, optional) – method to detect cycles, defaults to ‘peaks’ peaks Using a peak finding algorithm on the raw signal autocorrelation Detects cycles and period using autocorrelation use_filtered_signal (bool, optional) – uses filtered signal, if available.

filterSignal(freq_range=20)¶

computeParameters(use_filtered_signal=False)¶

class openhsv.analysis.parameters.GAW(raw_signal, dt=0.00025, use_filtered_signal=False, use_hanning=True, debug=False)¶

Bases: openhsv.analysis.parameters.Signal

The GAW class handles the glottal area waveform (GAW) to compute respective parameters.

Parameters:

raw_signal (numpy.ndarray) – raw GAW signal
dt (float, optional) – time between samples in seconds, defaults to 1/4000
use_filtered_signal (bool, optional) – use filtered signal for computations, defaults to False
use_hanning (bool, optional) – use hanning window for FFT, defaults to True
debug (bool, optional) – enable debugging mode, defaults to False

setLeftRightGAW(left_gaw, right_gaw)¶

computeParameters()¶

class openhsv.analysis.parameters.AnalysisPlatform(raw_signal, dt)¶: Bases: PyQt5.QtWidgets.QWidget

openhsv.analysis.midline module¶

class openhsv.analysis.midline.Midline(seg, maxima=None)¶

Bases: object

Midline prediction module.

Midline is predicted based on segmentation from neural net for each peak. Midline is interpolated between peaks. Computations are based on Kist et al., biorxiv 2020.

Parameters:	seg (numpy.ndarray) – segmentation masks (T x H x W) maxima (list, optional) – detected maxima in GAW, defaults to None

predict(method='pca', time_range=5)¶

Predicts midline with given method on each GAW peak.

Parameters:	method (str, optional) – ‘pca’ or ‘moments’, defaults to ‘pca’ time_range (int, optional) – time range around peak to improve prediction, defaults to 5

side()¶

Returns left and right GAW based on midline in each frame

Returns:	left and right GAW as array T x 2
Return type:	numpy.ndarray

pvg(steps=64)¶

Computes PVG in discrete steps for each side.

Parameters:	steps (int, optional) – resolution along each axis, defaults to 64
Returns:	phonovibrogram as T x steps*2
Return type:	numpy.ndarray

openhsv.analysis.midline.imageMoments(im, transpose=True, angle_correction=1.5707963267948966)¶

Predicts midline using image moments.

Parameters:	im ([type]) – [description] transpose (bool, optional) – [description], defaults to True angle_correction ([type], optional) – [description], defaults to +np.pi/2
Returns:	[description]
Return type:	[type]

openhsv.analysis.midline.principalComponents(im, use_2nd=False)¶

Midline prediction using principal component analysis.

Parameters:	im (numpy.ndarray) – input image use_2nd (bool, optional) – use second principal component, defaults to False
Returns:	slope and intercept of midline
Return type:	tuple(float, float)

openhsv.analysis.pvg module¶

openhsv.analysis.pvg.get_labels(x_low, x_high, coef, intercept, image_shape, steps=64)¶

Function for getting left/right step-id’d labels based on the AP axis

Parameters:	x_low (float) – lower x coordinate x_high (float) – higher x coordinate coef (float) – coefficient of linear regression AP intercept (float) – intercept of linear regression AP image_shape (tuple(int, int)) – image size (HxW) for label steps (int) – steps between A and P
Returns:	label map
Return type:	numpy.ndarray

openhsv.analysis.pvg.compute_pvg(s, labels, steps=64)¶

Calculates Phonovibrogram based on labels.

Parameters:	s (numpy.ndarray) – segmented area (TxYxX) labels (numpy.ndarray) – labelled image steps (int, optional) – PVG resolution, defaults to 64
Returns:	PVG, time x 2*steps
Return type:	numpy.ndarray