"""
Providing the "Show Signal Wobbler" experiment. It provides the same
functionality as the "Show Signal" experiment with the added benefit of
using the wobbler for phase cycling to remove scattering of the IR pump
pulse on the detector.
For this experiment to run properly the chopping scheme needs to be
adjusted to the according wobbler frequency. The algorithm identifies
the chopper states and the wobbler states, phase cycles over the wobbler
states and then calculates the difference signal (absorption) from the
different chopper states.
While this experiment has the same applications as "Show Signal" it can
also be used to setup the wobbler because it also displays the signal
for each of the different wobbler states. See the note for an in depth
explanantion.
Other typical applications of this experiment are:
* Optimizing spatial overlap of the pump and probe pulses with
respect to the resulting signal
* Finding a scatter free and generally optimal position in the
sample
* Scanning different delays using the interface on the GUI
* Adjusting the central wavelength of the spectrometer
Note:
**General:**
The wobbler electronics provides three different parameters to
adjust the wobbler:
1. The duty cycle of the square pulse that drives the harmonic
motion of the wobbler
2. The phase (delay of the square pulse) relative to the laser
trigger
3. The amplitude (voltage of the square pulse) of the harmonic
motion of the wobbler
The duty cycle should in general be set to 50 %. The wobbler
only needs 2 degrees of freedom to be adjusted correctly. And
the duty cycle can change both amplitude and phase.
Adjust the amplitude first since it also changes the phase
relationship between laser and wobbler. As the scatter
supression for a given wavelength depends on the amplitude make
sure to optimize it either to the central wavelength of the
spectrometer or to the wavelength region that is of most
interest. Afterwards adjust the phase. Note that it can take
seconds for the wobbler to reach equilibrium after the phase was
changed. Either wait until the signals for the different wobbler
states stop changing or use the reference laser diode to observe
when the dots stop shifting.
Sometimes when turning the chopper on and off, the choppers'
phase moves relative to the laser. In the case where there is no
wobbler this might cause the sign of the observed signal to be
flipped. When using a wobbler in combination with a chopper the
frequency of the chopper has to be reduced to accommodate the
fact that each wobbler position must be overserved for each
chopper state. If the phase of the chopper now changes, it
might only flip the sign of the signal of one or two of the
different wobbler states. This might cause the user to think
that he has adjusted the wobbler correctly, while it is not
adjusted correctly. This results from an incorrect setting of
the TTL delay electronics and can be fixed by following the
procedure below.
Note that alternatively this could be remedied by a different
method for referencing the chopper. E.g.: Using a photodiode
plus additional electronics to reference the laser light
directly or using a light barrier to reference the position on
the chopper where the laser passes through.
**Chopper adjustment for wobbler:**
**General procedure:**
First turn off the wobbler. Now move the TTL of the chopper
such that the four lines representing the different wobbler
states are identical (and the sign of the signal is
correct). Turn on the wobbler and adjust it in a way that
two pairs of line are opposite to each other (cancel out).
This should reduce the scattering.
**Detailed procedure:**
For adjusting the wobbler the moveable path of the
interferometer needs to be blocked. A pulse delay circuit
which can also set the duty cycle is required for the
choppers. The duty cycle of the outgoing chopper pulse
(delayed pulse) should be tuned to 50 % with the delay
electronics. The next step is to set the chopper phase
correctly. For that it is necessary to tune the delay of the
outgoing pulse. If only three lines are displayed (wobbler
off) the chopper duty cycle is not correct. If everything
was set correctly (especially the phase of the chopper via
the delay electronics) and the wobbler is off, the signals
for a given wobbler state all overlap and have the correct
sign.
#######################
Step by Step Algorithm:
#######################
**Acquisition:**
1. Preallocate dictionary (data container) which will contain data
and information about scan index, delay index etc.
2. Read the data from the ADC
3. Place data into dictionary and hand it over to
primary processing
**Primary Processing:**
1. Preallocate arrays for data, counts, weights, chopper states and
wobbler states. Here there are 2 states. On (chopper high)
and off (chopper low). Also there are (generally) 4 different
wobbler states (left, center, right, center position of wobbler)
2. Subtract background from raw data (dark noise)
3. Linearize response of pixels
4. Calculate transmission, or more precisely, relative intensity
(probe intensity / reference intensity) for each laser shot for
each pixel pair
5. Identify the chopper states for all laser shots using the
corresponding channel(s) in the ADCs' data
6. Identify the different wobbler states for all laser shots
7. Sort the data (transmissions) for each state and calculate
statistics
8. Phase cycle by averaging the wobbler states in the transmission
space. Calculate the resulting phase cycled counts and weights
9. Put this information into data container and hand it over to
secondary processing
10. Save phase cycled data (and raw data) including phase cycled
counts and weights if respective checkboxes on GUI were checked
**Secondary Processing:**
1. Calculate the absorption of the phase cycled data (-log10)
2. Calculate the pump probe signal from the phase cycled absorption
3. Calculate the absorption (-log10) of the sorted data (non phase
cycled)
4. Calculate the pump probe difference signal (chopper high - chopper low)
of the non phase cycled data
5. Calculate the average intensities and standard deviation of
the intensities for each pixel
6. Put this information into data container
7. Hand over data to Pyqtplotting thread
8. Calculate the numbers which are displayed in the
"statistics box" on the GUI
**Pyqt Plotting:**
1. Remove old plots
2. Setup the plot that displays:
* Signals (including the signals for each wobbler state
and the resulting phase cycled signal)
* Intensities and their standard deviation (multiplied by 5)
3. Plot the plots for the first time
4. Update plots
**Saving:**
Note that the phase cycled transmission is saved. Therefore there exists
no dimension for the Wobbler states.
.. code-block::
programming data dimension:
[(2 ([0] is current average scan, [1] is last single scan), n_probe_pixels, n_wobbler_states, n_chopper_states)]
saving dimension:
[(n_probe_pixels, n_chopper_states)]
raw data dimension:
[(n_channels, samples_to_acquire)]
################
Folder Structure
################
In **scans/** the first file of 3 contains the phase cycled data in the
saving dimensions. The counts file contains the number of times a given
state was observed. This should be used as weights when averaging
equally weighted. The weights file contains the inverse variances of a
given state for all pixels, etc. The dimensions of these files are as
saving dimension suggests. These files can be used to average in
different ways in order to obtain the complete resulting spectrum. Check
the actual code of the corresponding experiment to understand how to
calculate the desired end result (difference spectrum).
The data in **/averaged_data** is averaged equally weighted (using
counts). Likewise to the scan data the difference spectrum still needs
to be calculated from the transmissions for every state. Because the
array contains the transmissions averaged with counts it is only
intended to be used as a first indicator for the measurement.
The **/figure** folder is currently empty. It is possible to implement
plotting of figures which are saved here while the experiment is
running. This is however not implemented yet.
The **/hardware config** folder holds every file which contains hardware
configuration parameters. This folder is a compressed copy of the
hardware configuration folder in the software's directory. Here it is
possible to look up the ADC configuration to obtain what channel was
connected to which hardware element (e.g. chopper - ai78). It is also
possible to obtain the R-2R values, the FPAS configuration, the
linearization parameters, etc.
The **/raw data** folder is in the experiments directory only if the
"save raw data" checkbox on the GUI was checked. It contains the raw
(unedited... as raw as it can get) data. Basically, the voltages which
the ADC measured for each channel. The dimensions are as "raw data
dimensions" suggests.
The **background.npy** file is a copy of the most recently collected
dark noise background of the MCT detector array. It corrects for dark
noise. If no new background was collected before the experiment was
started the code will use the latest available background and display a
warning in the log.
**setupinfo.txt** is a ReadMe file that contains the most relevant
experimental parameters at first glance. Additionally, the user can
decide to write a comment in the readme editor of the GUI. The content
of this is written to the **notes.txt** file.
.. code-block::
username/
├── date1_experimentname1_000/
│ ├── averaged_data
│ │ └── date1_experimentname1_000.npy
│ ├── figures
│ ├── hardware config
│ ├── raw data
│ │ ├── s000000_date1_experimentname1_000_raw.npy
│ │ ├── ...
│ │ └── s000099_date1_experimentname1_000_raw.npy
│ ├── scans
│ │ ├── s000000_date1_experimentname1_000.npy
│ │ ├── s000000_counts_date1_experimentname1_000.npy
│ │ ├── s000000_weights_date1_experimentname1_000.npy
│ │ ├── ...
│ │ └── s000099_d000_date1_experimentname1_000.npy
│ ├── setupinfo_date1_experimentname1_000.txt
│ ├── notes_date1_experimentname1_000.txt
│ └── background_date1_experimentname1_000.npy
├── date1_experimentname1_001/
├── date2_experimentname1_000/
└── date2_experimentname2_000/
"""
if __name__ == "__main__":
# Add directories to path for imports
import os, sys, inspect
currentdir = os.path.dirname(
os.path.abspath(inspect.getfile(inspect.currentframe()))
)
parentdir = os.path.dirname(currentdir)
sys.path.insert(0, parentdir)
sys.path.insert(0, os.path.join(parentdir, "hardware_interfaces"))
sys.path.insert(0, os.path.join(parentdir, "gui"))
import multiprocessing
from multiprocessing import Process, Queue
import sys
import threading
import numpy as np
from numpy import ndarray
# Matplotlib
import matplotlib
from matplotlib.backends.backend_qt5agg import (
FigureCanvasQTAgg,
NavigationToolbar2QT as NavigationToolbar,
)
from matplotlib.figure import Figure
# PyQTGraph
from pyqtgraph import PlotWidget, plot
import pyqtgraph as pg
from PyQt5 import QtWidgets, QtCore # , QtGui
from PyQt5.QtCore import QRunnable, QThreadPool, pyqtSignal, QObject
from qt_multithreading_wrapper import Worker
# Data processing
import data_processing as dp
from data_processing import PixelResponseLinearization as PRL
from data_processing import ChopperStateFinder as CSF
from analog_digital_converter import AnalogDigitalConverter as ADC
from triax import Triax as Spectrometer
from save_data import SaveData, Background
[docs]class ShowSignalWobbler:
"""
Args:
widget_pyqtgraph (QWidget): WidgetPyqtgraph object on which the
plots are going to be displayed. Has methods for plot
manipulation (i.e. removal of plots, autoscale).
adc (ADC): Analog to digital converter hardware object which is
used to communicate with and read data from the ADC.
prl (PRL): Pixel response linearization object which grants
the functionality to linearize raw data according to the
linearization parameters specified in the corresponding
*pixel_linearization_fit_parameters.json* file (for each
lab).
chopper_info (dict): Contains the information that is necessary
to identify the different chopper states of the chopper that
chops the pump pulse. It contains the keys "high voltage
level" and "name". "high voltage level" is the voltage read
by the ADC when the chopper reference output is high. It is
needed as a reference for the digitization function that is
used. The "name" key is required to determine to which
channel of the adc the chopper is connected and its value
needs to match the corresponding key in index_dict.
wobbler_freq (float): Frequency in Hz with which the wobbler
oscillates.
background_handler (Background): Instance of Background class
which can access the most recently collected background.
This background is later subtracted from the raw data as
dark noise correction.
spectrometer (Spectrometer): Spectrometer/Triax hardware class
which grants functionality to control the triax
spectrometer. Needed to obtain e.g. wavenumber axis etc.
info_queue (Queue): Multiprocessing queue object which is used
to transfer/hand over information to lineEdits on GUI.
Contains (if applicable) scan index, delay index, interleave
index, values for statistics groupBox etc.
saver (SaveData): Object that manages saving of data (including
counts, weights, probe wavenumber axis etc.) into their
respective directories. If None is passed, no data is going
to be saved. If the raw data checkbox was checked on the GUI
the savers raw data attribute is set to True and the raw
data is saved automatically. Defaults to None.
"""
def __init__(
self,
widget_pyqtgraph,
adc: ADC,
prl: PRL,
chopper_info: dict,
wobbler_freq: float,
background_handler: Background,
spectrometer: Spectrometer,
info_queue: Queue,
saver: SaveData = None,
):
# Initialise Queues
self.acq_queue = Queue()
self.processing_queue = Queue()
self.plot_queue = Queue()
if saver:
saver.save_other(background_handler.load_background(), "background")
self.acquisition = Acquisition(adc, spectrometer, self.acq_queue)
self.primary_processing = PrimaryProcessing(
self.acq_queue,
self.processing_queue,
adc.pixel_idx,
adc.probe_pixel_idx,
adc.reference_pixel_idx,
adc.index_dict,
prl,
chopper_info,
wobbler_freq,
adc.laser_frequency,
background_handler,
saver,
)
self.secondary_processing = SecondaryProcessing(
self.processing_queue, self.plot_queue, info_queue
)
# self.plotting = Plotting(mpl_widget, adc, self.plot_queue)
self.plotting = PyqtPlotting(widget_pyqtgraph, adc, self.plot_queue)
[docs] def start(self):
self.plotting.threadpool.start(self.plotting.work)
self.secondary_processing.start()
self.primary_processing.start()
self.acquisition.start()
[docs]class Acquisition(threading.Thread):
"""
Acquisition class (python multithreaded). This is where the actual
experiment is conducted.
This class is used to control the hardware devices required for the
experiment. The sole purpose of it is to collect the data according
to the parameters specified for the experiment by moving the delay
stages, opening and closing shutters etc.
A dictionary which will contain data and other information (scan
index etc.) is instantiated here. The acquisition class passes the
collected information (raw data, scan index etc.) to the primary
processing class.
Note:
**No data processing beyond what is required to conduct the
experiment should be implemented in this class.** The rationale
behind this is to minimize down time/ maximize laser time. Data
processing costs computation time and will, generally speaking,
slow down the measurement process because the computer is busy
while the rest of the hardware is idle. If implemented correctly
the data processing could be carried out while the data
acquisition is waiting for all data to become available. But
even in this scenario the problem that the data processing takes
longer than the acquisition time can occur and is thus best
avoided through parallelisation.
The reason why this class is a child of the threading module instead
of the multiprocessing module is that to use multiprocessing all
objects passed to the function must be picklable. This is not the
case for some of the objects interfacing with the hardware (e.g.
ADC). In an ideal scenario the acquisition too, would run in its own
process seperated from the GUI thread but this would only be
possible with major restructuring of the software.
Args:
adc (ADC): Analog to digital converter hardware object which is
used to communicate with and read data from the ADC.
spectrometer (Spectrometer): Spectrometer/Triax hardware class
which grants functionality to control the triax
spectrometer. Needed to obtain e.g. wavenumber axis etc.
acq_queue (Queue): Multiprocessing queue object that the
acquisition thread uses to pass data to the primary
processing process.
"""
def __init__(self, adc: ADC, spectrometer: Spectrometer, acq_queue):
threading.Thread.__init__(self)
self.adc = adc
self.spectrometer = spectrometer
# Multiprocessing Queue
self.acq_queue = acq_queue
# Initialize scan index
self.scan_idx = 0
# Create dictionary that will hold data that is passed
# to other queues
self.data_container = {}
# Create dictionary that holds information
# in which scan we are (etc.)
self.info = {}
# Multiprocessing event to stop experiment
self.exit = threading.Event()
[docs] def run(self):
# * You might wonder why this acquisition
# * is structured differently from the acquisition
# * of the other (older/simpler) experiments
# * [Or you might not wonder -
# * Lets hope that someone fixed this discrepancy
# * already]
# * The way it is implemented here (the straight
# * forward way) was chosen because it simplifies
# * the indices counting a lot, and within multiprocessing
# * should not lead to any loss of laser time.
# * Before the 0th data set was acquired outside of
# * the loop and was processed while the second
# * acquisition was running
while not self.exit.is_set():
# Read data
self.adc.read()
# Put relevant information into data container
self.data_container["data"] = self.adc.data
self.data_container["scan index"] = self.scan_idx
self.data_container["probe axis"] = self.spectrometer.wn_axis
# Give data of to queue
self.acq_queue.put(self.data_container.copy())
# Update scan idx
self.scan_idx += 1
# Tell other processes to stop
self.acq_queue.put("stop")
[docs] def shutdown(self):
# Setting exit will stop the loop
# within run
self.exit.set()
[docs]class PrimaryProcessing(multiprocessing.Process):
"""
Primary processing class (python multiprocess). This class' purpose
is to process the raw data to a state where it can be saved onto the
hard drive as npy (binary) files.
This step generally includes linearization, normalisation, sorting
and averaging. Besides the actual data additional
information that is required for post processing purposes is
calculated and saved. I.e. counts and weights. The last step
of primary processing should always be to pass the data to
secondary processing and save it to the hard disk.
Note:
The actual signal(s) are generally not intended to be calculated
here. Signals and other information that is supposed to be
displayed on the GUI should be calculated in secondary
processing. The main reason for this is minimizing the risk of
an error leading to a crash of the software which then in turn
ruins the measurement. The more code that has to run the more
likely a crash becomes. Others reasons mostly imply open
questions regarding averaging. Generally, shot to shot
normalized intensities that are sorted and averaged by their
state are saved (*m2 method*). From this - for a simple
experiment at least - the signal can be easily calculated while
offering different choices of averaging in post processing. For
more complex experiments e.g. VIPER or time domain experiments
the argument of saving sorted transmissions instead of signals
is even more compelling. In VIPER experiments more than one
signal of interest is present in the different states. Saving
each signal separately would actually increase the amount of
data that has to be saved. For time domain experiments we want
to save the data in the time domain for post processing reasons
like zeropadding and apodization.
The goal of primary processing is to make the data as compact as
possible while keeping as much information and flexibility as
possible. Even if the processes later on crash, the data is
secured and can be analysed in post processing.
Args:
acq_queue (Queue): Multiprocessing queue object that the
acquisition thread uses to pass data to the primary
processing process.
processing_queue (Queue): Multiprocessing queue object that the
primary processing process uses to pass data to the secondary
processing process.
pixel_idx (ndarray): Array that contains the indices of the rows
in the ADCs' data that correspond to pixel input channels.
These are specified in the "analog input configuration.json"
for each laboratory and can be easily accessed with the
attribute "pixel_idx" of the ADC.
* shape: 1D
* E.g.: (64) or (128)
probe_pixel_idx (ndarray): Array that contains the indices of
the rows in the ADCs' data that correspond to *probe* pixel
input channels. These are specified in the "analog input
configuration.json" for each laboratory and can be easily
accessed with the attribute "probe_pixel_idx" of the ADC. It
is highly relevant that the order the pixels are listed in
this array match the order of the array in the
reference_pixel_idx argument. This means that if reference
pixel 3 is listed first in the other array here probe pixel
3 needs to be listed first as well and so on. Otherwise the
intensities on the probe array are not going to be
normalized correctly. For the plotting to work correctly the
pixels also need to be listed in the order of the wavenumber
axis array of the spectrometer.
* shape: 1D
* E.g.: (32) or (64)
reference_pixel_idx (ndarray): Array that contains the indices
of the rows in the ADCs' data that correspond to *reference*
pixel input channels. These are specified in the "analog
input configuration.json" for each laboratory and can be
easily accessed with the attribute "reference_pixel_idx" of
the ADC. It is highly relevant that the order the pixels are
listed in this array match the order of the array in the
probe_pixel_idx argument. This means that if probe pixel 10
is listed first in the other array here reference pixel 10
needs to be listed first as well and so on. Otherwise the
intensities on the probe array are not going to be
normalized correctly. For the plotting to work correctly the
pixels also need to be listed in the order of the wavenumber
axis array of the spectrometer.
* shape: 1D
* E.g.: (32) or (64)
index_dict (dict): Dictionary that maps the names of the input
channels to their corresponding row in the ADCs' data as
they are specified in the "analog input configuration.json"
for each laboratory. I.e.: It contains the information which
entries of the ADCs' data array belong choppers, wobblers
etc. This dictionary can be easily accessed with the
attribute "index_dict" of the ADC.
adc (ADC): Analog to digital converter hardware object which is
used to communicate with and read data from the ADC.
prl (PRL): Pixel response linearization object which grants
the functionality to linearize raw data according to the
linearization parameters specified in the corresponding
*pixel_linearization_fit_parameters.json* file (for each
lab).
chopper_info (dict): Contains the information that is necessary
to identify the different chopper states of the chopper that
chops the pump pulse. It contains the keys "high voltage
level" and "name". "high voltage level" is the voltage read
by the ADC when the chopper reference output is high. It is
needed as a reference for the digitization function that is
used. The "name" key is required to determine to which
channel of the adc the chopper is connected and its value
needs to match the corresponding key in index_dict.
wobbler_freq (float): Frequency in Hz with which the wobbler
oscillates.
laser_freq (float): Frequency (repetition rate) of the laser in
Hz.
background_handler (Background): Instance of Background class
which can access the most recently collected background.
This background is later subtracted from the raw data as
dark noise correction.
saver (SaveData): Object that manages saving of data (including
counts, weights, probe wavenumber axis etc.) into their
respective directories. If None is passed, no data is going
to be saved. If the raw data checkbox was checked on the GUI
the savers raw data attribute is set to True and the raw
data is saved automatically. Defaults to None.
"""
def __init__(
self,
acq_queue: Queue,
processing_queue: Queue,
pixel_idx: ndarray,
probe_pixel_idx: ndarray,
reference_pixel_idx: ndarray,
index_dict: dict,
prl: PRL,
chopper_info: dict,
wobbler_freq: float,
laser_freq: float,
background_handler: Background,
saver: SaveData = None,
):
super(multiprocessing.Process, self).__init__()
# Multiprocessing Queue
# Gets data from acquisition class
self.acq_queue = acq_queue
# Gives data to secondary processing class
self.processing_queue = processing_queue
# Pixel response linearisation
self.prl = prl
# Wobbler frequency in Hz
self.wobbler_freq = wobbler_freq
# Laser frequency in Hz
self.laser_freq = laser_freq
# Data saving instance
self.saver = saver
# Pixel index is an array which tells us which
# entries in our adc data are pixels
# Append probe and reference pixel indices
self.pixel_idx = pixel_idx
self.probe_pixel_idx = probe_pixel_idx
self.ref_pixel_idx = reference_pixel_idx
self.index_dict = index_dict
# Load background data from file
self.background = background_handler.load_background()
# In this experiment we have two different chopper states
self.n_chopper_states = np.array(2)
# Save chopper high voltage level divided by 2 as list
# We need this for the digitize function that identifies
# the High and Low Chopper
# We select half of the high voltage as the limit
# at which the distinction between states is done
self.chopper_voltage_level = [chopper_info["high voltage level"][0] / 2]
self.chopper_name = chopper_info["name"][0]
# Calculate the number of wobbler states we are going to observe
self.wobbler_states = int(laser_freq // wobbler_freq)
# Stack the different number of states into one array.
self.number_of_possible_states = np.array(
[self.n_chopper_states, self.wobbler_states]
)
# Initialize array that
# sorted data is written into.
# This is different from other
# experiments (without wobbler)
# because the delay axis as well
# as the average and single scan
# dimension have been removed.
# Here we need to phase cycle directly:
# 1. The get_wobbler_states algorithm
# is not "safe" in fringe cases.
# This could lead to averaging
# different wobbler states together
# when averaging different scans.
# If we directly average wobbler states
# for each scan this problem does not
# arise.
# 2. It makes the most sense to directly
# phase cycle the data of one scan:
# this has to do with the
# imprecision of the delay stage
# when measuring two time the same delay
# it will move to slightly different
# locations yielding slightly different
# phases. Averaging these my yield
# artifacts.
# 3. This reduces the size of the data
# set that is going to be saved by
# a factor of 4.
self.data = np.zeros(
(
self.probe_pixel_idx.size,
*self.number_of_possible_states,
) # all chopper states, all wobbler states
)
self.counts = np.zeros(self.data.shape)
self.weights = np.zeros(self.data.shape)
# Because we use a Wobbler in this experiment
# we need a second set of arrays where
# the phase cycled/ scatter free transmissions
# are written into.
# Initialize array that holds both
# temporary and averaged data
# In this case the data is average
# relative intensity
# (transmission (probe/ref))
# for each pixel pair.
# By convention the averaged is the
# 0 th entry of the 0th axis of the array
# and the temp data is the 1st entry of the 0th
# axis of the array.
# dimensions: 2 = (current average scan, last single scan)
# dimensions: (2, n_probe_pixels, n_chopper_states)
# Phase cycled = scatter free
self.phase_cycled_data = np.zeros(
(
2, # averaged and non averaged (scan)
self.probe_pixel_idx.size,
self.n_chopper_states,
) # all chopper states
)
self.phase_cycled_counts = np.zeros(self.phase_cycled_data.shape)
self.phase_cycled_weights = np.zeros(self.phase_cycled_data.shape)
[docs] def run(self):
while True:
# Get data / information from acquisition process
data_container = self.acq_queue.get()
if type(data_container) == str:
if data_container == "stop":
break
# Write data in dictionary into variable
raw_data = data_container["data"]
scan_idx = data_container["scan index"]
# Subtract background from raw data
# of pixels and pixels only
background_corrected_data = (
raw_data[self.pixel_idx] - self.background[self.pixel_idx, np.newaxis]
)
# Linearize response of Pixels (and pixels only)
intensities = self.prl.linearize(background_corrected_data)
# Calculate transmission/ relative intensity
# (probe intensity / reference intensity)
transmission = (
intensities[self.probe_pixel_idx] / intensities[self.ref_pixel_idx]
)
# Get the corresponding chopper state for each shot
chopper_states = np.digitize(
raw_data[self.index_dict[self.chopper_name]], self.chopper_voltage_level
)
# Get Wobbler States
wobbler_states = dp.get_wobbler_states(
raw_data[self.index_dict["wobbler"]],
self.laser_freq,
wobbler_freq=self.wobbler_freq,
)
# Generate states array by stacking chopper states and wobbler states
# ? There is proably a faster way to do this than stacking by preallocating arrays? Nvm for now
states = np.vstack((chopper_states, wobbler_states))
# Sort and average data
self.data[:], self.weights[:], self.counts[:], statistics = dp.sort_data(
transmission, states, self.number_of_possible_states
)
# --------- Phase cycling ----------
# phase cycled = scatter free
# -- Phase cycle transmissions --
# Remove scattering by averaging the different wobbler states
# * In VB6 the interleaves are averaged
# * once the difference spectra are calculated
# * this (below) first phase cycles and then
# * offers data for signal calculating
# ** Update: We tested/compared the difference
# ** for one data set and the absolute difference
# ** was consistently smaller 10e-3 smaller
# ** than the signal (this was done with interleaves
# ** but the same logic applies to the wobbler)
# ** We (Jens and us) agree that it conceptually
# ** makes more sense to average the relative
# ** intensities.
#! It should be tested whether it makes sense to first average
#! scans and then phase cycle (we are first phase cycling then averaging here)
# * In this scenario we don't actually need to compute the wobbler states,
# * which wobbler position came first etc.
# * It would suffice to separate the data in 4 different states like
# * this: 0,1,2,3,0,1,2,3,0,1....
# * When averaging the way it is done here the actual position of the
# * does not matter because they are averaged out before we average
# * two different scans.
self.phase_cycled_data[1] = np.average(
self.data,
# Do not use weights to phase cycle because each wobbler position needs to be weighted equally
axis=-1, # The last axis of the array is the wobbler axis
)
# Now we need to calculate the total counts
# that were observed in each chopper state
# for all wobbler states
self.phase_cycled_counts[1] = self.counts.sum(axis=-1)
# We also need to update the weights (inverse variance of each state)
self.phase_cycled_weights[1] = self.weights.sum(axis=-1)
# Add everything to data container
# and give it to processing queue
data_container["sorted data"] = self.data.copy()
data_container["phase cycled data"] = self.phase_cycled_data[1].copy()
data_container["intensities"] = intensities
data_container["transmission"] = transmission
data_container["statistics"] = statistics
data_container["chopper states"] = chopper_states
self.processing_queue.put(data_container.copy())
# ----------------------------------------
# Save data (if specified)
if self.saver:
# ----- Average data ----
# Create / Average "averaged" phase cycled data by using
# weighted average between the phase cycled temp data
# (weight = counts for each state in "temp" data set)
# and the already existing average data
# (weight = counts for each state in average data set)
self.phase_cycled_data[0] = np.average(
self.phase_cycled_data[:],
axis=0,
weights=self.phase_cycled_counts[:],
)
# Use saver class to save data
self.saver.save_scan(self.phase_cycled_data[1], scan_idx)
self.saver.save_avg(self.phase_cycled_data[0])
self.saver.save_counts(self.phase_cycled_counts[1], scan_idx)
self.saver.save_weights(self.phase_cycled_weights[1], scan_idx)
if self.saver.raw_data:
self.saver.save_raw_data(raw_data, scan_idx)
# Update counts
self.phase_cycled_counts[0] += self.phase_cycled_counts[1]
# Once stop signal was received the function breaks out of the loop.
# Now we need to tell the secondary processing to stop.
self.processing_queue.put("stop")
[docs]class SecondaryProcessing(multiprocessing.Process):
"""
Secondary processing class (python multiprocess). This class is used
to process the data from the primary processing class such that it
can be displayed on the GUI within plots and lineEdits. This
generally implies (if applicable):
* calculation of signals
* calculation of statistics like standard deviation of
intensities and shot to shot standard deviation of signal
* interpolation for 2D / heatmap plots (see comments in code why
this is necessary)
* Fourier transform and phasing for time domain data
This data is handed over to the plotting thread.
Note:
The feature of saving figures/plots to the hard drive should be
implemented here if it is needed.
Args:
processing_queue (Queue): Multiprocessing queue object that the
primary processing process uses to pass data to the secondary
processing process.
plot_queue (Queue): Multiprocessing queue object that the
secondary processing process uses to pass data to the plot
thread.
info_queue (Queue): Multiprocessing queue object which is used
to transfer/hand over information to lineEdits on GUI.
Contains (if applicable) scan index, delay index, interleave
index, values for statistics groupBox etc.
"""
def __init__(
self, processing_queue: Queue, plot_queue: Queue, info_queue: Queue,
):
super(multiprocessing.Process, self).__init__()
# Multiprocessing Queue
# Gets data from acquisition class
self.processing_queue = processing_queue
# Gives data to secondary processing class
self.plot_queue = plot_queue
# Give experimental status and statistics to
# GUI
self.info_queue = info_queue
[docs] def run(self):
while True:
# Get data / information from acquisition process
data_container = self.processing_queue.get()
if type(data_container) == str:
if data_container == "stop":
break
# ----------------------------------------
# Calculate information for plotting and GUI
# We first calculate everything that is required
# for plotting and pass it to the plot queue because
# plotting can also cost time. Only then we
# calculate the values that we want to display on
# line edits on the GUI
# --- Note considering naming:
# For experiments with interleaves we choose
# to call the non phase cycled signals="signal"
# and the phase cycled signal="phase_cycled_signal"
# here we reverse this order. So non phase cycled
# signals = "npc_signal" and phase cycled signal
# = "signal".
# We do this because with interleaves it takes
# time to measure all delay stage positions.
# Here all different phase states are collected
# within one acquisition. ---
# Calculate the pump probe signal from the sorted
# and phase cycled data
# phase cycled = scatter free
phase_cycled_data = data_container["phase cycled data"]
absorption = -np.log10(phase_cycled_data)
signal = absorption[:, 1] - absorption[:, 0]
# Additionally calculate the non phased cycled signal
# for each wobbler state
# npc = non-phase cycled
sorted_data = data_container["sorted data"]
npc_absorption = -np.log10(sorted_data)
npc_signal = npc_absorption[:, 1, :] - npc_absorption[:, 0, :]
# Calculate the average intensity for each pixel
avg_intensities = np.average(data_container["intensities"], axis=1)
# Calculate the standard deviation of the
# intensities
std_intensities = np.std(data_container["intensities"], axis=1, ddof=1)
# Add everything to data container
# and give to plot queue
data_container["signal"] = signal
data_container["non phase cycled signal"] = npc_signal
data_container["average intensity"] = avg_intensities
data_container["std intensity"] = std_intensities
self.plot_queue.put(data_container.copy())
# ----------------------------------------
# Calculate everything that is needed for statistical information
# on GUI and add it to data container and give it to info queue
# Hand over the mean state information for every pixel
data_container["mean state intensity"] = np.average(
data_container["transmission"], axis=1
)
data_container["mean state std"] = data_container["statistics"][1]
self.info_queue.put(data_container.copy())
# Once stop signal was received the function breaks out of the loop.
# Now we need to tell the plotting and updating of info on GUI to stop.
self.plot_queue.put("stop")
self.info_queue.put("stop")
[docs]class PyqtPlotting:
"""
Pyqt Plotting class (Qt multithreaded). This class is necessary for
displaying plots on the GUI. PyQtGraph is used as the plotting
engine. Generally the plots are set up first (type of plot, layout,
title etc.). On the first run, the plots are drawn for the first
time. Then the plots are updated every iteration. We update the same
plot references every time to make it more efficient.
Args:
widget_pyqtgraph (QWidget): WidgetPyqtgraph object on which the
plots are going to be displayed. Has methods for plot
manipulation (i.e. removal of plots, autoscale).
adc (ADC): Analog to digital converter hardware object which is
used to communicate with and read data from the ADC.
plot_queue (Queue): Multiprocessing queue object that the
secondary processing process uses to pass data to the plot
thread.
"""
[docs] class Signals(QObject):
new_data = pyqtSignal(dict)
def __init__(self, widget_pyqtgraph, adc: ADC, plot_queue: Queue):
# Assign attributes
self.adc = adc
self.widget_pyqtgraph = widget_pyqtgraph
self.graphics_layout = widget_pyqtgraph.graphics_layout
self.plot_queue = plot_queue
# Setup signals and threadpool
self.threadpool = QThreadPool()
self.signals = self.Signals()
# Clear old plots
self.widget_pyqtgraph.remove_plots()
# Setup plot that displays pump probe signal
self.widget_pyqtgraph.plots["signal"] = self.graphics_layout.addPlot(
row=0, col=0, colspan=2
)
self.widget_pyqtgraph.plots["signal"].setTitle("Average signal")
self.widget_pyqtgraph.plots["signal"].setLabel(
"bottom", "wavenumber [cm<sup>-1</sup>]"
)
self.widget_pyqtgraph.plots["signal"].setLabel("left", "difference signal[OD]")
self.widget_pyqtgraph.set_style(self.widget_pyqtgraph.plots["signal"])
# Setup plot that displays intensities
self.widget_pyqtgraph.plots["intensities"] = self.graphics_layout.addPlot(
row=2, col=0
)
self.widget_pyqtgraph.plots["intensities"].setTitle("Average intensities")
self.widget_pyqtgraph.plots["intensities"].setLabel(
"bottom", "wavenumber [cm<sup>-1</sup>]"
)
self.widget_pyqtgraph.plots["intensities"].setLabel("left", "intensity [a.u.]")
self.widget_pyqtgraph.set_style(self.widget_pyqtgraph.plots["intensities"])
# Create dictionary that holds reference to lines etc.
self.plot_ref = {}
# Connect signal that data has arrived to update the plot
self.signals.new_data.connect(
lambda data_container: self.update_plot(data_container)
)
# Start loop that gets data from queue in Qt Thread
self.work = Worker(self.run)
[docs] def run(self):
while True:
data_container = self.plot_queue.get()
if type(data_container) == str:
if data_container == "stop":
break
self.signals.new_data.emit(data_container)
[docs] def update_plot(self, data_container):
# Plot for the first time to get line references
signal = data_container["signal"]
npc_signal = data_container["non phase cycled signal"]
probe_axis = data_container["probe axis"]
avg_intensities = data_container["average intensity"]
std_intensities = data_container["std intensity"]
if self.plot_ref:
# Update non phase cycled signal / Signal for each wobbler state
for i in range(npc_signal.shape[-1]):
self.plot_ref["signal wobbler state {}".format(i)].setData(
x=probe_axis, y=npc_signal[:, i]
)
# Update signal plot
self.plot_ref["signal"].setData(x=probe_axis, y=signal)
# Update intensity error bars for probe array
self.plot_ref["probe error bars"].setData(
x=probe_axis,
y=avg_intensities[self.adc.probe_pixel_idx],
height=5 * std_intensities[self.adc.probe_pixel_idx],
)
# Update intensity error bars for reference array
self.plot_ref["ref error bars"].setData(
x=probe_axis,
y=avg_intensities[self.adc.reference_pixel_idx],
height=5 * std_intensities[self.adc.reference_pixel_idx],
)
# Update intensities
self.plot_ref["probe intensities"].setData(
x=probe_axis, y=avg_intensities[self.adc.probe_pixel_idx]
)
self.plot_ref["ref intensities"].setData(
x=probe_axis, y=avg_intensities[self.adc.reference_pixel_idx]
)
else:
# First time plotting
signal_pen = pg.mkPen(color="#d62728", width=2.5, style=QtCore.Qt.SolidLine)
npc_signal_pen = pg.mkPen(
color="#A9A9A9", width=1.2, style=QtCore.Qt.SolidLine
)
probe_pen = pg.mkPen(color="#1f77b4", width=2.5, style=QtCore.Qt.SolidLine)
reference_pen = pg.mkPen(
color="#ff7f0e", width=2.5, style=QtCore.Qt.SolidLine
)
std_intensity_pen = pg.mkPen(
color="#7f7f7f", width=1.5, style=QtCore.Qt.SolidLine
) # ? dashed lines?
std_signal_pen = pg.mkPen(
color="#2ca02c", width=2.5, style=QtCore.Qt.SolidLine
)
# Plot non phase cycled signal / Signal for each wobbler state
for i in range(npc_signal.shape[-1]):
self.plot_ref[
"signal wobbler state {}".format(i)
] = self.widget_pyqtgraph.plots["signal"].plot(
x=probe_axis,
y=npc_signal[:, i],
name="Pump-probe signal wobbler state {}".format(i),
pen=npc_signal_pen,
)
# Plot signal
self.plot_ref["signal"] = self.widget_pyqtgraph.plots["signal"].plot(
x=probe_axis,
y=signal,
name="Phase cycled pump-probe signal",
pen=signal_pen,
)
# Create intensity error bars for probe array
self.plot_ref["probe error bars"] = pg.ErrorBarItem(
x=probe_axis,
y=avg_intensities[self.adc.probe_pixel_idx],
height=5 * std_intensities[self.adc.probe_pixel_idx],
beam=0.3,
pen=std_intensity_pen,
)
self.widget_pyqtgraph.plots["intensities"].addItem(
self.plot_ref["probe error bars"]
)
# Create intensity error bars for reference array
self.plot_ref["ref error bars"] = pg.ErrorBarItem(
x=probe_axis,
y=avg_intensities[self.adc.reference_pixel_idx],
height=5 * std_intensities[self.adc.reference_pixel_idx],
beam=0.3,
pen=std_intensity_pen,
)
self.widget_pyqtgraph.plots["intensities"].addItem(
self.plot_ref["ref error bars"]
)
# Plot intensities
self.plot_ref["probe intensities"] = self.widget_pyqtgraph.plots[
"intensities"
].plot(
x=probe_axis,
y=avg_intensities[self.adc.probe_pixel_idx],
name="Average intensities on probe array",
pen=probe_pen,
)
self.plot_ref["ref intensities"] = self.widget_pyqtgraph.plots[
"intensities"
].plot(
x=probe_axis,
y=avg_intensities[self.adc.reference_pixel_idx],
name="Average intensities on reference array",
pen=reference_pen,
)
self.widget_pyqtgraph.disable_autoscale()