notes.txt

Requirements: NI VISA software, Python 3, pyvisa 1.9.1

The Matisse device ID is constant under visa.ResourceManager().list_resources()
Always USB0::0x17E7::0x0102::07-40-01::INSTR at least on my machine :/ Have to confirm this for other computers

PyVisa-Py doesn't work on Linux, and the proprietary VISA libraries are a nightmare to install, don't bother

Average execution time of various commands:
Grabbing DC diode power: ~0.5 ms
Grabbing BiFi motor pos: ~3.3 ms
Grabbing BiFi wavelength: ~4.4 ms
Grabbing TE motor pos: ~3.8 ms
Grabbing refcell pos (SCAN:NOW?): ~0.5 ms
Note: SCAN:DEVICE? returns 2, which is the ID of the refcell piezo. This is expected.

An error will lock all commands :(
Have to use ERROR:CLEAR to continue querying.
Error display syntax: "!ERROR N" where N is the number of errors on the stack

Trying to run/stop the control loops for slow/fast piezos multiple times in a row is fine, no errors.
When stabilizing the laser, the reference cell, slow piezo, fast piezo, and piezo etalon all change.
The reference cell limits for the scan are usually left alone, and the up/down directions specified in the commander
software indicate where the reference cell mirror is going, continuing the scan until it hits one of those limits.

SCAN:MODE? -> up = 0, down = 1. up = increase wavelength, down = decrease wavelength
We can set the scan mode while a scan is running without any issues.

Detailed process for setting wavelength:
1. Set approx. wavelength using BiFi.
2. Scan the BiFi back and forth and measure the total laser power at each point. Power looks like upside-down parabolas.
3. Find all local maxima of the laser power data. Move the BiFi to the maximum that's closest to the desired wavelength.
4. Scan the thin etalon back and forth and measure the thin etalon reflex at each point.
5. Find all local minima of the reflex data. Move the TE to the minimum that's closest to the desired wavelength. Maybe
   use a binary search?
6. Shift the TE to the left or right a little bit. We want to be on the "flank" of the chosen parabola. Distance to
   shift: about 1/5 the distance between current minimum and next local maximum.
7. Adjust the piezo etalon until desired wavelength is reached.
8. Lock slow piezo, thin etalon, piezo etalon, then fast piezo. Stabilize at the desired wavelength.

Wavemeter is read via serial port connection. Since I don't have a serial port I'll have to look at the existing MATLAB
code that's currently used to read the wavelength and hope for the best. Maybe try testing out Python -> serial commands
on the Fianium laser.

Reading from wavemeter: use VAL? command, value is the last 8 bytes of the response. Split it at the comma.
IDN?: b'*IDN$ Coherent Inc,WaveMaster,W0364,A1.1V1.6\r\n'
Error format: b'ERR$ 01\r\n'
Value format: b'VAL$ 59445457, 737.776\r\n'

Adjusting the refcell will simultaneously change positions of other piezos (notably the slow piezo and piezo etalon).
Other piezos appear to travel opposite the direction of the refcell.
For larger drifts on stabilization scan the refcell is not great, so jump over another mode using the TE piezo.

If you reach the limits of any piezo while adjusting the wavelength:
  - For SMALL corrections:
    - To increase wavelength: set slow piezo and piezo etalon to large values, set refcell to small value. This
      shouldn't change the wavelength very much. Then you can increase the refcell more.
    - If the drift is still too high after this adjustment, consider the case for larger corrections instead or ask for
      manual correction from the user
  - For LARGER corrections:
    - For larger corrections, you're going to need to adjust the TE piezo. Might be better to just repeat the whole
      wavelength-setting process here.

Piezo etalon value to set is PZETL:BASE, the baseline of the modulation waveform. This is the value set by the slider in
Matisse Commander.

Today I got 'VAL$ 67325238,NO SIGNAL' in response to 'VAL?' from the WaveMaster, though it was working a couple minutes
beforehand. Turns out you need to actually turn on the Matisse before you can use it.

Doing a birefringent filter scan and a thin etalon scan trigger the error lights on the Matisse :(
Sending individual movement commands works fine, but as soon as I try looping through a large list of positions it stops
working. Might be a timing issue? The voltage data I'm reading as I scan is just noise in one position.

The manual says there is some communication overhead in talking to the motor, so the commands need a bit of time to
execute. Look at the error code set by the motor controller to confirm this is the case. The motor will ignore commands
while it's busy, which is what I suspect is happening.

Error code given by the BiFi motor is 8840 when doing BiFi scan, which indicates "motor is running (command ignored)".
Idle status is 8194, "system is waiting for command / button pressed"
While moving motor using 'MOTBI:REL': 8969, "execute relative move command"

When running loop while checking for idle status, I keep getting code 8706, sometimes 514. No change in motor position.
Got it working, just use the last 8 bits of the motor status to decide whether to send the command, since the first few
bits change depending on other factors.

Important feature to add from Matisse Commander: RefCell properties measurement. Scan the RefCell, measure the RefCell
spectrum, fit to an Airy function. Do it when the laser is locked? When doing a fit, change the parameters of the fit to
the new fit so that the next fit uses the previous fit. This allows us to slowly correct a fit.

More:
RefCell waveform dialog, used primarily to find a recommended setpoint for the fast piezo (amplitude of the transmission
spectrum at full-width-half-max points). Scans RefCell and measures transmitted intensity.
RefCell properties measurement: measures transmitted intensity of the RefCell while scanning it. This is less of a
priority.
Control loop live view, shows live feed of values of internal variables used by various control loops.

Trying to open IDLE from the GUI results in an error since the WaveMaster can only be accessed with one instance at a
time. Trying to set the wavelength via the GUI results in a timeout error:
VI_ERROR_TMO (-1073807339): Timeout expired before operation completed.
Trying to access the BiFi motor status many times in a row causes a different error:
VI_ERROR_INP_PROT_VIOL (-1073807305): Device reported an input protocol error during transfer.
Restarting the GUI repeatedly gives the same error.
Continuing to execute commands via a shell sometimes gives another error:
Error executing Matisse command 'MOTBI:STATUS?' :ERR:CODE: 208
Error 208 is: "transfer buffer too small; the USB message will not fit into the host computer's transfer buffer."
This is unresolved until the control box is switched off and then on again.

Trying to write the RefCell position info when stabilizing gives an error:
print(f"Within tolerance.      Drift is {drift}, RefCell pos {self._matisse.query('SCAN:NOW?', numeric_result=True)}")
VI_ERROR_RSRC_LOCKED (-1073807345): Specified type of lock cannot be obtained, or specified operation cannot be
performed, because the resource is locked.
Might need to look into the pyvisa locking methods.

I'm getting none of the above issues when executing commands from a shell, no GUI. This leads me to believe it's a
threading issue and/or a resource locking issue. The Matisse can handle many commands thrown to it at once, but when
executing commands from different threads there seems to be an issue. Might need to stop other threads while stuff like
scans are running, then start the threads again when we finish. Simple one-query commands like setting the approx. BiFi
wavelength don't cause locking issues, so hopefully we only need to worry about the bigger stuff.

Disabling the status monitor widget in the GUI works!! Definitely a threading/locking issue. Might have to scrap the
status monitor or figure out a workaround.

Another thing I noticed is that the output while a long operation is running is not displayed in the log area, so
something is blocking the main thread. Do longer work in a separate thread so log area can update normally.

While the control loops are on, the fast piezo is not necessarily locked. Suggested wait time is about 7 seconds for a
lock, if not locked within this timeout or if any components reach their limits within that timeout, then stop the
locking procedure. If the fast piezo locks within the timeout, keep checking limits. If a component is at its limit, you
can adjust it in the opposite direction. When the fast piezo is locked we want to avoid disabling the lock and opt for
automatic corrections to components instead, then stop locking and perform the whole set_wavelength procedure again.

Need 3 options in stabilization menu: locking laser without setting fast piezo setpoint, locking and automatically
setting fast piezo setpoint, and a separate dialog to just set the fast piezo setpoint.

Don't give feedback unless something important is happening or you're changing something.
Pop up a dialog to set parameters of RefCell stabilization.
Adjust reference cell after setting wavelength to get it just right, then perform much smaller BiFi/TE scans after
setting wavelength to ensure you're still on the peaks. Also nice to see a scan while it's happening.

Timer doesn't seem to be working for lock correction timeout. Also, make warning messages orange, errors red.

If can't obtain laser lock, suggest manual stabilizing of output power of the laser, or suggest redoing the lock while
setting the fast piezo setpoint. When we try locking again, put the 3 piezos back in the middle if they're too far to
one side.

Make a configuration window where you can manually set some of the constants you've defined, like the lock correction
timeout or the scan parameters.

Scan plots aren't showing, there's probably some issue with which thread we're on when we have to show the plots.

When BiFi goes up, wavelength goes down. When thin etalon goes down, wavelength goes down. When piezo etalon goes down,
wavelength goes up. When scanning the reference cell, the piezo etalon and thin etalon change too (the piezo etalon
moves opposite the reference cell, and the thin etalon moves with the reference cell). Double checked the usefulness of
the 'flank' option in Matisse Commander, didn't see any change or difficulty in setting the laser if flank was 'correct'
or not. Just setting the position of the reference cell while the laser is unlocked doesn't change any other piezo. It's
important to specifically do a scan instead.

Distance between each BiFi scan maximum is about 0.45 nm.
Distance between each thin etalon scan minimum is about 0.033 nm.

New and improved scanning process:
1. Initial BiFi set and possible scan
2. Initial TE set and possible scan
3. Small BiFi scan to keep it on the peak
4. Small TE scan to keep it on the dip

Check wavelength difference between current and target.
If it's greater than about 0.4 nm, do a large birefringent scan to choose a better peak.
If it's between about 0.15 nm and 0.4 nm, do a small birefringent scan to keep it on the peak.
If it's between 0.02 nm and 0.15 nm, skip the first birefringent scan and go right to the thin etalon scan.
If it's less than 0.02 nm, skip all BiFi and TE scans, and just do a RefCell scan.

For the thin etalon scan: figure out how to move the wavelength a certain amount with just the motor. Move to this
approximate position and then do the TE scan. Average motor steps between each minima in my sample_scans data: 467
Approximate change in wavelength between each minimum: 0.033 nm, so nm/pos = 7.08589e-05

Took some large scans this morning, here is my analysis:
Separation between BiFi maxima: mean 128.3 motor steps, median 120 motor steps. mean -0.467 nm, median -0.458 nm.
Approx. conversion from BiFi motor pos to nm: nm = -0.00384829*pos + 1123.63323709
Approx. conversion from nm to BiFi motor pos: pos = -256.481*nm + 289489.236
Separation between thin etalon minima: mean 548.5 motor steps, median 540 motor steps. mean 0.033 nm, median 0.033 nm.
Change in wavelength per thin etalon step: 5.797738e-05, a bit smaller than the value found before
Approx. conversion from nm to thin eta pos: pos = 17006.868*nm - 12541923.113

Large scans of thin etalon sometimes cause large jumps in wavelength, this happens when BiFi falls into a valley.
Check difference in wavelength at each minimum and if one is very large then do a small BiFi scan to get back on track.
Also do directional scans based on the wavelength difference. No need to scan up if the wavelength needs to go down.

Suggestion for when wavelength starts to jump a lot during a thin etalon scan due to the BiFi being stuck in a valley:
Jump over to the next mode (approximately) on the BiFi by using a nm/pos constant, redo the thin etalon scan. The BiFi
should do another small scan afterwards to keep it on the top of the new mode.

It's safe to say that during ANY thin etalon scan, if the wavelength makes very large jumps due to the BiFi getting
stuck, then you can adjust the BiFi as described above.

Good default for cancelling set_wavelength. If wavelength difference is more than 10 nm, confirm if they're sure they
want such a large change. Make the default option on that dialog to cancel, for safety reasons. Also make the precision
for setting the wavelength go to 3 decimal places!

Add interface to initiate small scans of the device with a given speed and direction, for PLE data collection.

Throwing errors from other threads seems to hang the error dialog. Trying to run a BiFi/TE scan while stabilizing makes
GUI hang and cannot exit.

For PLE, software is Andor SOLIS for Spectroscopy, spectrograph is the Andor Shamrock SR-750, CCD is the Andor Newton.
Found a library that may control the spectrograph - https://github.com/sdickreuter/python-andor which provides bindings
to the SDK. Only the basic functionality needed, set center, exposure time, etc.

Got the Shamrock SDK loaded in Python via ctypes! Had to replace the FTDI 2XX driver with the 64-bit version, silly that
it's not already included. Now I can call any function from the C API straight from Python.

Can't control the CCD with the Shamrock library, need the full Andor SDK. https://pypi.org/project/andor/ is written for
Linux and for Python 2.7, so it might not be suitable for this project. I can look through the code and see if there's
anything useful.

Need to know:
- What is the readout mode and the acquisition mode used to take spectra with the Shamrock?
- Does the temperature remain constant even when the USB cable is unplugged?

While testing control program again, got this error:
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Python3.7\Lib\multiprocessing\spawn.py", line 105, in spawn_main
    exitcode = _main(fd)
  File "C:\Python3.7\Lib\multiprocessing\spawn.py", line 114, in _main
    prepare(preparation_data)
  File "C:\Python3.7\Lib\multiprocessing\spawn.py", line 225, in prepare
    _fixup_main_from_path(data['init_main_from_path'])
  File "C:\Python3.7\Lib\multiprocessing\spawn.py", line 277, in _fixup_main_from_path
    run_name="__mp_main__")
  File "C:\Python3.7\Lib\runpy.py", line 263, in run_path
    pkg_name=pkg_name, script_name=fname)
  File "C:\Python3.7\Lib\runpy.py", line 96, in _run_module_code
    mod_name, mod_spec, pkg_name, script_name)
  File "C:\Python3.7\Lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Users\Luc\Documents\Python\matisse-controller\matisse_controller\gui\control_application.py", line 9, in <module>
    from PyQt5.QtWidgets import QVBoxLayout, QMainWindow, QWidget, QInputDialog, QMessageBox, QApplication
ImportError: DLL load failed: The paging file is too small for this operation to complete.
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Python3.7\Lib\multiprocessing\spawn.py", line 105, in spawn_main
    exitcode = _main(fd)
  File "C:\Python3.7\Lib\multiprocessing\spawn.py", line 114, in _main
    prepare(preparation_data)
  File "C:\Python3.7\Lib\multiprocessing\spawn.py", line 225, in prepare
    _fixup_main_from_path(data['init_main_from_path'])
  File "C:\Python3.7\Lib\multiprocessing\spawn.py", line 277, in _fixup_main_from_path
    run_name="__mp_main__")
  File "C:\Python3.7\Lib\runpy.py", line 261, in run_path
    code, fname = _get_code_from_file(run_name, path_name)
  File "C:\Python3.7\Lib\runpy.py", line 236, in _get_code_from_file
    code = compile(f.read(), fname, 'exec')
MemoryError

Would be nice to measure the drift of the laser over time as it's being stabilized. Every time the wavelength drifts by
more than 0.001 nm, log the time, the wavelength, positions of stabilization piezos, and also every time you reset the
stabilization piezos with the same information logged.

Having issues reading from CCD. Doing all the initialization doesn't seem to give us any counts when doing a scan.
One possibility is that the GetAcquiredData function is returning an error code and not modifying the data array (check
this by setting data array to some non-physical value). Another possibility is that there are more configuration calls
needed before a scan will work. I'll try some of the SDK examples to make sure it's not an issue with my machine.

Getting issues again when doing a thin etalon scan, motor gives an error and refuses to run :( temporary solution is to
just add a delay, another solution is to look at the status again. Update: this occurred because the laser was locked
when opening the program, and the control loops were not disabled when starting the thin etalon scan. This issue should
now be corrected.

Wavelength axis on single acquisition measurement from CCD is wrong, looks like the data coming out of the screen is
different from the data shown in SOLIS!? In SOLIS, to correct the offset, perform a small -28.9 pixel shift of the axis.
Update: The CCD data was coming out backwards! This is resolved.