Tutorial 2: Getting started with LSL (multiple streams)
In this tutorial, you learn to record two streams simultaneously, one data stream (here Audio), and one marker stream (key presses)
For this tutorial you need
- A Windows Computer with a microphone
- MATLAB (for visualization and data import)
- The LSL MATLAB Viewer (for visualizing the stream)
- The LSL MATLAB Importer (for loading the data into MATLAB)
- The LSL LabRecorder version 12.2b (for data acquisition)
- The LSL AudioCaptureWin App (for streaming the microphone input)
- The LSL Keyboard App (for streaming key presses and releases)
The LSL App can be downloaded from the ftp as .zip files.
- Start LabRecorder.exe
- Start AudioCaptureWin.exe
- Start Keyboard.exe
Now we conduct a little experiment in which we record button presses and microphone input
-
Start a new recording in the LabRecorder: Click 'Start'
-
Repeat the following steps for all letters on the middle row of the keyboard (from ‘a’ to ‘l’, on a QWERTY keyboard)
- Press down the key on the keyboard
- Pronounce the letter
- Release the key (make sure that you release the key once you are finished pronouncing this letter).
-
Stop the recording
-
In MATLAB change to the directory of MATLAB Importer
- Load the .xdf file you just recorded:
data=load_xdf(‘C:\yourfilepath\’)
- Load the .xdf file you just recorded:
The load_xdf function provides several advanced options for time synchronization, support for breaks in the data, as well as other defects (see help load_xdf), that is beyond this tutorial.
- You should see that the resulting cell array contains two entries
1×2 cell array
{1×1 struct} {1×1 struct}
type data{1} to see the first stream. The info structure contains additional information that describes this stream in more detail (see below), the time series (i.e. the actual audio data, here a mono recording) and the corresponding time stamps (i.e. for each data sample you have the exact time when this sample was acquired.
info: [1×1 struct]
segments: [1×1 struct]
time_series: [1×667233 single]
time_stamps: [1×667233 double]
Here the first stream is the Audio (The order of streams could be different for your experiment).
type data{1}.info to see more details. You see the name and type of the stream, the nominal_srate (i.e. the sampling rate the device is intending to send), the source_id etc.
struct with fields:
name: 'AudioCaptureWin'
type: 'Audio'
channel_count: '1'
nominal_srate: '44100'
channel_format: 'float32'
source_id: 'DESKTOP-I9R6TCB{0.0.1.00000000}.{9349f85e-b1ac-40d9-a83f-3c3cf9edbd91}'
version: '1.1000000000000001'
created_at: '437615.75132321799'
uid: 'fdb68f1e-c192-486c-a289-871506575d60'
session_id: 'default'
hostname: 'DESKTOP-I9R6TCB'
v4address: [1×1 struct]
v4data_port: '16573'
v4service_port: '16579'
v6address: [1×1 struct]
v6data_port: '16573'
v6service_port: '16579'
desc: [1×1 struct]
clock_offsets: [1×1 struct]
first_timestamp: '438262.4887584534'
last_timestamp: '438277.6188940192'
sample_count: '667233'
effective_srate: 4.4099e+04
type data{2} to see the second stream. The second stream is the Keyboard stream. In time_series you find which key was used and whether it was pressed or released. In time_stamps you find the corresponding time steps of when the key was pressed or released.
info: [1×1 struct]
time_series: {1×20 cell}
time_stamps: [1×20 double]
type data{2}.info to see more details.
struct with fields:
name: 'Keyboard'
type: 'Markers'
channel_count: '1'
nominal_srate: '0'
channel_format: 'string'
source_id: 'KeyboardCapture_DESKTOP-I9R6TCB'
version: '1.1000000000000001'
created_at: '437621.11354082898'
uid: 'ea8b74eb-7101-4543-87f3-1337ecabea6b'
session_id: 'default'
hostname: 'DESKTOP-I9R6TCB'
v4address: [1×1 struct]
v4data_port: '16574'
v4service_port: '16581'
v6address: [1×1 struct]
v6data_port: '16574'
v6service_port: '16581'
desc: [1×1 struct]
clock_offsets: [1×1 struct]
first_timestamp: '438265.853400029'
last_timestamp: '438276.141606314'
sample_count: '20'
Check the first 6 entries of your marker stream by typing data{2}.time_series(1:6) you should see something like:
1×6 cell array
{'A pressed'} {'A released'} {'S pressed'} {'S released'} {'D pressed'} {'D released'}
To visualize the results you can type: Note, the order of streams might be reversed in your case.
figure;
plot(data{1}.time_stamps, data{1}.time_series)
hold all
plot(data{2}.time_stamps, zeros(size(data{2}.time_stamps)),'o', 'LineWidth',2)
The audio data is shown in blue. You can nicely see the moments when a letter was pronounced. The orange markers indicate key presses and releases (not differentiated here). The key was press before the utterance of the letter and released right after.
In the following we make the plot a bit nicer. To get an easier interpretable x-axis we subtract from both time stamp vectors the first value, so that the time vector starts at zero. Note, we have to subtract the save value from both so that their respective time axis correspond (in this case we use the first time stamp of the audio stream). To differentiate key presses and releases more easily we plot them with different symbols. It is save to asume here that the odd entries in our vector correspond to the button presses and the even values to the releases.
figure;
plot(data{1}.time_stamps-data{1}.time_stamps(1), data{1}.time_series)
hold all
plot(data{1}.time_stamps(1:2:end)-data{1}.time_stamps(1), zeros(size(data{2}.time_stamps(1:2:end)))-0.1,'o', 'LineWidth',2)
plot(data{2}.time_stamps(2:2:end)-data{1}.time_stamps(1), zeros(size(data{2}.time_stamps(1:2:end)))+0.1,'v', 'LineWidth',2)
legend({'Audio', 'Key press','Key release'})
xlabel('Time in seconds')
For EEGlab users there is and xdf_import plugin.
- Start EEG lab
- You can load the data either from the gui or the command line
- EEG=pop_loadxdf(filepath,’streamtype’, ‘Audio’)
- Specify in this case that the stream has the type ‘Audio’
- With the option ‘exclude_markerstreams’ you can select to exclude some marker streams if needed
- You now can use the standard functionality of EEGLab such as data visualization or epoching