Buffersize does not change when sector size is changed

[Sunderlandkyl]

The vtkDataCollectorHardwareDevice works fine on its own, but when the sector size is changed, then the received frame size and the buffer size do not match.

The following code snippet produces the error, where the dataCollector is a pointer of vtkDataCollectorHardwareDevice type.

// Change sector size to 50%
vtkSonixVideoSource* sonixDataCollector= static_cast<vtkSonixVideoSource*> (dataCollector->GetVideoSource() );
sonixDataCollector->SetSector(50);

dataCollector->Start();

121911_172507.717 [ERROR] [016.838000] Received frame size (249604 bytes) doesn't match the buffer size (499204 bytes)! [in ......\PlusLib\src\ImageAcquisition\SonixVideo\vtkSonixVideoSource.cxx(292)]

Migrated from https://app.assembla.com/spaces/plus/tickets/415/details

[Sunderlandkyl]

2011-12-19 22:10
Siavash Khallaghi
Maybe I am using vtkDataCollectorHardwareDevice in a way that I am not supposed to. I am guessing that maybe I should use dataCollector->GetVideoSource()->StartRecording() and StopRecording() instead of Start() and Stop().

2011-12-20 07:08
Andras Lasso
Changing frame size during data collection is currently not supported in Plus. We were thinking about implementing it, but we didn't have any specific application in mind and also it was unclear how to process, use, and store data with varying frame size in general. Also, for the Ultrasonix scanners I think the frame counting was reset to 0 whenever we changed the frame size, so the timestamp filtering was also impacted.

Do you have a couple of different image streams (e.g., a full-frame low-FPS stream and one or more reduced-sector-size high-FPS stream)? How would you like to process and store these different streams? Are the number and properties (frame size, custom transform) of these streams are constant during a recording session (don't change between DataCollector->Start and ->Stop)?

I see these design options to acquire image data with different frame sizes:
If the received pixel data is smaller than the buffer size then copy it inside the buffer => quite simple to implement, but slight performance and memory waste
Stop and start data collection after each frame size change => no PlusLib code change is needed, but the overhead of this may not be acceptable, depending on how frequently you need to change the frame size,
Implementing support for multiple constant streams. It could be useful for interleaved acquisition of high-FPS/low-FPS imaging data, biplane or multi-plane probes, and motorized probes. => would be a nice and generic solution, but it would add some complexity to PlusLib and implementation workload is probably several weeks

2012-01-05 21:09
Siavash Khallaghi
Hi Andras,

Thank you for your comments. Let me explain my situation here. What we are trying to do is to capture all of the data throughout a biopsy procedure. This means that we have three different image streams, one high FPS RF image with 50% sector size, one low FPS RF image with 100% sector size and bmode images whenever we are not acquiring RF.

Right now we have buffer size issues whenever we switch from bmode to RF and also whenever we change the sector size of the RF image.

Resorting to connect and disconnect is the last thing that I want to do, since it will introduce additional delays to the system. What I would like to do is to add a custom function, similar to vtkSonixVideoSource::InternalConnect() which changes the frame size and pixel type. Basically, whenever the sector size is changed the frame size and pixel type are updated. Before I start developing, I would like to ask you if you have any suggestions.

2012-01-05 23:54
Andras Lasso
I would suggest then to have a separate video stream object for each stream in vtkPlusVideoSource (similarly to the tracker, which contains a separate stream for each tracker tool: std::map<std::string, vtkTrackerTool*>).
The video stream object would need to contain a vtkVideoBuffer* and some other members, such as frame rate, frame number, etc.
vtkSonixVideoSource::AddFrameToBuffer would determine which stream the new frame belongs to and would add it to that stream.
Each stream would be saved into files separately (as the frame size and pixel type within a single file is constant).
The streams (stream name, acquisition type, sector size, etc.) should be described in the config xml.

2012-01-06 18:40
Siavash Khallaghi
So basically a new class (let's call it vtkSonixStreams) that would inherit from vtkPlusVideoSource and will have three different streams, i.e. something like the following?

class vtkSonixStreams: public vtkPlusVideoSource
{
public:
 vtkVideoBuffer* buffer1;
 vtkVideoBuffer* buffer2;
 vtkVideoBuffer* buffer3;

 //Other relevant information for each stream, e.g. frame rate, pixel size,...

protected:
 PlusStatus AddNewFrameToSpecificBuffer( int  );
}

and the implementation of AddNewFrameToSpecificBuffer would be the following?

PlusStatus vtkSonixStreams::AddNewFrameToSpecificBuffer(int bufferNumber ) //bufferNumber goes from 1 to 3
{
 switch (bufferNumber)
 {
  case 1:
  //Set frame size, buffer and pixel type for the first buffer (bmode)
  //Call AddFrameToBuffer
  return PLUS_SUCCESS;  

  case 2:
  //Set frame size, buffer and pixel type for the second buffer (50% RF sector size)
  //Call AddFrameToBuffer
  return PLUS_SUCCESS;
  
  case 3:
  //Set frame size, buffer and pixel type for the third buffer (100% RF sector size)
  //Call AddFrameToBuffer
  return PLUS_SUCCESS;

  case else:
  LOG_ERROR( "Could not write new frame to specific buffer!" );
  return PLUS_FAIL;
 }
}

2012-01-06 18:50
Siavash Khallaghi
After five minutes of looking at what I wrote, I really feel that this is not what you mean. Forgive me but I am very confused.

2012-01-10 01:20
Andras Lasso
vtkPlusVideoSource could be extended to handle multiple streams like this:

struct StreamInfo
{
float FrameRate;
int FrameCount;
unsigned long FrameNumber;
double FrameTimeStamp;
vtkVideoBuffer Buffer;
VideoBufferItem CurrentVideoBufferItem;
US_IMAGE_ORIENTATION UsImageOrientation;
}

class VTK_EXPORT vtkPlusVideoSource : public vtkImageAlgorithm
{
public:
virtual PlusStatus SetFrameSize(int x, int y, int streamIndex=0);
virtual int GetFrameSize(int streamIndex=0);
virtual PlusStatus SetPixelType(PlusCommon::ITKScalarPixelType pixelType, int streamIndex=0);
...
protected:
std::vector<StreamInfo> VideoStreams;
...
}

---

Then the video source the implementation would be something like this:

PlusStatus vtkSonixVideoSource::AddFrameToBuffer(void dataPtr, int type, int sz, bool cine, int frmnum)
{
int streamIndex=GetStreamIndex(type, sz, cine, frmnum);
if (streamIndex<0 || streamIndex>...)
{
error...
}
return AddFrameToBuffer(dataPtr, type, sz, cine, frmnum, streamIndex) ;
}

PlusStatus vtkSonixVideoSource::AddFrameToBuffer(void* dataPtr, int type, int sz, bool cine, int frmnum, int streamIndex)
{
add the frame to the streamIndex-th stream
}

2012-01-10 01:31
Siavash Khallaghi
Thank you Andras, you are the greatest!