OrientedPoint conventions

[jojoelfe]

Hi,

I've been trying to understand how the xyz_rotation_matrix field of the OrientedPoint Annotation works. As an example I've been using this run:

https://cryoetdataportal.czscience.com/runs/506

And I'm looking at the 9th point:

{"type": "orientedPoint", "location": {"x": 321.4944515, "y": 278.835509, "z": 144.076534}, "xyz_rotation_matrix": [[0.6324349759945438, -0.7008336330863288, 0.3299366907056534], [-0.7701264082642606, -0.5231035231397906, 0.3650589258910513], [-0.08325452800160772, -0.48496899160024204, -0.8705594527391417]]}

In this example it looks like the spike is pointing down, so the the symmetry axis should roughly go along (0,-1,0),

Since in the reconstruction the symmetry axis goes along Z, my assumption was that I should get the direction vector by multiplying the matrix with (0,0,1):

$$ {\left\lbrack \matrix{0.632 & -0.700 & 0.329 \cr -0.770 & -0.523 & 0.365 \cr -0.08 & -0.484 & -0.870} \right\rbrack} * \left\lbrack \matrix{0 \cr 0 \cr 1} \right\rbrack = \left\lbrack \matrix{0.329 \cr 0.365 \cr -0.87} \right\rbrack $$

Clearly that does not look right.

Is there anything I am misunderstanding? Is there a place where the math is described? Alternatively, are there somewhere the original RELION eulers and/or the code used to convert them to the matrix?

Thanks,

Johannes

[jojoelfe]

After some tinkering, it seems like running np.flip on the matrix makes things look reasonable:

$$ {\left\lbrack \matrix{ -0.870& -0.484 & -0.08 \cr 0.365 & -0.523 & -0.770 \cr 0.329 & -0.700 & 0.632} \right\rbrack} * \left\lbrack \matrix{0 \cr 0 \cr 1} \right\rbrack = \left\lbrack \matrix{-0.08 \cr -0.77 \cr 0.632} \right\rbrack $$

Maybe this is the old numpy zyx vs xyz problem?

[uermel]

Hi @jojoelfe, indeed the rotation matrices are in ZYX-order at the moment.

In general, the rotation matrices parametrize a transformation from 'average density'-space to 'tomogram'-space, but were ingested with flipped rotation matrices during the initial import. This is fixed by chanzuckerberg/cryoet-data-portal-backend#63 and chanzuckerberg/cryoet-data-portal-backend#95

Annotations with corrected rotation matrices will be available next week, pending syncing the correctly imported points to the portal's public bucket.

In the meantime, please find the STAR-file source for this dataset here: https://gist.github.com/uermel/8b7f43207b972f8fe13251eaa6c44856

[BradyAJohnston]

Is there some documentation for what data / annotations are possible to be supplied in the .ndjson files? We have been working on a reader into Blender and I haven't been able to find spec for what possible data might come up in comparison to .star files.

[uermel]

Hi @BradyAJohnston, excited to hear about the reader!

Unfortunately there is no formal documentation yet, but I can point you towards the implementation. All point formats are defined here, they can either contain only a location in pixels of the associated tomogram (Point), a location and rotation matrix (OrientedPoint) or a location and an ID (InstancePoint). The rotation matrix specifies the transformation from average density coordinate frame to tomogram coordinate frame.

Compare this conversion for how we ingest STAR files.

[uermel]

@jojoelfe @BradyAJohnston Hi both, the rotation matrices have now been updated in the datasets to reflect the conventions as discussed above.