index.bs

<pre class="metadata">
Shortname: webxr-anchors
Title: WebXR Anchors Module
Group: immersivewebwg
Status: ED
Level: 1
ED: https://immersive-web.github.io/anchors/
Repository: immersive-web/anchors
Mailing List Archives: https://lists.w3.org/Archives/Public/public-immersive-web-wg/

!Participate: <a href="https://github.com/immersive-web/anchors/issues/new">File an issue</a> (<a href="https://github.com/immersive-web/anchors/issues">open issues</a>)
!Participate: <a href="https://lists.w3.org/Archives/Public/public-immersive-web-wg/">Mailing list archive</a>
!Participate: <a href="irc://irc.w3.org:6665/">W3C's #immersive-web IRC</a>

Editor: Piotr Bialecki 114482, Google https://google.com/, bialpio@google.com

Abstract: Describes a method to create anchors tracked by underlying system.
</pre>

<pre class="link-defaults">
spec:webxr device api - level 1; type:dfn; for:/; text:xr device
spec:webxr-ar-module-1; type:enum-value; text:"immersive-ar"
</pre>

<pre class="anchors">
spec: WebXR Device API - Level 1; urlPrefix: https://www.w3.org/TR/webxr/#
    for: XRFrame;
        type: dfn; text: active; url: xrframe-active
        type: dfn; text: session; url: dom-xrframe-session
        type: dfn; text: time; url: xrframe-time
    for: XRSession;
        type: dfn; text: list of frame updates; url: xrsession-list-of-frame-updates
        type: dfn; text: mode; url: xrsession-mode
        type: dfn; text: XR device; url: xrsession-xr-device
    for: XRSpace;
        type: dfn; text: effective origin; url: xrspace-effective-origin
        type: dfn; text: native origin; url: xrspace-native-origin
        type: dfn; text: origin offset; url: xrspace-origin-offset
        type: dfn; text: session; url: xrspace-session
    type: dfn; text: capable of supporting; url: capable-of-supporting
    type: dfn; text: feature descriptor; url: feature-descriptor
    type: dfn; text: identity transform; url: identity-transform
    type: dfn; text: inline XR device; url: inline-xr-device
    type: dfn; text: XR device; url: xr-device
spec: WebXR Hit Test Module; urlPrefix: https://immersive-web.github.io/hit-test/#
    type: interface; text: XRHitTestResult; url: xrhittestresult
    for: XRHitTestResult;
        type: dfn; text: frame; url: xrhittestresult-frame
        type: dfn; text: native origin; url: xrhittestresult-native-origin
    type: dfn; text: hit-test; url: hit-test
</pre>

<style>
  .non-normative::before {
    content: "This section is non-normative.";
    font-style: italic;
  }
  .tg {
    border-collapse: collapse;
    border-spacing: 0;
  }
  .tg th {
    border-style: solid;
    border-width: 1px;
    background: #90b8de;
    color: #fff;
    font-family: sans-serif;
    font-weight: bold;
    border-color: grey;
  }
  .tg td {
    padding: 4px 5px;
    background-color: rgb(221, 238, 255);
    font-family: monospace;
    border-style: solid;
    border-width: 1px;
    border-color: grey;
    overflow: hidden;
    word-break: normal;
  }
  .unstable::before {
    content: "This section is not stable";
    display: block;
    font-weight: bold;
    text-align: right;
    color: red;
  }
  .unstable {
    border: thin solid pink;
    border-radius: .5em;
    padding: .5em;
    margin: .5em calc(-0.5em - 1px);
    background-image: url("data:image/svg+xml,<svg xmlns='http://www.w3.org/2000/svg' width='300' height='290'><text transform='rotate(-45)' text-anchor='middle' font-family='sans-serif' font-weight='bold' font-size='70' y='210' opacity='.1' fill='white'>Unstable</text></svg>");
    background-repeat: repeat;
    background-color: #282828;
  }
  @media (prefers-color-scheme: light) {
    .unstable {
      background-image: url("data:image/svg+xml,<svg xmlns='http://www.w3.org/2000/svg' width='300' height='290'><text transform='rotate(-45)' text-anchor='middle' font-family='sans-serif' font-weight='bold' font-size='70' y='210' opacity='.1'>Unstable</text></svg>");
      background-color: #FFF4F4;
    }
  }
  .unstable h3:first-of-type {
    margin-top: 0.5rem;
  }
  .unstable.example:not(.no-marker)::before {
    content: "Example " counter(example) " (Unstable)";
    float: none;
  }
</style>

Introduction {#intro}
============

<section class="non-normative">

WebXR Anchors module enables applications to ask the underlying XR system to track changes to a particular three-dimensional pose (position and orientation) as the system's understanding of the world changes. This allows the application to adjust the location of the virtual objects that it placed in the scene in a way that helps with maintaining the illusion that the placed objects are really present in the user's environment.

</section>

Terminology {#terminology}
-----------

Pose, as understood by this document, signifies a position and orientation in 3D space.

Anchor, as understood by this document, is an entity that keeps track of the pose that is fixed relative to the real world, and is created by the application.

Initialization {#anchor-feature-initialization}
==================

Feature descriptor {#anchor-feature-descriptor}
------------------

In order for the applications to signal their interest in using anchors during a session, the session must be requested with appropriate [=feature descriptor=]. The string <dfn>anchors</dfn> is introduced by this module as a new valid feature descriptor for anchors feature.

A device is [=capable of supporting=] the anchors feature if the device's tracking system exposes a [[#native-anchor|native anchors]] capability. The [=inline XR device=] MUST NOT be treated as [=capable of supporting=] the anchors feature. The user agents SHOULD attempt to support anchors feature for sessions with [=XRSession/mode=] equal to {{XRSessionMode/"immersive-ar"}}.

Note: Even though anchors feature support is not required, anchors are one of the basic features needed by the AR-enabled applications to provide good user experience. Therefore, it is *highly* recommended that the user agents support them for <code>"immersive-ar"</code> sessions.

Anchors {#anchors-section}
=======

XRAnchor {#xr-anchor}
--------

<script type="idl">
[SecureContext, Exposed=Window]
interface XRAnchor {
  readonly attribute XRSpace anchorSpace;

  Promise<DOMString> requestPersistentHandle();

  undefined delete();
};
</script>

An {{XRAnchor}} contains {{XRAnchor/anchorSpace}} that can be used to locate the anchor relative to other {{XRSpace|XRSpaces}}. If this attribute is accessed when [=XRAnchor/deleted=] is set to <code>true</code>, the user agent MUST throw an {{InvalidStateError}}.

Each {{XRAnchor}} has an associated <dfn for=XRAnchor>deleted</dfn> boolean value that is initially set to <code>false</code>.

Each {{XRAnchor}} has an associated <dfn for=XRAnchor>native origin</dfn>.

Each {{XRAnchor}} has an associated <dfn for=XRAnchor>session</dfn>.

<div class="algorithm" data-algorithm="create-anchor-object">
In order to <dfn>create new anchor object</dfn> from |native origin| and |session|, the user agent MUST run the following steps:
    1. Let |anchor| be a new {{XRAnchor}}.
    1. Set |anchor|'s [=XRAnchor/native origin=] to |native origin|.
    1. Set |anchor|'s [=XRAnchor/session=] to |session|.
    1. Set |anchor|'s [=XRAnchor/deleted=] to <code>false</code>.
    1. Set |anchor|'s {{XRAnchor/anchorSpace}} to a new {{XRSpace}} object created with [=XRSpace/session=] set to |anchor|'s [=XRAnchor/session=] and [=XRSpace/native origin=] set to [=XRAnchor/native origin=].
    1. Return |anchor|.
</div>

<div class="algorithm unstable" data-algorithm="restore-anchor-from-frame">
The {{XRAnchor/requestPersistentHandle()}} method, when invoked on an {{XRAnchor}} |anchor|, MUST run the following steps:
    1. Let |promise| be [=a new Promise=] in the [=relevant realm=] of this {{XRSystem}}.
    1. If |anchor|'s [=XRAnchor/deleted=] is <code>true</code>, [=/reject=] |promise| with {{InvalidStateError}}, return |promise|, and abort these steps.
    1. Let |session| be |anchor|'s [=XRAnchor/session=].
    1. If |session|'s [=XRSession/map of persistent anchors=] contains the value |anchor|, run the following steps:
        1. Let |uuid| be the key of the mapping of the |anchor| value on |session|'s [=XRSession/map of persistent anchors=].
        1. [=/Resolve=] |promise| with |uuid|.
        1. Return |promise|.
        1. Abort these steps.
    1. Let |uuid| be the empty string.
    1. Generate a UUID [[!RFC4122]] as a string and append it to |uuid|.
    1. Add |uuid| and |anchor| to |session|'s [=XRSession/map of persistent anchors=].
    1. [=/Resolve=] |promise| with |uuid|.
    1. Return |promise|.
</div>

Note: when the {{XRSystem}} creates the unique ID, it MUST be unique and only known to the current origin.

Anchor creation {#anchor-creation}
===============

The {{XRSession}} is extended to contain an associated <dfn for=XRSession>set of tracked anchors</dfn>.

<div class="unstable">

The {{XRSession}} is extended to contain an associated <dfn for=XRSession>map of persistent anchors</dfn> that is keyed with a UUID string and maps to an {{XRAnchor}}.

</div>

The {{XRSession}} is extended to contain an associated <dfn for=XRSession>map of new anchors</dfn> that is keyed with {{XRAnchor}} object and maps to <code>{{Promise}}&lt;{{XRAnchor}}&gt;</code> object.

<script type="idl">
partial interface XRFrame {
  Promise<XRAnchor> createAnchor(XRRigidTransform pose, XRSpace space);
};

partial interface XRSession {
  readonly attribute FrozenArray<DOMString> persistentAnchors;

  Promise<XRAnchor> restorePersistentAnchor(DOMString uuid);
  Promise<undefined> deletePersistentAnchor(DOMString uuid);
};

partial interface XRHitTestResult {
  Promise<XRAnchor> createAnchor();
};
</script>

The {{XRHitTestResult}} is extended to contain an associated <dfn for=XRHitTestResult>native entity</dfn>. If the underlying system does not provide information about native entity that resulted in computing the result, it will be assumed that [=XRHitTestResult/native entity=] is set to <code>null</code>.

The application can create an anchor using one of the 2 ways:
- By [=create an anchor from frame|creating an anchor from frame=] - created anchor will not be attached to any particular real world object.
- By [=create an anchor from hit test result|creating an anchor from hit test result=] - created anchor will be attached to a real world object if the underlying [=/XR device=] supports it.

In order to <dfn>create an anchor from frame</dfn>, the application can call {{XRFrame}}'s {{XRFrame/createAnchor(pose, space)}} method.

<div class="algorithm" data-algorithm="create-anchor-from-frame">
The {{XRFrame/createAnchor(pose, space)}} method, when invoked on an {{XRFrame}} |frame| with |pose| and |space|, MUST run the following steps:
    1. Let |promise| be [=a new Promise=].
    1. If |frame|'s [=XRFrame/active=] boolean is <code>false</code>, [=/reject=] |promise| with {{InvalidStateError}}, return |promise|, and abort these steps.
    1. Let |session| be |frame|'s [=XRFrame/session=].
    1. Add [=update anchors=] algorithm to |session|’s [=XRSession/list of frame updates=] if it is not already present there.
    1. Let |device| be |session|'s [=XRSession/XR device=].
    1. Let |effective origin| be |space|'s [=XRSpace/effective origin=].
    1. Let |anchor native origin| be a new native origin returned from the |device|'s call to create a new anchor using |pose|, interpreted as if expressed relative to |effective origin| at the |frame|'s [=XRFrame/time=].
    1. [=Create new anchor object=] |anchor| using |anchor native origin| and |session|.
    1. Add |anchor| to |session|'s [=XRSession/set of tracked anchors=].
    1. Add a mapping from |anchor| to |promise| to |session|'s [=XRSession/map of new anchors=].
    1. Return |promise|.
</div>

Note: It is the responsibility of user agents to ensure that the physical origin tracked by the anchor returned by each {{XRFrame/createAnchor(pose, space)}} call aligns as closely as possible with the physical location of <var ignore=''>pose</var> within <var ignore=''>space</var> at the time represented by the frame on which the method is called. Specifically, this means that for spaces that are dynamically changing, user agents should attempt to capture the native origin of such spaces at the app's specified time. This text is non-normative, but expresses the intent of the specification author(s) and contributors and thus it is highly recommended that it is followed by the implementations to ensure consistent behavior across different vendors.

<div class="algorithm unstable">
{{XRSession/persistentAnchors}} returns a list of the known persistent anchors. This list MUST reflect the keys of the [=XRSession/map of persistent anchors=].
</div>

<div class="algorithm unstable" data-algorithm="restore-persistent-anchor-from-frame">
The {{XRSession/restorePersistentAnchor(uuid)}} method, when invoked on an {{XRSession}} |session| with |uuid|, MUST run the following steps:
    1. Let |promise| be [=a new Promise=] in the [=relevant realm=] of this {{XRSystem}}.
    1. If |session|'s [=XRSession/map of persistent anchors=] does not contain a mapping from |uuid| to an {{XRAnchor}}, [=/reject=] |promise| with {{InvalidStateError}}, return |promise|, and abort these steps.
    1. If |session|’s [=ended=] value is `true`, [=/reject=] |promise| with {{InvalidStateError}}, return |promise|, and abort these steps.
    1. Let |anchor| be the value of mapping from |uuid| on |session|'s [=XRSession/map of persistent anchors=].
    1. If |session|'s [=XRSession/map of new anchors=] contains a mapping from |anchor| to |promise|, [=/reject=] the |promise| with {{InvalidStateError}}, return |promise|, and abort these steps.
    1. Add |anchor| to |session|'s [=XRSession/set of tracked anchors=].
    1. Add a mapping from |anchor| to |promise| to |session|'s [=XRSession/map of new anchors=].
    1. Return |promise|.
</div>

<div class="algorithm unstable" data-algorithm="delete-peristence-from-anchor">
The {{XRSession/deletePersistentAnchor()}} method, when invoked on an {{XRSession}} |session| with |uuid|, MUST run the following steps:
    1. Let |promise| be [=a new Promise=] in the [=relevant realm=] of this {{XRSystem}}.
    1. If |session|'s [=XRSession/map of persistent anchors=] does not contain a mapping to |anchor|, [=/reject=] |promise| with {{InvalidStateError}}, return |promise|, and abort these steps.
    1. Let |anchor| be the value of mapping from |uuid| on |session|'s [=XRSession/map of persistent anchors=].
    1. Remove the mapping from |session|'s [=XRSession/map of persistent anchors=].
    1. Invoke {{XRAnchor/delete()} on |anchor|.
    1. Return |promise|.
</div>

In order to <dfn>create an anchor from hit test result</dfn>, the application can call {{XRHitTestResult}}'s {{XRHitTestResult/createAnchor()}} method.

<div class="algorithm" data-algorithm="create-anchor-from-hit-test-result">
The {{XRHitTestResult/createAnchor()}} method, when invoked on an {{XRHitTestResult}} |hitTestResult|, MUST run the following steps:
    1. Let |promise| be [=a new Promise=].
    1. Let |frame| be |hitTestResult|'s [=XRHitTestResult/frame=].
    1. If |frame|'s [=XRFrame/active=] boolean is <code>false</code>, [=/reject=] |promise| with {{InvalidStateError}}, return |promise|, and abort these steps.
    1. Let |session| be |frame|'s [=XRFrame/session=].
    1. Add [=update anchors=] algorithm to |session|’s [=XRSession/list of frame updates=] if it is not already present there.
    1. Let |device| be |session|'s [=XRSession/XR device=].
    1. Let |nativeEntity| be the |hitTestResult|'s [=XRHitTestResult/native entity=].
    1. Let |anchor native origin| be a new native origin returned from the |device|'s call to create a new anchor located at |hitTestResult|'s [=XRHitTestResult/native origin=] and [=attached=] to |nativeEntity|, at the |frame|'s [=XRFrame/time=].
    1. [=Create new anchor object=] |anchor| using |anchor native origin| and |session|.
    1. Add |anchor| to |session|'s [=XRSession/set of tracked anchors=].
    1. Add a mapping from |anchor| to |promise| to |session|'s [=XRSession/map of new anchors=].
    1. Return |promise|.
</div>

Note: The same remark that is present on {{XRFrame/createAnchor(pose, space)}} method applies here as well.

Issue: Session's "list of frame updates" might need to have a way of specifying ordering - some algorithms may depend on others.

Anchor updates {#anchor-updates}
==============

<script type="idl">
[Exposed=Window]
interface XRAnchorSet {
  readonly setlike<XRAnchor>;
};

partial interface XRFrame {
  [SameObject] readonly attribute XRAnchorSet trackedAnchors;
};
</script>

An {{XRAnchorSet}} interface is introduced as a way to expose a collection of anchors.

An {{XRFrame}} is extended to contain a {{XRFrame/trackedAnchors}} attribute which contains all anchors still tracked in the frame. At {{XRFrame}} creation, the set is initially empty and will be populated by the [=update anchors=] algorithm.

<div class="algorithm" data-algorithm="update-anchors">
In order to <dfn>update anchors</dfn> for |frame|, the user agent MUST run the following steps:
    1. Let |session| be a |frame|'s [=XRFrame/session=].
    1. Let |device| be a |session|'s [=XRSession/XR device=].
    1. For each |anchor| in |session|'s [=XRSession/set of tracked anchors=], run:
        1. Query the |device|'s tracking system for the |anchor|'s [=XRAnchor/native origin=] information.
        1. If the |device|'s tracking system no longer [=tracked|tracks=] the |anchor|, run the following steps:
            1. Remove |anchor| from the |session|'s [=XRSession/set of tracked anchors=].
            1. If |session|'s [=XRSession/map of new anchors=] contains a mapping from |anchor| to |promise|, [=/reject=] the |promise| and remove the mapping.
            1. Continue to the next entry in |session|'s [=XRSession/set of tracked anchors=].
        1. Add |anchor| to |frame|'s {{XRFrame/trackedAnchors}} set.
        1. If |session|'s [=XRSession/map of new anchors=] contains a mapping from |anchor| to |promise|, [=/resolve=] the |promise| with |anchor| and remove the mapping.
</div>

Anchor removal {#anchor-removal}
==============

When an application is no longer interested in receiving updates to an anchor, it can [=delete an anchor=] by calling {{XRAnchor/delete()}}.

<div class="algorithm" data-algorithm="detach-anchor">
The {{XRAnchor/delete()}} method, when invoked on an {{XRAnchor}} |anchor|, MUST <dfn>delete an anchor</dfn> by running the following steps:
    1. If |anchor|'s [=XRAnchor/deleted=] is <code>true</code>, abort these steps.
    1. Set |anchor|'s {{XRAnchor/anchorSpace}} to <code>null</code>.
    1. Set |anchor|'s [=XRAnchor/deleted=] to <code>true</code>.
    1. Let |session| be an |anchor|'s [=XRFrame/session=].
    1. Let |device| be a |session|'s [=XRSession/XR device=].
    1. Let |native origin| be an |anchor|'s [=XRAnchor/native origin=].
    1. Inform the |device|'s tracking system that the |native origin| is no longer to be tracked.
    1. Remove |anchor| from |session|'s [=XRSession/set of tracked anchors=].
</div>

Native device concepts {#native-device-concepts}
======================

<section class="non-normative">

In order for the user agents to be able to implement the anchors API, the underlying XR device should provide native support of this feature. This section describes requirements placed on the underlying XR system, and is by necessity under-specified to allow the implementation of anchors API on top of various frameworks / devices.

</section>

Native anchor {#native-anchor}
-------------

The underlying [=XR device=] is [=capable of supporting=] the [=anchors=] feature if it supports the following capabilities related to anchors:
- Native anchor creation is possible, with the following requirements:
    - The native anchors API attempts to maintain the pose of the anchor as if it were fixed relative to the real world.
    - It accepts pose of the newly created anchor at some specific time <code>t</code>.
    - It optionally accepts a native entity information in order to express the intent of [=attached|attaching=] newly created anchor to the given entity. If the underlying XR system does not support attaching anchors to a native entity (i.e. does not accept a native entity), the newly created anchor will be free-floating.
    - It returns a result that can be treated as a [=native origin=] by the user agents.
- Native anchors are continuously <dfn>tracked</dfn> by the underlying XR system and can be queried for their most up-to-date state. When the underlying system deems that the native anchor's location is never going to be known, it SHOULD report that the native anchor is no longer tracked.
- Optionally, native anchors can be <dfn>attached</dfn> to native entities. A native anchor that is attached to a native entity attempts to maintain its position fixed relative to the entity to which it is attached (as opposed to the real world in case of free-floating anchors). When the device detects that the pose of the object to which the anchor is attached changes, the anchor will be updated accordingly. This does not imply that the object itself moved - it may be that the system's understanding of its location changed. If the underlying system is capable of tracking moving objects, the native anchors attached to moving objects should be updated as well.

In case the underlying device does not support native anchors, the user agents MAY decide to implement the anchors API through <dfn>emulated native anchors</dfn>. This approach assumes that anchors, once created, never change their pose, and their pose is the pose passed in by the application to the anchor creation methods such as {{XRFrame}}'s {{XRFrame/createAnchor(pose, space)}} or {{XRHitTestResult}}'s {{XRHitTestResult/createAnchor()}}.

Note: The [=emulated native anchors=] approach is equivalent to the applications emulating the anchors feature themselves and can help reduce the number of different cases that the applications need to handle. It might also allow the applications to re-use the same code for {{XRSessionMode/"immersive-ar"}} and {{XRSessionMode/"immersive-vr"}} sessions.

Privacy & Security Considerations {#privacy-security}
=================================

<section class="non-normative">

The anchors API does not directly expose access to the information about the user environment. In case the session was created with [=hit-test=] support enabled and the device supports tracking moving objects, it may be possible for the application to infer the layout of the user's environment by observing changes to anchor poses of objects that are, for example, carried around. This offers similar level of information about user's environment as hit-testing API. Anchor feature has to be declared when creating an XR session, which will allow the user agents to notify the users about potential privacy implications of allowing the anchors API to be used by the website.

</section>

Issue: I'm not aware of devices that support tracking general objects that move - should they be considered by the spec at all? Are there any examples of such devices?

Acknowledgements {#ack}
================

The following individuals have contributed to the design of the WebXR Anchors specification: