Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reorganize sources & introduce WASM SDK #130

Merged
merged 132 commits into from
Nov 11, 2023
Merged
Changes from 1 commit
Commits
Show all changes
132 commits
Select commit Hold shift + click to select a range
9d7d143
Add support for multiple unique sessions
Zubnix Apr 18, 2023
1fadb2d
Update program args to support single static session id
Zubnix Apr 18, 2023
52066aa
Begin implementing per proxy session i/o
Zubnix Apr 21, 2023
a47aeb0
make multiple proxy sessions work
Zubnix Apr 24, 2023
be362bb
Make cors work & fix tests
Zubnix Apr 24, 2023
1586f56
make closing remote sessions work properly
Zubnix Apr 25, 2023
8c65932
be able to launch child apps based on http url
Zubnix Apr 30, 2023
5742c85
use a signaling connection per launched app
Zubnix May 16, 2023
3d96b43
Rework proxy app launching
Zubnix May 24, 2023
7e3b232
rename client signaling to app context
Zubnix May 25, 2023
ff2da7b
begin separating compositor-proxy as CLI and as lib
Zubnix May 27, 2023
f9a278d
begin separating compositor-proxy as CLI and as lib
Zubnix May 30, 2023
4c906fb
begin separating compositor-proxy as CLI and as lib (more wip)
Zubnix May 31, 2023
f877355
rub some tailwindcss on the demo compositor
Zubnix Jun 1, 2023
594bd37
rub some more tailwindcss on the demo compositor (wip)
Zubnix Jun 8, 2023
d57369c
rub some more tailwindcss on the demo compositor (wip)
Zubnix Jun 8, 2023
4251927
Fix socket reconnection bug when proxy instances is restarted
Zubnix Jun 8, 2023
53f1ce7
Update deps
Zubnix Jun 12, 2023
1ab708f
Update cosmetics
Zubnix Jun 12, 2023
9d798ca
Update yarn from v1 to v3
Zubnix Jun 12, 2023
dd8de93
Update yarn from v1 to v3
Zubnix Jun 12, 2023
6b8225f
migrate (nearly) all interaction to node websockets
Zubnix Jun 18, 2023
32cabf0
migrate (nearly) all interaction to node websockets
Zubnix Jun 20, 2023
ed48a13
migrate (nearly) all interaction to node websockets
Zubnix Jun 20, 2023
f635021
migrate (nearly) all interaction to node websockets
Zubnix Jun 20, 2023
082b874
add some basic auth and move some code around
Zubnix Jun 22, 2023
1e1c2ba
Begin updating docs
Zubnix Jun 22, 2023
a3aaaaf
updating docs
Zubnix Jun 22, 2023
6883bdf
doc update & chrome webgl fix
Zubnix Jun 27, 2023
e141d48
use a separate app mapping file
Zubnix Jun 27, 2023
6e4494a
Immediately adjust feedback timer ticks
Zubnix Jun 29, 2023
aa74886
Make sure we can decode high level h264 on the client
Zubnix Jun 29, 2023
2cd03f2
Fix XWayland apps not being able to launch as first app
Zubnix Jun 29, 2023
b4c611b
start adding webapp demos
Zubnix Aug 31, 2023
0420517
Start restructuring project
Zubnix Sep 13, 2023
19ae07d
update gitignore
Zubnix Sep 13, 2023
0251464
more yarn workspace configuration
Zubnix Sep 13, 2023
710eda8
more wip
Zubnix Sep 14, 2023
d168c1c
more wip
Zubnix Sep 14, 2023
dc42335
wip
Zubnix Sep 15, 2023
e173681
add sdk to monorepo
Zubnix Sep 15, 2023
a135ad5
move stuff around
Zubnix Sep 15, 2023
e5f25ff
wip
Zubnix Sep 15, 2023
3376b20
wip
Zubnix Sep 16, 2023
3bb4b2a
wip
Zubnix Sep 17, 2023
9ea6c2c
add gitignores
Zubnix Sep 18, 2023
8ad46c4
make sdk work on mac
Zubnix Sep 19, 2023
de14706
make shell deps work
Zubnix Sep 19, 2023
1f3972c
make shell work
Zubnix Sep 19, 2023
1bfa799
make shell work
Zubnix Sep 19, 2023
3dfd18f
fix xtsb build, fix compositor wasm libs
Zubnix Sep 19, 2023
2ca3316
Fix shell worker resolution
Zubnix Sep 19, 2023
e962a1d
Fix shell stuff
Zubnix Sep 19, 2023
92cf76f
more wip
Zubnix Sep 20, 2023
327d61a
more wip
Zubnix Sep 20, 2023
8a97af7
more wip
Zubnix Sep 20, 2023
b84b011
more wip
Zubnix Sep 22, 2023
faa98ce
Make web apps work
Zubnix Sep 23, 2023
eec89ab
Make web apps work
Zubnix Sep 25, 2023
c601160
more wip
Zubnix Sep 25, 2023
7236758
more wip
Zubnix Sep 26, 2023
5178d85
add ffmpeg h264 decoder
Zubnix Sep 26, 2023
d16378e
fix sdk build on linux
Zubnix Sep 27, 2023
943c7a8
streamline webapp & remoteapp launcher
Zubnix Sep 27, 2023
aad5c16
add starter jekyll site
Zubnix Sep 27, 2023
33cbc20
remove ffmpeg git repo
Zubnix Sep 27, 2023
6a433b6
update site config
Zubnix Sep 27, 2023
dc64990
docs site fiddling
Zubnix Sep 28, 2023
6e0979b
update site config
Zubnix Sep 28, 2023
68bff56
update shell
Zubnix Sep 28, 2023
f093e6d
update shell
Zubnix Sep 30, 2023
aa47f07
split out compositor-proxy into lib & cli package
Zubnix Oct 1, 2023
86cb809
more wip
Zubnix Oct 3, 2023
77f28fd
more wip
Zubnix Oct 3, 2023
efe4cbc
more wip
Zubnix Oct 4, 2023
1499500
more wip
Zubnix Oct 4, 2023
52c7f01
begin adding docs site layout
Zubnix Oct 4, 2023
07f6a9e
update gitignore
Zubnix Oct 4, 2023
50f5399
more wip
Zubnix Oct 5, 2023
2a29be2
more wip
Zubnix Oct 5, 2023
24234f0
more wip
Zubnix Oct 5, 2023
7877401
more wip
Zubnix Oct 6, 2023
eaa467e
more wip
Zubnix Oct 6, 2023
03c08a7
more wip
Zubnix Oct 6, 2023
79edc87
more wip
Zubnix Oct 6, 2023
c6ee002
more wip
Zubnix Oct 6, 2023
bab96e6
more wip
Zubnix Oct 7, 2023
5c384ad
more wip
Zubnix Oct 10, 2023
1f7fbe5
more wip
Zubnix Oct 10, 2023
6911592
more wip
Zubnix Oct 10, 2023
8445d6b
more wip
Zubnix Oct 10, 2023
2201841
more wip
Zubnix Oct 11, 2023
ddec939
more wip
Zubnix Oct 11, 2023
3714025
more wip
Zubnix Oct 11, 2023
b8fd366
more wip
Zubnix Oct 11, 2023
02cb0fd
more wip
Zubnix Oct 13, 2023
0cd7e6a
more wip
Zubnix Oct 13, 2023
6cc2c8a
more wip
Zubnix Oct 13, 2023
f3f8ed6
more wip
Zubnix Oct 15, 2023
ec73f88
more wip
Zubnix Oct 16, 2023
ae3d428
more wip
Zubnix Oct 21, 2023
377095b
more wip
Zubnix Oct 22, 2023
0a71d39
more wip
Zubnix Oct 22, 2023
74a4402
more wip
Zubnix Oct 24, 2023
ba39d7a
more wip
Zubnix Oct 24, 2023
d885a09
more wip
Zubnix Nov 11, 2023
2f2eecb
more wip
Zubnix Oct 25, 2023
60bcc34
more wip
Zubnix Oct 25, 2023
b6f4332
more wip
Zubnix Oct 25, 2023
05ffbb4
more wip
Zubnix Oct 26, 2023
2de66d0
more wip
Zubnix Oct 27, 2023
71da5ee
more wip
Zubnix Oct 27, 2023
3e1696a
more wip
Zubnix Oct 31, 2023
eb7952f
more wip
Zubnix Oct 31, 2023
fe8b98d
more wip
Zubnix Oct 31, 2023
4dc0f06
more wip
Zubnix Oct 31, 2023
cd5a8f0
more wip
Zubnix Nov 1, 2023
e2e0972
more wip
Zubnix Nov 1, 2023
fbf9db7
more wip
Zubnix Nov 2, 2023
ad71c85
more wip
Zubnix Nov 2, 2023
53f7853
more wip
Zubnix Nov 2, 2023
0cc8f0b
more wip
Zubnix Nov 2, 2023
8fb3399
more wip
Zubnix Nov 3, 2023
4f851ae
more wip
Zubnix Nov 4, 2023
a8940d5
more wip
Zubnix Nov 4, 2023
ccd146b
more wip
Zubnix Nov 5, 2023
8d3ca26
more wip
Zubnix Nov 6, 2023
f4e2132
more wip
Zubnix Nov 6, 2023
3c7b92e
more wip
Zubnix Nov 7, 2023
a760722
more wip
Zubnix Nov 8, 2023
62e4f2e
Create CNAME
Zubnix Nov 11, 2023
b0e6ca1
more wip
Zubnix Nov 11, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
be able to launch child apps based on http url
  • Loading branch information
Zubnix committed Nov 11, 2023
commit 8c6593251a7dc881b2ce31609f7b08107a48b901
31 changes: 26 additions & 5 deletions compositor-module/compositor-proxy-api.yaml
Original file line number Diff line number Diff line change
@@ -12,7 +12,7 @@ components:
ApiKeyAuth:
type: apiKey
in: header
name: X-Proxy-Identity-Id
name: X-Greenfield-Proxy-Session-Key
responses:
UnauthorizedError:
description: Authentication information is missing or invalid
@@ -30,7 +30,7 @@ components:
- handle
- type
- host
- identity
- proxySessionKey
properties:
handle:
description: The native FD
@@ -57,13 +57,34 @@ components:
format: uri
pattern: '^(https?)://'
readOnly: true
identity:
description: id of the proxy this ProxyFD originated from
proxySessionKey:
description: key of the proxy session this ProxyFD originated from
type: string
readOnly: true

paths:
/{clientId}/{surfaceId}/encoder/keyframe:
/session:
post:
tags:
- proxy
summary: Create a new proxy session
operationId: session
responses:
201:
description: Session was successfully created.
content:
application/json:
schema:
type: object
required:
- sessionKey
properties:
sessionKey:
description: The proxy session id
type: string
readOnly: true

/client/{clientId}/surface/{surfaceId}/encoder/keyframe:
post:
tags:
- encoder
2 changes: 1 addition & 1 deletion compositor-module/demo-compositor/src/ProxyConnection.tsx
Original file line number Diff line number Diff line change
@@ -23,7 +23,7 @@ export type ProxyConnectionProps = {
proxyListener: RemoteClientConnectionListener
clients: Signal<ClientProps[]>
remove: () => void
identity: string
proxySessionKey: string
}
export function ProxyConnection(props: ProxyConnectionProps) {
const [connectionState, setConnectionState] = useState(props.proxyListener.state)
10 changes: 5 additions & 5 deletions compositor-module/demo-compositor/src/ProxyConnector.tsx
Original file line number Diff line number Diff line change
@@ -16,10 +16,10 @@ function addConnection(
compositorProxyConnector: RemoteCompositorConnector,
clients: Signal<ClientProps[]>,
) {
const url = new URL(`ws://${connectionURL.value}`)
const url = new URL(connectionURL.value)
url.searchParams.append('compositorSessionId', session.compositorSessionId)

const proxyListener = compositorProxyConnector.listen(url)
const proxyListener = compositorProxyConnector.launch(url)
const proxyConnectionProps: ProxyConnectionProps = {
session,
url,
@@ -29,13 +29,13 @@ function addConnection(
removeConnection(proxyConnectionProps, proxyListener)
},
clients,
identity: proxyListener.remoteIdentity ?? '',
proxySessionKey: proxyListener.proxySessionKey ?? '',
}

connections.value = [...connections.value, proxyConnectionProps]

proxyListener.remoteIdentityChanged = (remoteIdentity) => {
proxyConnectionProps.identity = remoteIdentity
proxyConnectionProps.proxySessionKey = remoteIdentity
connections.value = [...connections.value]
}
}
@@ -68,7 +68,7 @@ export function ProxyConnector(props: ProxyConnectorProps) {
<ul>
{connections.value.map((connection) => (
<li>
<ProxyConnection key={connection.identity} {...connection} />
<ProxyConnection key={connection.proxySessionKey} {...connection} />
</li>
))}
</ul>
2 changes: 1 addition & 1 deletion compositor-module/demo-compositor/src/WebAppLauncher.tsx
Original file line number Diff line number Diff line change
@@ -12,7 +12,7 @@ function launchWebApp(
clients: Signal<ClientProps[]>,
) {
const url = new URL(webAppURL.value)
const webAppListener = webConnector.listen(url)
const webAppListener = webConnector.launch(url)
const webApp: WebAppProps = { url, onClose: () => webAppListener.close(), loaded: new Signal(false) }
webApps.value = [...webApps.value, webApp]
webAppListener.onClose = () => {
6 changes: 3 additions & 3 deletions compositor-module/src/index.ts
Original file line number Diff line number Diff line change
@@ -87,7 +87,7 @@ export interface ClientConnectionListener {
export interface RemoteClientConnectionListener extends ClientConnectionListener {
readonly state: 'closed' | 'closing' | 'connecting' | 'open'
readonly type: 'remote'
readonly remoteIdentity?: string
readonly proxySessionKey?: string

remoteIdentityChanged: (remoteIdentity: string) => void
onConnectionStateChange: (state: 'closed' | 'open') => void
@@ -102,12 +102,12 @@ export interface WebClientConnectionListener extends ClientConnectionListener {
}

export interface RemoteCompositorConnector {
listen(url: URL, auth?: string): RemoteClientConnectionListener
launch(url: URL, auth?: string): RemoteClientConnectionListener
readonly type: 'remote'
}

export interface WebCompositorConnector {
listen(url: URL): WebClientConnectionListener
launch(url: URL): WebClientConnectionListener
readonly type: 'web'
}

59 changes: 14 additions & 45 deletions compositor-module/src/remote/RemoteConnectionHandler.ts
Original file line number Diff line number Diff line change
@@ -27,48 +27,11 @@ import XWaylandShell from './xwayland/XWaylandShell'
import { XWindowManager } from './xwayland/XWindowManager'
import { XWindowManagerConnection } from './xwayland/XWindowManagerConnection'
import { Configuration, EncoderApi, ProxyFD } from '../api'
import { ClientEncodersFeedback, createClientEncodersFeedback } from './EncoderFeedback'
import { createClientEncodersFeedback } from './EncoderFeedback'
import { deliverContentToBufferStream } from './BufferStream'
import { createRemoteInputOutput } from './RemoteInputOutput'
import type { Channel } from './Channel'

const alphabet = 'ABCDEFGHIJKLMNOPQRSTUVWXYZ234567' as const

export type RemoteClientContext = {
clientEncoderFeedback: ClientEncodersFeedback
encoderApi: EncoderApi
}

function base32Encode(data: Uint8Array) {
const view = new DataView(data.buffer, data.byteOffset, data.byteLength)

let bits = 0
let value = 0
let output = ''

for (let i = 0; i < view.byteLength; i++) {
value = (value << 8) | view.getUint8(i)
bits += 8

while (bits >= 5) {
output += alphabet[(value >>> (bits - 5)) & 31]
bits -= 5
}
}

if (bits > 0) {
output += alphabet[(value << (5 - bits)) & 31]
}

return output
}

export function randomString(): string {
const randomBytes = new Uint8Array(16)
window.crypto.getRandomValues(randomBytes)
return `ra${base32Encode(randomBytes).toLowerCase()}`
}

export function isProxyFD(fd: any): fd is ProxyFD {
return typeof fd?.handle === 'number' && typeof fd?.host === 'string' && typeof fd?.type === 'string'
}
@@ -83,7 +46,16 @@ export class RemoteConnectionHandler {
return new RemoteConnectionHandler(session)
}

onProtocolChannel(protocolChannel: Channel, compositorProxyURL: URL, client: Client, proxyIdentityId: string): void {
onProtocolChannel(
protocolChannel: Channel,
proxySessionProps: {
proxySessionKey: string
baseURL: string
proxySessionSignalURL: string
},
client: Client,
proxySessionKey: string,
): void {
this.session.logger.info('[ProtocolChannel] - created.')
let wasOpen = protocolChannel.isOpen
protocolChannel.onClose = () => {
@@ -144,21 +116,18 @@ export class RemoteConnectionHandler {
id: client.id,
})

const protocol = compositorProxyURL.protocol === 'wss:' ? 'https:' : 'http:'
const pathname = compositorProxyURL.pathname === '/' ? '' : compositorProxyURL.pathname
const basePath = `${protocol}//${compositorProxyURL.host}${pathname}`
const encoderApi = new EncoderApi(
new Configuration({
basePath,
basePath: proxySessionProps.baseURL,
headers: {
['x-proxy-identity-id']: proxyIdentityId,
['x-greenfield-proxy-session-key']: proxySessionKey,
},
}),
)
client.userData = {
encoderApi,
clientEncodersFeedback: createClientEncodersFeedback(client.id, encoderApi),
inputOutput: createRemoteInputOutput(basePath, proxyIdentityId),
inputOutput: createRemoteInputOutput(proxySessionProps.baseURL, proxySessionKey),
}
}

Loading