[QUESTION] How to stream an existing h265 encoded video over webRTC? #2072

CarlosYeverino · 2024-10-28T18:54:51Z

CarlosYeverino
Oct 28, 2024

How to stream an existing h265 encoded video/frames over webRTC?
I modified kvsWebrtcClientMasterGstSample.c to use a custom senderPipeline which uses multifilesrc to load the existing h265SampleFrames from the repo (goal is to have similar use case but with different data).

senderPipeline = gst_parse_launch("multifilesrc location=\"./h265SampleFrames/frame-%04d.h265\" index=1 loop=true "
                                                          "caps=\"video/x-h265,stream-format=byte-stream,alignment=au,framerate=25/1\" ! "
                                                          "h265parse ! appsink sync=TRUE emit-signals=TRUE name=appsink-video",
                                                          &error);

I noticed that connection to peer is successful, however, the data is not streamed since there is no indication about it on kvsWebrtcClientMasterGstSample output nor in AWS test page and webRTC internals. How can I make this use case work?

NOTE: I confirmed that webrtc with testsrc h265 works, I am able to see the streamed video on AWS test page on Windows Chrome. Furthermore, I tested similar pipeline as the custom senderPipeline locally with autovideosink and I was able to see the video (not webRTC here).

gst-launch-1.0 multifilesrc location="./h265SampleFrames/frame-%04d.h265" index=1 loop=true caps="video/x-h265,stream-format=byte-stream,alignment=au,framerate=25/1" ! h265parse ! avdec_h265 ! videoconvert ! autovideosink sync=true

kvsWebrtcClientMasterGstSample output

yeverino@yeverino-pc:~/Documents/git/aws_webrtc_h265/amazon-kinesis-video-streams-webrtc-sdk-c/build$ ./samples/kvsWebrtcClientMasterGstSample MySignalChannel video-only filesrc h265
2024-10-28 18:58:21.231 PROFILE createRtcCertificate(): [Certificate creation time] Time taken: 0 ms
2024-10-28 18:58:21.238 INFO    main(): [KVS Gstreamer Master] Finished initializing GStreamer and handlers
2024-10-28 18:58:21.238 INFO    main(): [KVS Gstreamer Master] Streaming video only
2024-10-28 18:58:21.238 INFO    main(): [KVS GStreamer Master] Using file source in GStreamer
2024-10-28 18:58:21.238 INFO    main(): [KVS GStreamer Master] streaming type video-only
2024-10-28 18:58:21.238 INFO    initKvsWebRtc(): Initializing WebRTC library...
2024-10-28 18:58:21.240 INFO    initKvsWebRtc(): SDK version: 30f52d18e647747de1ec29ad6f6d3f62128f5494
2024-10-28 18:58:21.240 INFO    main(): [KVS GStreamer Master] KVS WebRTC initialization completed successfully
2024-10-28 18:58:21.240 INFO    createSignalingClientSync(): Creating Signaling Client Sync
[2024/10/28 19:58:21:2405] N: lws_create_context: LWS: 4.3.3-v4.3.3, NET CLI H1 H2 WS ConMon IPv6-absent
[2024/10/28 19:58:21:2405] N: __lws_lc_tag:  ++ [wsi|0|pipe] (1)
[2024/10/28 19:58:21:2405] N: __lws_lc_tag:  ++ [vh|0|netlink] (1)
[2024/10/28 19:58:21:2405] N: __lws_lc_tag:  ++ [vh|1|default||-1] (2)
2024-10-28 18:58:21.240 PROFILE executeGetTokenSignalingState(): [Get token call] Time taken: 0 ms
2024-10-28 18:58:21.240 PROFILE createSignalingClientSync(): [Create signaling client] Time taken: 0 ms
2024-10-28 18:58:21.240 INFO    signalingClientFetchSync(): Signaling Client Fetch Sync
2024-10-28 18:58:21.240 PROFILE executeDescribeSignalingState(): [Describe signaling call] Time taken: 0 ms
2024-10-28 18:58:21.240 PROFILE executeGetEndpointSignalingState(): [Get endpoint signaling call] Time taken: 0 ms
2024-10-28 18:58:21.240 INFO    lwsWssCallbackRoutine(): WSS callback with reason 27
[2024/10/28 19:58:21:2408] N: __lws_lc_tag:  ++ [wsicli|0|POST/h1/default/r-c50d6457.kinesisvideo.us-east-1.am] (1)
2024-10-28 18:58:21.242 INFO    lwsWssCallbackRoutine(): WSS callback with reason 71
2024-10-28 18:58:21.242 INFO    lwsWssCallbackRoutine(): WSS callback with reason 71
2024-10-28 18:58:21.242 INFO    lwsWssCallbackRoutine(): WSS callback with reason 71
[2024/10/28 19:58:21:8532] N: __lws_lc_untag:  -- [wsicli|0|POST/h1/default/r-c50d6457.kinesisvideo.us-east-1.am] (0) 612.368ms
2024-10-28 18:58:21.853 PROFILE executeGetIceConfigSignalingState(): [Get ICE config signaling call] Time taken: 613 ms
2024-10-28 18:58:21.853 PROFILE signalingClientFetchSync(): [Fetch signaling client] Time taken: 612 ms
2024-10-28 18:58:21.853 INFO    lwsWssCallbackRoutine(): WSS callback with reason 85
[2024/10/28 19:58:21:8538] N: __lws_lc_tag:  ++ [wsicli|1|WS/h1/default/m-b94e17e0.kinesisvideo.us-east-1.amaz] (1)
2024-10-28 18:58:21.857 INFO    lwsWssCallbackRoutine(): WSS callback with reason 105
2024-10-28 18:58:21.857 INFO    lwsWssCallbackRoutine(): WSS callback with reason 19
2024-10-28 18:58:21.962 INFO    lwsWssCallbackRoutine(): WSS callback with reason 29
2024-10-28 18:58:22.162 INFO    lwsWssCallbackRoutine(): WSS callback with reason 24
2024-10-28 18:58:22.231 PROFILE createRtcCertificate(): [Certificate creation time] Time taken: 0 ms
2024-10-28 18:58:22.367 INFO    lwsWssCallbackRoutine(): WSS callback with reason 44
2024-10-28 18:58:22.367 INFO    lwsWssCallbackRoutine(): WSS callback with reason 2
2024-10-28 18:58:22.367 PROFILE executeConnectSignalingState(): [Connect signaling call] Time taken: 514 ms
2024-10-28 18:58:22.367 PROFILE signalingClientConnectSync(): [Connect signaling client] Time taken: 514 ms
2024-10-28 18:58:22.367 PROFILE initSignaling(): [Signaling Get token] 0 ms
2024-10-28 18:58:22.367 PROFILE initSignaling(): [Signaling Describe] 0 ms
2024-10-28 18:58:22.367 PROFILE initSignaling(): [Signaling Describe Media] 0 ms
2024-10-28 18:58:22.367 PROFILE initSignaling(): [Signaling Create Channel] 0 ms
2024-10-28 18:58:22.367 PROFILE initSignaling(): [Signaling Get endpoint] 0 ms
2024-10-28 18:58:22.367 PROFILE initSignaling(): [Signaling Get ICE config] 613 ms
2024-10-28 18:58:22.367 PROFILE initSignaling(): [Signaling Connect] 514 ms
2024-10-28 18:58:22.367 PROFILE initSignaling(): [Signaling create client] 0 ms
2024-10-28 18:58:22.367 PROFILE initSignaling(): [Signaling fetch client] 612 ms
2024-10-28 18:58:22.367 PROFILE initSignaling(): [Signaling connect client] 514 ms
2024-10-28 18:58:22.367 INFO    main(): [KVS GStreamer Master] Channel MySignalChannel set up done 
2024-10-28 18:58:23.231 PROFILE createRtcCertificate(): [Certificate creation time] Time taken: 0 ms
2024-10-28 18:58:27.793 PROFILE createDtlsSession(): [Create SSL Context] Time taken: 0 ms
2024-10-28 18:58:27.793 PROFILE createDtlsSession(): [Create SSL session] Time taken: 0 ms
2024-10-28 18:58:27.793 PROFILE createPeerConnection(): [Create DTLS Session object] Time taken: 0 ms
2024-10-28 18:58:27.793 PROFILE createPeerConnection(): [Create connection listener] Time taken: 0 ms
2024-10-28 18:58:27.793 INFO    iceAgentValidateKvsRtcConfig(): 
	iceLocalCandidateGatheringTimeout: 10000 ms
	iceConnectionCheckTimeout: 12000 ms
	iceCandidateNominationTimeout: 12000 ms
	iceConnectionCheckPollingInterval: 50 ms
2024-10-28 18:58:27.794 WARN    onSetStunServerIp(): WebRTC Client object Object not initialized
2024-10-28 18:58:27.794 INFO    getIpWithHostName(): ICE SERVER Hostname received: stun.kinesisvideo.us-east-1.amazonaws.com
2024-10-28 18:58:27.794 WARN    getIpAddrFromDnsHostname(): Received unexpected hostname format: stun.kinesisvideo.us-east-1.amazonaws.com
2024-10-28 18:58:27.794 WARN    getIpWithHostName(): Parsing for address failed for stun.kinesisvideo.us-east-1.amazonaws.com, fallback to getaddrinfo
2024-10-28 18:58:27.797 PROFILE parseIceServer(): ICE Server address for stun.kinesisvideo.us-east-1.amazonaws.com: 34.234.148.89
2024-10-28 18:58:27.797 PROFILE createIceAgent(): [ICE server parsing] Time taken: 3 ms
2024-10-28 18:58:27.797 INFO    getIpWithHostName(): ICE SERVER Hostname received: 3-91-79-248.t-99577840.kinesisvideo.us-east-1.amazonaws.com
2024-10-28 18:58:27.797 PROFILE parseIceServer(): ICE Server address for 3-91-79-248.t-99577840.kinesisvideo.us-east-1.amazonaws.com: 3.91.79.248
2024-10-28 18:58:27.797 PROFILE createIceAgent(): [ICE server parsing] Time taken: 0 ms
2024-10-28 18:58:27.797 INFO    getIpWithHostName(): ICE SERVER Hostname received: 3-91-79-248.t-99577840.kinesisvideo.us-east-1.amazonaws.com
2024-10-28 18:58:27.797 PROFILE parseIceServer(): ICE Server address for 3-91-79-248.t-99577840.kinesisvideo.us-east-1.amazonaws.com: 3.91.79.248
2024-10-28 18:58:27.797 PROFILE createIceAgent(): [ICE server parsing] Time taken: 0 ms
2024-10-28 18:58:27.797 INFO    getIpWithHostName(): ICE SERVER Hostname received: 3-91-79-248.t-99577840.kinesisvideo.us-east-1.amazonaws.com
2024-10-28 18:58:27.797 PROFILE parseIceServer(): ICE Server address for 3-91-79-248.t-99577840.kinesisvideo.us-east-1.amazonaws.com: 3.91.79.248
2024-10-28 18:58:27.797 PROFILE createIceAgent(): [ICE server parsing] Time taken: 0 ms
2024-10-28 18:58:27.797 PROFILE createPeerConnection(): [Create ICE agent object] Time taken: 3 ms
2024-10-28 18:58:27.797 PROFILE createPeerConnection(): [Peer connection object creation time] Time taken: 4 ms
2024-10-28 18:58:27.799 PROFILE iceAgentStartGathering(): [Host candidate gathering from local interfaces] Time taken: 0 ms
2024-10-28 18:58:27.800 PROFILE iceAgentStartGathering(): [Host candidates setup time] Time taken: 0 ms
2024-10-28 18:58:27.800 PROFILE iceAgentStartGathering(): [Srflx candidates setup time] Time taken: 0 ms
2024-10-28 18:58:27.800 PROFILE iceAgentStartGathering(): [Relay candidates setup time] Time taken: 0 ms
2024-10-28 18:58:27.801 INFO    lwsWssCallbackRoutine(): WSS callback with reason 71
2024-10-28 18:58:27.801 PROFILE signalingSendMessageSync(): [Offer Received to Answer Sent time] Time taken: 9 ms
2024-10-28 18:58:27.801 PROFILE sendSignalingMessage(): [Signaling offer received to answer sent time] 9 ms
2024-10-28 18:58:27.802 ERROR   iceAgentAddRemoteCandidate(): operation returned status code: 0x5a000003
2024-10-28 18:58:27.802 ERROR   iceAgentAddRemoteCandidate(): operation returned status code: 0x5a000003
2024-10-28 18:58:27.802 INFO    stepIceAgentStateMachine(): Ice agent state changed from ICE_AGENT_STATE_NEW to ICE_AGENT_STATE_CHECK_CONNECTION.
2024-10-28 18:58:27.802 INFO    onConnectionStateChange(): New connection state 2
2024-10-28 18:58:27.803 WARN    socketConnectionIsConnected(): socket connection check failed with errno Operation already in progress(114)
2024-10-28 18:58:27.803 INFO    lwsWssCallbackRoutine(): WSS callback with reason 71
2024-10-28 18:58:27.804 INFO    lwsWssCallbackRoutine(): WSS callback with reason 71
2024-10-28 18:58:27.804 INFO    lwsWssCallbackRoutine(): WSS callback with reason 71
2024-10-28 18:58:27.853 WARN    socketConnectionIsConnected(): socket connection check failed with errno Operation already in progress(114)
2024-10-28 18:58:27.897 ERROR   iceAgentAddRemoteCandidate(): operation returned status code: 0x5a000003
2024-10-28 18:58:27.897 ERROR   iceAgentAddRemoteCandidate(): operation returned status code: 0x5a000003
2024-10-28 18:58:27.903 WARN    socketConnectionIsConnected(): socket connection check failed with errno Operation already in progress(114)
2024-10-28 18:58:27.911 PROFILE turnConnectionHandleStunError(): [0x7f4052093010 - TURN Get Credentials] Time taken: 108 ms
2024-10-28 18:58:27.912 PROFILE turnConnectionHandleStunError(): [0x7f4052093010 - TURN Get Credentials] Time taken: 108 ms
2024-10-28 18:58:27.955 INFO    lwsWssCallbackRoutine(): WSS callback with reason 71
2024-10-28 18:58:27.962 PROFILE turnConnectionHandleStunError(): [0x7f4052093010 - TURN Get Credentials] Time taken: 159 ms
2024-10-28 18:58:28.012 PROFILE turnConnectionHandleStunError(): [0x7f4052093010 - TURN Get Credentials] Time taken: 209 ms
2024-10-28 18:58:28.053 INFO    stepIceAgentStateMachine(): Ice agent state changed from ICE_AGENT_STATE_CHECK_CONNECTION to ICE_AGENT_STATE_CONNECTED.
2024-10-28 18:58:28.053 INFO    stepIceAgentStateMachine(): Ice agent state changed from ICE_AGENT_STATE_CONNECTED to ICE_AGENT_STATE_NOMINATING.
2024-10-28 18:58:28.053 PROFILE iceAgentReadyStateSetup(): Selected pair gNlYDt4zR_MtPrUCu37, local candidate type: host. remote candidate type: prflx. Round trip time 3 ms. Local candidate priority: 2130706431, ice candidate pair priority: 7926380424115257342
2024-10-28 18:58:28.053 PROFILE executeReadyIceAgentState(): [ICE Agent ready for media exchange from check connection start] Time taken: 251 ms
2024-10-28 18:58:28.053 INFO    stepIceAgentStateMachine(): Ice agent state changed from ICE_AGENT_STATE_NOMINATING to ICE_AGENT_STATE_READY.
2024-10-28 18:58:28.164 PROFILE dtlsSessionChangeState(): [DTLS initialization completion] Time taken: 110 ms
2024-10-28 18:58:28.164 PROFILE changePeerConnectionState(): [ICE Hole Punching Time] Time taken: 361 ms
2024-10-28 18:58:28.164 INFO    onConnectionStateChange(): New connection state 3
2024-10-28 18:58:28.164 INFO    sendGstreamerAudioVideo(): [KVS Gstreamer Master] VIDEO_ONLY FILESOURCE H265
2024-10-28 18:58:28.232 PROFILE createRtcCertificate(): [Certificate creation time] Time taken: 0 ms
2024-10-28 18:58:32.474 INFO    lwsWssCallbackRoutine(): WSS callback with reason 9
2024-10-28 18:58:37.832 INFO    iceAgentGatherCandidateTimerCallback(): Candidate gathering completed.
2024-10-28 18:58:37.832 PROFILE iceAgentGatherCandidateTimerCallback(): [Candidate gathering time] Time taken: 10033 ms
2024-10-28 18:58:42.588 INFO    lwsWssCallbackRoutine(): WSS callback with reason 9

AWS test page Viewer

[2024-10-28T18:58:27.075Z] [INFO] [VIEWER] Starting viewer connection
[2024-10-28T18:58:27.085Z] [INFO] [VIEWER] Time to sign the request: 7 ms
[2024-10-28T18:58:27.088Z] [INFO] [VIEWER] Connecting to KVS Signaling...
[2024-10-28T18:58:27.533Z] [INFO] [VIEWER] Connected to signaling service
[2024-10-28T18:58:27.536Z] [INFO] [VIEWER] Time to connect to signaling: 444 ms
[2024-10-28T18:58:27.539Z] [INFO] [VIEWER] Creating SDP offer
[2024-10-28T18:58:27.557Z] [INFO] [VIEWER] Sending SDP offer
[2024-10-28T18:58:27.561Z] [INFO] [VIEWER] Generating ICE candidates
[2024-10-28T18:58:27.570Z] [INFO] [VIEWER] Generated ICE candidate
[2024-10-28T18:58:27.574Z] [INFO] [VIEWER] Sending ICE candidate
[2024-10-28T18:58:27.578Z] [INFO] [VIEWER] Generated ICE candidate
[2024-10-28T18:58:27.581Z] [INFO] [VIEWER] Sending ICE candidate
[2024-10-28T18:58:27.584Z] [INFO] [VIEWER] Generated ICE candidate
[2024-10-28T18:58:27.588Z] [INFO] [VIEWER] Sending ICE candidate
[2024-10-28T18:58:27.591Z] [INFO] [VIEWER] Generated ICE candidate
[2024-10-28T18:58:27.594Z] [INFO] [VIEWER] Sending ICE candidate
[2024-10-28T18:58:27.678Z] [INFO] [VIEWER] Generated ICE candidate
[2024-10-28T18:58:27.680Z] [INFO] [VIEWER] Sending ICE candidate
[2024-10-28T18:58:27.681Z] [INFO] [VIEWER] Generated ICE candidate
[2024-10-28T18:58:27.682Z] [INFO] [VIEWER] Sending ICE candidate
[2024-10-28T18:58:27.834Z] [INFO] [VIEWER] Generated ICE candidate
[2024-10-28T18:58:27.836Z] [INFO] [VIEWER] Sending ICE candidate
[2024-10-28T18:58:27.837Z] [INFO] [VIEWER] Generated ICE candidate
[2024-10-28T18:58:27.838Z] [INFO] [VIEWER] Sending ICE candidate
[2024-10-28T18:58:27.843Z] [INFO] [VIEWER] Generated ICE candidate
[2024-10-28T18:58:27.844Z] [INFO] [VIEWER] Sending ICE candidate
[2024-10-28T18:58:27.847Z] [INFO] [VIEWER] Generated ICE candidate
[2024-10-28T18:58:27.847Z] [INFO] [VIEWER] Sending ICE candidate
[2024-10-28T18:58:27.949Z] [INFO] [VIEWER] Received SDP answer
[2024-10-28T18:58:27.951Z] [INFO] [VIEWER] Received ICE candidate
[2024-10-28T18:58:27.953Z] [INFO] [VIEWER] Received ICE candidate
[2024-10-28T18:58:27.954Z] [INFO] [VIEWER] Received remote track with id: myKvsVideoStream
[2024-10-28T18:58:27.955Z] [INFO] [VIEWER] Received remote track with id: myKvsVideoStream
[2024-10-28T18:58:27.956Z] [INFO] [VIEWER] Received ICE candidate
[2024-10-28T18:58:28.141Z] [INFO] [VIEWER] All ICE candidates have been generated
[2024-10-28T18:58:28.143Z] [INFO] [VIEWER] Received ICE candidate
[2024-10-28T18:58:28.145Z] [INFO] [VIEWER] Connection to peer successful!

stefankiesz · 2024-10-28T19:04:44Z

stefankiesz
Oct 28, 2024
Maintainer

Hi Carlos, can you please confirm the pipeline you implemented is functional by writing to a filesink and playing the file.

2 replies

CarlosYeverino Oct 28, 2024
Author

Hello Stefan,
I updated my question with further details.
I verified I was able to load files and display video in autovideosink with

gst-launch-1.0 multifilesrc location="./h265SampleFrames/frame-%04d.h265" index=1 loop=true caps="video/x-h265,stream-format=byte-stream,alignment=au,framerate=25/1" ! h265parse ! avdec_h265 ! videoconvert ! autovideosink sync=true

stefankiesz Oct 28, 2024
Maintainer

I was able to reproduce your results when I ran the sample with the following changes:

Setting the senderPipeline to the one you provided to use multifilesrc with the h.265 sample frames.
Setting RTC_CODEC videoCodec = RTC_CODEC_H265 to select h.265 for the SDP.

I also was able to reproduce these results for the h.264 frames by doing the following steps:

Setting the senderPipeline to the one you provided to use multifilesrc but with h.264 elements and sample frames.
Leaving RTC_CODEC videoCodec = RTC_CODEC_H264_PROFILE_42E01F_LEVEL_ASYMMETRY_ALLOWED_PACKETIZATION_MODE as is.

So this is not h.265-specific, but rather our GStreamer sample does not support steaming sample frames through the provided GStreamer pipeline. Will continue investigating this.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[QUESTION] How to stream an existing h265 encoded video over webRTC? #2072

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Replies: 1 comment 2 replies

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Select a reply

[QUESTION] How to stream an existing h265 encoded video over webRTC? #2072

CarlosYeverino Oct 28, 2024

Replies: 1 comment · 2 replies

stefankiesz Oct 28, 2024 Maintainer

CarlosYeverino Oct 28, 2024 Author

stefankiesz Oct 28, 2024 Maintainer

CarlosYeverino
Oct 28, 2024

Replies: 1 comment 2 replies

stefankiesz
Oct 28, 2024
Maintainer

CarlosYeverino Oct 28, 2024
Author

stefankiesz Oct 28, 2024
Maintainer