[QUESTION] video+audio stream does not playback audio in Quicktime (or browser) after downloading media clip

[mjdietzx]

My live stream works fine, I don't have errors or glitches and everything looks good. If I play the video in VLC or Audacity I hear the audio as expected. However, I can't hear the audio in QuickTime. If I pass over it w/ ffmpeg, ie ffmpeg -I input.mp4 -c copy -map 0 output.mp4 then output.mp4 will play audio in QuickTime as expected.

Here is the output if I run this ffmpeg command on a video I download from KVS (I share this incase you see ffmpeg is doing any re-encoding.. I am trying to ensure it does not re-encode or modify the streams at al. If ffmpeg re-encodes the audio then that also fixes it and will playback in QT correctly):

[mp4 @ 0x11ce0f6e0] Non-monotonous DTS in output stream 0:1; previous: 3894757, current: 3894736; changing to 3894758. This may result in incorrect timestamps in the output file.
frame= 2592 fps=0.0 q=-1.0 Lsize=    6060kB time=00:01:48.46 bitrate= 457.7kbits/s speed=6.83e+03x    
video:4825kB audio:1165kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 1.175100%

The clue I have right now is the video's metadata. Ffmpeg seems to add encoder : Lavf59.27.100 to top level metadata and it changes compatible_brands from iso2mp41avc1 to isomiso2mp41. I am not sure if this is what solves the issue, if so I need some guidance on how to set this correctly in c producer lib.

Before ffmpeg (does not play audio in QuickTime):

Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'BIRD_BUDDY_V1-1cb38ded-f311-471d-96dc-68b30acce9cb-RTS-51.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: iso2mp41avc1
    creation_time   : 2023-04-04T17:17:10.000000Z
  Duration: 00:01:59.02, start: 0.000000, bitrate: 341 kb/s
  Stream #0:0[0x1](eng): Video: hevc (Main) (hvc1 / 0x31637668), yuvj420p(pc, bt709), 720x960, 245 kb/s, 24.20 fps, 29.92 tbr, 1k tbn (default)
    Metadata:
      creation_time   : 2023-04-04T17:17:10.000000Z
      handler_name    : USP Video Handler
      vendor_id       : [0][0][0][0]
  Stream #0:1[0x2](eng): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, mono, fltp, 88 kb/s (default)
    Metadata:
      creation_time   : 2023-04-04T17:17:10.000000Z
      handler_name    : SoundHandler
      vendor_id       : [0][0][0][0]

After ffmpeg (does play audio in QT):

Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'output.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2mp41
    encoder         : Lavf59.27.100
  Duration: 00:01:48.47, start: 0.000000, bitrate: 457 kb/s
  Stream #0:0[0x1](eng): Video: hevc (Main) (hvc1 / 0x31637668), yuvj420p(pc, bt709), 720x960, 364 kb/s, 23.90 fps, 29.83 tbr, 16k tbn (default)
    Metadata:
      handler_name    : USP Video Handler
      vendor_id       : [0][0][0][0]
  Stream #0:1[0x2](eng): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, mono, fltp, 85 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
      vendor_id       : [0][0][0][0]

I setup the live stream very similar to the audio and video sample provided in this repo. But here is my exact code where I set this up incase it helps. My audio stream is encoded as AAC_LC (44.1 kHz, 88,200bps, 1 channel mono)

CHK_STATUS(createRealtimeAudioVideoStreamInfoProviderWithCodecs(kvs_name, DEFAULT_RETENTION_PERIOD, DEFAULT_BUFFER_DURATION, VIDEO_CODEC_ID_H265, AUDIO_CODEC_ID_AAC, &pStreamInfo));

pAudioTrack = pStreamInfo->streamCaps.trackInfoList[0].trackId == DEFAULT_AUDIO_TRACK_ID ? &pStreamInfo-streamCaps.trackInfoList[0]
                                                                                   : &pStreamInfo->streamCaps.trackInfoList[1];
pAudioTrack->codecPrivateData = aacAudioCpd;
pAudioTrack->codecPrivateDataSize = KVS_AAC_CPD_SIZE_BYTE;
CHK_STATUS(mkvgenGenerateAacCpd(p_h265_settings->aac_type, p_h265_settings->audio_attr.rate, p_h265_settings-audio_attr.channels, pAudioTrack->codecPrivateData, pAudioTrack->codecPrivateDataSize));

[jdelapla]

The formats listed hvc1/mp4a are valid formats for quicktime according to this: http://mp4ra.org/#/codecs

However the change in duration leads me to believe that the tail end of the media could be cut off like in this exoplayer reported issue google/ExoPlayer#6441
What API and command did you use to download the video from KVS?

[mjdietzx]

What API and command did you use to download the video from KVS?

Right now I am just downloading it from KVS media playback in AWS console by clicking the Download media clip button in the UI and then downloading

That's a good point - it is weird that the duration is slightly shorter after I pass it through ffmpeg