Page 1 of 1

Trim&concat 2 videos with overlaying via frame-number-images from pipe not works as expected

Posted: Tue Aug 06, 2019 4:07 pm
by komonaut
I have next task:

as input, I have 2 video files (videoFilePath1, videofilePath2) and pipe, which I should use to programmatically send images with frame numbers, created in memory .

I need to
1. cut both videos using input parameters: start/duration for each of them (startSeconds1, durationSeconds1, startSeconds2, durationSeconds2)
2. concat 2 videos in 1
3. overlay result of 2. with images from pipe, so each frame will have it's number (e.g. frame 256 should have number 256),
and should make it with one ffmpeg call

My solution
a.for correct frame number images generation, I set frameRate and use it for calculation of necessary amount of images:

Code: Select all

 framesCount = (durationSeconds1 + durationSeconds2) * FRAME_RATE
b.I use this arguments for ffmpeg calling:

Code: Select all

-y -loop 1 -thread_queue_size {framesCount} -f image2pipe -framerate {FRAME_RATE} -i pipe:0 -i {videoFilePath1} -i {videofilePath2} -filter_complex 
    "[1:v]trim=start={startSeconds1}:duration={durationSeconds1},fifo,setpts=PTS-STARTPTS[av];    
    [1:a]atrim=start={startSeconds1}:duration={durationSeconds1},afifo,asetpts=PTS-STARTPTS[aa]; 
    [2:v]trim=start={startSeconds2}:duration={durationSeconds2},fifo,setpts=PTS-STARTPTS[bv]; 
    [2:a]atrim=start={startSeconds2}:duration={durationSeconds2},afifo,asetpts=PTS-STARTPTS[ba];    
    [av][aa][bv][ba]concat=n=2:v=1:a=1[outv][outa];
    [outv][0:v]overlay=shortest=1[outvv] " 
    -r {FRAME_RATE} -map [outvv] -map [outa] -vcodec libx264 -pix_fmt yuv420p -crf 27 -level 3.1 -preset slow -b:v 1200000 -acodec aac -subq 7 -me_range 16 -threads 2 result.mp4
c. In C# I call ffmpeg in Process and send framesCount dynamically generated images

It works, but in result video I have wrong frame numbering, it's not completely synced with overlayed video, e.g. number 1 have first 2 frames, but at frame 9 - there no frame_number overlayed.

It's interesting, that:

1. In 10 seconds of result 24fps video I have 243 frames (should be 240)
2. When I generate extraframes and use shortest=1 in overlay to cut it, there frame numbering stopped at 241
3. I got numerous Non-monotonous DTS in output stream 0:1 warnings

Also, when I make it in two steps:
a. cut&concat
b. overlay result video from 1. with images from pipe
it works as expected and have no any warnings.

But it not works correct, when I try to do it in 1 operation.

Where could be a reason of wrong overlaying? Or it could be on earlier steps?