What is going on here?

Questions involving a Windows version of FFmpeg.
Post Reply
Videogamer555
Posts: 42
Joined: Sat Nov 09, 2013 1:16 am

What is going on here?

Post by Videogamer555 » Thu Mar 07, 2019 9:06 am

Ok, so I try to send raw video over a tcp connection from one program to another. In this case, FFMPEG to FFPLAY. I use the following command lines:

Code: Select all

ffplay -f rawvideo -pix_fmt yuv420p -s 640x480 -framerate 60 tcp://127.0.0.1:1234?listen=1

Code: Select all

ffmpeg -re -f lavfi -i testsrc2=s=640x480:r=60 -f rawvideo -pix_fmt yuv420p tcp://127.0.0.1:1234
The first of these sets up FFPLAY to listen for an incoming TCP connection. The second tells FFMPEG to connect to FFPLAY, and send it a raw video signal (a test pattern in this case).
Despite the frame rate being set in both sending and receiving sides to 60fps, I get only 10fps out of this setup.

Meanwhile, if I use a pixel format of yuvj420p, instead of yuv420p, like this:

Code: Select all

ffplay -f rawvideo -pix_fmt yuvj420p -s 640x480 -framerate 60 tcp://127.0.0.1:1234?listen=1

Code: Select all

ffmpeg -re -f lavfi -i testsrc2=s=640x480:r=60 -f rawvideo -pix_fmt yuvj420p tcp://127.0.0.1:1234
it has no problem running at 60fps. The pixel format yuvj420p uses the full range of levels on all YUV channels from 0 to 255, instead of only 16 to 235 (for Y) and 16 to 240 (for U and V) as the standard yuv420p pixel format does.

Creating a missmatch (sending yuvj420p but receiving it as yuv420p, or sending yuv420p but receiving it as yuvj420p) slightly changes the brightness and contrast of the image, but nothing else. One thing I did notice is that only the pixel format of the sender affects the frame rate, while the pixel format of the receiver has no effect. I have tested all 4 possible combinations of receiving yuv420p and yuvj420p, while sending yuv420p and yuvj420p. I have found that regardless of if there is missmatch in the pixel format or not, changing the pixel format of the receiver does not affect the frame rate, while changing the pixel format of the sender does change the frame rate.

Whenever the sender is using yuv420p the frame rate is only 10fps, but whenever the sender is using yuvj420p the frame rate is 60fps.
Can anybody here tell me why this is happening?

pandy
Posts: 184
Joined: Mon Feb 24, 2014 1:46 pm

Re: What is going on here?

Post by pandy » Mon Mar 11, 2019 8:50 pm

Provide detailed report, most likely a hidden (automatically invoked) libswscale is launched to adjust data from one format to another one. Try use NUT format https://ffmpeg.org/nut.html - it may help or not and perhaps use proper designation for quantization range

Code: Select all

  -color_range       <int>        ED.V..... color range (from 0 to INT_MAX) (default unknown)
     unknown                      ED.V..... Unspecified
     tv                           ED.V..... MPEG (219*2^(n-8))
     pc                           ED.V..... JPEG (2^n-1)
     unspecified                  ED.V..... Unspecified
     mpeg                         ED.V..... MPEG (219*2^(n-8))
     jpeg                         ED.V..... JPEG (2^n-1)

Videogamer555
Posts: 42
Joined: Sat Nov 09, 2013 1:16 am

Re: What is going on here?

Post by Videogamer555 » Thu Mar 21, 2019 8:09 pm

pandy wrote:
Mon Mar 11, 2019 8:50 pm
Provide detailed report, most likely a hidden (automatically invoked) libswscale is launched to adjust data from one format to another one.
When I use

Code: Select all

ffplay -f rawvideo -pix_fmt yuv420p -s 640x480 -framerate 60 tcp://127.0.0.1:1234?listen=1
for FFPLAY to receive, and use

Code: Select all

ffmpeg -re -f lavfi -i testsrc2=s=640x480:r=60 -f rawvideo -pix_fmt yuv420p tcp://127.0.0.1:1234
for FFMPEG to send, and wait for several seconds before stopping it (pressing q in FFPLAY), I get this console log from FFPLAY.

Code: Select all

ffplay -f rawvideo -pix_fmt yuv420p -s 640x480 -framerate 60 tcp://127.0.0.1:1234?listen=1
ffplay version N-93298-g0ce759defd Copyright (c) 2003-2019 the FFmpeg developers
  built with gcc 8.2.1 (GCC) 20190212
  configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libdav1d --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-libvo-amrwbenc --enable-libmysofa --enable-libspeex --enable-libxvid --enable-libaom --enable-libmfx --enable-amf --enable-ffnvcodec --enable-cuvid --enable-d3d11va --enable-nvenc --enable-nvdec --enable-dxva2 --enable-avisynth --enable-libopenmpt
  libavutil      56. 26.100 / 56. 26.100
  libavcodec     58. 47.103 / 58. 47.103
  libavformat    58. 26.101 / 58. 26.101
  libavdevice    58.  6.101 / 58.  6.101
  libavfilter     7. 48.100 /  7. 48.100
  libswscale      5.  4.100 /  5.  4.100
  libswresample   3.  4.100 /  3.  4.100
  libpostproc    55.  4.100 / 55.  4.100
Option -pix_fmt is deprecated, use -pixel_format.
Option -s is deprecated, use -video_size.
Input #0, rawvideo, from 'tcp://127.0.0.1:1234?listen=1':0B f=0/0
  Duration: N/A, start: 0.000000, bitrate: 221184 kb/s
    Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 640x480, 221184 kb/s, 60 tbr, 60 tbn, 60 tbc
[rawvideo @ 050abfc0] Invalid buffer size, packet size 269444 < expected frame_size 460800
And I get this console log from FFMPEG.

Code: Select all

ffmpeg -re -f lavfi -i testsrc2=s=640x480:r=60 -f rawvideo -pix_fmt yuv420p tcp://127.0.0.1:1234
ffmpeg version N-93298-g0ce759defd Copyright (c) 2000-2019 the FFmpeg developers
  built with gcc 8.2.1 (GCC) 20190212
  configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libdav1d --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-libvo-amrwbenc --enable-libmysofa --enable-libspeex --enable-libxvid --enable-libaom --enable-libmfx --enable-amf --enable-ffnvcodec --enable-cuvid --enable-d3d11va --enable-nvenc --enable-nvdec --enable-dxva2 --enable-avisynth --enable-libopenmpt
  libavutil      56. 26.100 / 56. 26.100
  libavcodec     58. 47.103 / 58. 47.103
  libavformat    58. 26.101 / 58. 26.101
  libavdevice    58.  6.101 / 58.  6.101
  libavfilter     7. 48.100 /  7. 48.100
  libswscale      5.  4.100 /  5.  4.100
  libswresample   3.  4.100 /  3.  4.100
  libpostproc    55.  4.100 / 55.  4.100
Input #0, lavfi, from 'testsrc2=s=640x480:r=60':
  Duration: N/A, start: 0.000000, bitrate: N/A
    Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 640x480 [SAR 1:1 DAR 4:3], 60 tbr, 60 tbn, 60 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (rawvideo (native) -> rawvideo (native))
Press [q] to stop, [?] for help
Output #0, rawvideo, to 'tcp://127.0.0.1:1234':
  Metadata:
    encoder         : Lavf58.26.101
    Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 640x480 [SAR 1:1 DAR 4:3], q=2-31, 221184 kb/s, 60 fps, 60 tbn, 60 tbc
    Metadata:
      encoder         : Lavc58.47.103 rawvideo
frame=    7 fps=0.0 q=-0.0 size=    3150kB time=00:00:00.11 bitrate=221183.4kbits/s speeframe=   13 fps= 12 q=-0.0 size=    5850kB time=00:00:00.21 bitrate=221183.7kbits/s speeframe=   19 fps= 12 q=-0.0 size=    8550kB time=00:00:00.31 bitrate=221183.8kbits/s speeframe=   25 fps= 12 q=-0.0 size=   11250kB time=00:00:00.41 bitrate=221183.8kbits/s speeframe=   31 fps= 12 q=-0.0 size=   13950kB time=00:00:00.51 bitrate=221183.9kbits/s speeframe=   37 fps= 12 q=-0.0 size=   16650kB time=00:00:00.61 bitrate=221183.9kbits/s speeframe=   43 fps= 12 q=-0.0 size=   19350kB time=00:00:00.71 bitrate=221183.9kbits/s speeframe=   49 fps= 12 q=-0.0 size=   22050kB time=00:00:00.81 bitrate=221183.9kbits/s speeav_interleaved_write_frame(): Unknown error
Error writing trailer of tcp://127.0.0.1:1234: Error number -10054 occurred
frame=   54 fps= 11 q=-0.0 Lsize=   24170kB time=00:00:00.90 bitrate=220000.7kbits/s speed=0.19x
video:24300kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
Conversion failed!
This runs at only 12fps, even though it should be running at 60fps. Something is slowing it down.
Last edited by Videogamer555 on Thu Mar 21, 2019 8:17 pm, edited 1 time in total.

Videogamer555
Posts: 42
Joined: Sat Nov 09, 2013 1:16 am

Re: What is going on here?

Post by Videogamer555 » Thu Mar 21, 2019 8:14 pm

Then when I use

Code: Select all

ffplay -f rawvideo -pix_fmt yuvj420p -s 640x480 -framerate 60 tcp://127.0.0.1:1234?listen=1
for FFPLAY to receive, and use

Code: Select all

ffmpeg -re -f lavfi -i testsrc2=s=640x480:r=60 -f rawvideo -pix_fmt yuvj420p tcp://127.0.0.1:1234
for FFMPEG to send, and wait for several seconds before stopping it (pressing q in FFPLAY), I get this console log from FFPLAY.

Code: Select all

ffplay -f rawvideo -pix_fmt yuvj420p -s 640x480 -framerate 60 tcp://127.0.0.1:1234?listen=1
ffplay version N-93298-g0ce759defd Copyright (c) 2003-2019 the FFmpeg developers
  built with gcc 8.2.1 (GCC) 20190212
  configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libdav1d --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-libvo-amrwbenc --enable-libmysofa --enable-libspeex --enable-libxvid --enable-libaom --enable-libmfx --enable-amf --enable-ffnvcodec --enable-cuvid --enable-d3d11va --enable-nvenc --enable-nvdec --enable-dxva2 --enable-avisynth --enable-libopenmpt
  libavutil      56. 26.100 / 56. 26.100
  libavcodec     58. 47.103 / 58. 47.103
  libavformat    58. 26.101 / 58. 26.101
  libavdevice    58.  6.101 / 58.  6.101
  libavfilter     7. 48.100 /  7. 48.100
  libswscale      5.  4.100 /  5.  4.100
  libswresample   3.  4.100 /  3.  4.100
  libpostproc    55.  4.100 / 55.  4.100
Option -pix_fmt is deprecated, use -pixel_format.
Option -s is deprecated, use -video_size.
Input #0, rawvideo, from 'tcp://127.0.0.1:1234?listen=1':0B f=0/0
  Duration: N/A, start: 0.000000, bitrate: 221184 kb/s
    Stream #0:0: Video: rawvideo, yuvj420p, 640x480, 221184 kb/s, 60 tbr, 60 tbn, 60 tbc
[swscaler @ 0e5540c0] deprecated pixel format used, make sure you did set range correctly
   4.65 M-V: -0.033 fd=   9 aq=    0KB vq= 3600KB sq=    0B f=0/0
And I get this console log from FFMPEG.

Code: Select all

ffmpeg -re -f lavfi -i testsrc2=s=640x480:r=60 -f rawvideo -pix_fmt yuvj420p tcp://127.0.0.1:1234
ffmpeg version N-93298-g0ce759defd Copyright (c) 2000-2019 the FFmpeg developers
  built with gcc 8.2.1 (GCC) 20190212
  configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libdav1d --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-libvo-amrwbenc --enable-libmysofa --enable-libspeex --enable-libxvid --enable-libaom --enable-libmfx --enable-amf --enable-ffnvcodec --enable-cuvid --enable-d3d11va --enable-nvenc --enable-nvdec --enable-dxva2 --enable-avisynth --enable-libopenmpt
  libavutil      56. 26.100 / 56. 26.100
  libavcodec     58. 47.103 / 58. 47.103
  libavformat    58. 26.101 / 58. 26.101
  libavdevice    58.  6.101 / 58.  6.101
  libavfilter     7. 48.100 /  7. 48.100
  libswscale      5.  4.100 /  5.  4.100
  libswresample   3.  4.100 /  3.  4.100
  libpostproc    55.  4.100 / 55.  4.100
Input #0, lavfi, from 'testsrc2=s=640x480:r=60':
  Duration: N/A, start: 0.000000, bitrate: N/A
    Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 640x480 [SAR 1:1 DAR 4:3], 60 tbr, 60 tbn, 60 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (rawvideo (native) -> rawvideo (native))
Press [q] to stop, [?] for help
[swscaler @ 07945500] deprecated pixel format used, make sure you did set range correctly
Output #0, rawvideo, to 'tcp://127.0.0.1:1234':
  Metadata:
    encoder         : Lavf58.26.101
    Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuvj420p, 640x480 [SAR 1:1 DAR 4:3], q=2-31, 221184 kb/s, 60 fps, 60 tbn, 60 tbc
    Metadata:
      encoder         : Lavc58.47.103 rawvideo
frame=   31 fps=0.0 q=-0.0 size=   13950kB time=00:00:00.51 bitrate=221183.9kbits/s speeframe=   61 fps= 60 q=-0.0 size=   27450kB time=00:00:01.01 bitrate=221183.9kbits/s speeframe=   92 fps= 60 q=-0.0 size=   41400kB time=00:00:01.53 bitrate=221184.0kbits/s speeframe=  122 fps= 60 q=-0.0 size=   54900kB time=00:00:02.03 bitrate=221184.0kbits/s speeframe=  153 fps= 60 q=-0.0 size=   68850kB time=00:00:02.55 bitrate=221184.0kbits/s speeframe=  183 fps= 60 q=-0.0 size=   82350kB time=00:00:03.05 bitrate=221184.0kbits/s speeframe=  213 fps= 60 q=-0.0 size=   95850kB time=00:00:03.55 bitrate=221184.0kbits/s speeframe=  244 fps= 60 q=-0.0 size=  109800kB time=00:00:04.06 bitrate=221184.0kbits/s speeframe=  274 fps= 60 q=-0.0 size=  123300kB time=00:00:04.56 bitrate=221184.0kbits/s speeav_interleaved_write_frame(): Unknown error
Error writing trailer of tcp://127.0.0.1:1234: Error number -10054 occurred
frame=  293 fps= 60 q=-0.0 Lsize=  131784kB time=00:00:04.88 bitrate=221073.3kbits/s speed=0.999x
video:131850kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
Conversion failed!
In this case, it runs at 60fps, just as it should.

pandy
Posts: 184
Joined: Mon Feb 24, 2014 1:46 pm

Re: What is going on here?

Post by pandy » Mon Mar 25, 2019 9:30 am

Once again - full report is a key to understand ALL steps.

https://ffmpeg.org/ffmpeg-all.html#Generic-options
-report

Dump full command line and console output to a file named program-YYYYMMDD-HHMMSS.log in the current directory. This file can be useful for bug reports. It also implies -loglevel debug.

Setting the environment variable FFREPORT to any value has the same effect. If the value is a ’:’-separated key=value sequence, these options will affect the report; option values must be escaped if they contain special characters or the options delimiter ’:’ (see the “Quoting and escaping” section in the ffmpeg-utils manual).

The following options are recognized:

file

set the file name to use for the report; %p is expanded to the name of the program, %t is expanded to a timestamp, %% is expanded to a plain %
level

set the log verbosity level using a numerical value (see -loglevel).

For example, to output a report to a file named ffreport.log using a log level of 32 (alias for log level info):

FFREPORT=file=ffreport.log:level=32 ffmpeg -i input output

Errors in parsing the environment variable are not fatal, and will not appear in the report.
Try to encapsulate RAW stream. It may improve for example timestamping dependent behaviour.

Post Reply