Linux V4L2 notes(3)

An example to use V4L2 API (MMAP streaming I/O).

1.  Open the device

v4l_fd = open (“/dev/videox”, O_RDWR, 0); 

ioctl(v4l_fd, VIDIOC_QUERYCAP, &cap); 

check if V4L2_CAP_VIDEO_CAPTURE and V4L2_CAP_STREAMING is set;

2. Change device properties, select a video and audio input, video standard, cropping, picture brightness, etc

ioctl(v4l_fd, VIDIOC_S_INPUT, &input); 

ioctl(v4l_fd, VIDIOC_G_STD, &std); 

ioctl(v4l_fd, VIDIOC_S_STD, &std); 

ioctl(v4l_fd, VIDIOC_CROPCAP, &cropcap); 

ioctl(v4l_fd, VIDIOC_S_CROP, &crop); 

ioctl(v4l_fd, VIDIOC_S_PARAM, &param);  //high quality capture, frame rate, etc

3. negotiate a data format

ioctl(v4l_fd, VIDIOC_S_FMT, &fmt);  // set data format, interlaced or not

ioctl(v4l_fd, VIDIOC_G_FMT, &fmt);  // get the captured size: fmt.fmt.pix.width/height

4. negotiate an input method. For MMAP streaming IO, request number of buffers, and queue them.

 

req.count = num;

req.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;

req.memory = V4L2_MEMORY_MMAP;

ioctl(v4l_handle, VIDIOC_REQBUFS, &req); 

for each buffer,

struct v4l2_buffer buf_info;

ioctl(, VIDIOC_QUERYBUF, &buf_info); // query the buffer information for each requested buffer;

buffers[i]. length/offset = output_buf_info.length/offset;

buffers[i].start = mmap(NULL, buffers[i].length, READ|WRITE|MAP_SHARED, buffers[i].offset;

 

buf_info.memory = V4L2_MEMORY_CAPTURE;

ioctl(, VIDIOC_QBUF, &buf_info);

5. the actual input loop

ioctl(v4l_fd, VIDIOC_STREAMON, V4L2_BUF_TYPE_VIDEO_CAPTURE);

// loop until get the desired number of buffers

{

    ioctl(v4l_fd, VIDIOC_DQBUF, &capture_buf); // get a captured buffer

process/display the buffer

 

    ioctl(v4l_fd, VIDIOC_QBUF, &capture_buf);

}

6. close the device

ioctl(v4l_fd, VIDIOC_STREAMOFF, …);

for each buffer,

munmap(buffers[j].start, buffers[j].length);

close(v4l_fd);

Linux V4L2 notes(2)

Data input/output methods

• Read/write (by default): data is copied between application and driver.
  • pros: simplest I/O method, requiring little or no setup to exchange data.
  • cons:
    • no meta-information, fram counters, timestamps, are passed. Therefore, there is no way to recognize frame dropping, etc.
    • less efficient, even using DMA to copy, compare to other input/output methods, which merely exchange buffer pointers.
• Streaming I/O: only pointers to the buffers are exchanged between application and driver, the data itself is not copied.
  • Indicated in capabilities field (V4L2_CAP_STRAMING)of struct v4l2_capability, returned by VIDIOC_QUERYCAP.
  • For user pointers, it has be to determined further, by calling the VIDIOC_REQBUFS ioctl.
  • Two methods
    • Memory mapping: map buffers in device memory, or DMA-able main memory, into the application’s address space.
    • User pointers: buffers are allocated by the application itself, and can reside in virtual or shared memory.
• Video overlay: Only used if the hardware supports storing captured images directly in video memory of a graphics card.

 

For memory mapping streaming I/O,

Applications:

• ioctl(handle, VIDIOC_REQBUFS, &reqbuf);
  • Ask the driver to allocate buffers, with the number of buffers and type: reqbuf.memory = V4L2_MEMORY_MAP;
  • Can also be used to change the number of buffers, or free the allocated memory.
• Map the buffers into the application’s address space: mmap();
• Enqueue all mapped buffers via ioctl VIDIOC_QBUF
• Start capturing: ioctl VIDIOC_STREAMON
• enter the read loop: ioctl VIDIOC_DQBUF blocks when no buffer is in the outgoing queue.
  • If O_NONBLOCK is given to the open() function, VIDIOC_DQBUF returns immediately with an EAGAIN error code if no buffer is avialble.
• Can use VIDIOC_QUERYBUF to query the buffer status: mapped, enqueued, full, empty.
• Re-enque the buffer when no needed: VIDIOC_QBUF;

The driver:

• Maintain two buffer queues: incoming and outgoing, organized as FIFOs.
• Initially, all mapped buffers are in dqueued state, inaccessible by the driver.

 

For user pointers,

Applications:

• ioctl VIDIOC_REQBUFS
  • Ask the driver to switch into user pointer I/O mode, with reqbuf.memory = V4L2_MEMORY_USERPTR;
• Allocate the buffers, and pass them to the driver: VIDIOC_QBUF;

 

V4L2 Buffers

struct v4L2_buffer {

               _u32 index;

                         Type;

               _u32 bytesused;  // for capture, the driver must set this field.

               _u32 flags;   

                         field;

                         timestamp;

                         timecode;

               _u32 sequence;

                          Memory;

               Union m;

                          _u32 offset; // for MMAP streaming;

                                    Userptr; // for USRPTR streaming;

               _u32 length; // size of the buffer (not the payload) in bytes

               _u32 input;

}

Buffer.flags:

1. MAPPED;
2. QUEUED (when this flag is set, the buffer is currently on the incoming queue, and it automatically moves to the outgoing queue after the buffer has been filled(for capture))
3. DONE: on the outgoing queue.
4. KEYFRAME, FRAME, BFRAME, TIMECODE (the timecode field is valid)
5. INPUT

Timecode contains information like 24FPS/30FPS,  etc

 

Linux V4L2 notes(1)

(The notes is intended for capture stream/input device only.)

Video inputs and outputs are physical connectors of a device.

VIDIOC_ENUMINPUT: get the available inputs.

• VIDIOC_S_INPUT: select a different input.
  • Note: audio and video inputs are associated: selecting a video source also selects an audio source.
• VIDIOC_G_INPUT: return the index of current video input.
• VIDIOC_QUERYCAP

struct v4l2_capability

{

               _u8 driver[16];  // name of the driver

               _u8 card[32];    // name of the device

               -u8 bus_info[32]; // location of the device in the system, e.g. “PCI Slot 4”

               _u32 version;

               _u32 capabilities;  // bit definition to indicate device capabilities

}

some capabilities Flags: V4L2_CAP_VIDEO_(VBI)_CAPTURE/OUTPUT/OVERLAY,   V4L2_CAP_AUDIO/ READWRITE/ASYNCIO/STREAMING.

• VIDIOC_ENUM/G/S/TRY_FMT:  data format and layout of an image in memory.

struct v4l2_pix_format {

               _u32 width;

               _u32 height;

               _u32 pixelformat;

                        field;

               _u32 bytesperline;

               _u32 sizeofimage;

                         colorspace

}

• VIDIOC_CROPCAP, VIDIOC_S_CROP

Struct v4l2_cropcap {

enum v4l2_buf_type type; // V4L2_BUF_TYPE_VIDEO_CAPTURE/OUTPUT/OVERLEY.

Struct v4l2_rect bound;      // the capturing window, in unit of pixels.

Struct v4l2_rect defrect;    // default cropping rectangle

Struct v4l2_fract;   // pixel aspect(y/x) when no scaling is applied

}

Note:

1. the original source image size(.bound) is returned by ioctl(VIDIOC_CROPCAP);
2. the default cropping size (.defrect) is returned by it, too.
3. to set a new cropping size, use VIDIOC_S_CROP.
4. To use scaling, set format.fmt.pix.width/height, to be different from the cropping size.

• VIDIOC_S/G_PARM  optimize the video capture process as well as I/O.

Struct v4l2_streamparm

{

               enum v4l2_buf_type type;

               union parm

                              struct v4l2_captureparm capture;

                              struct v4l2_outputparm output;

}

struct v4l2_captureparm

{

                              capability;           // V4L2_CAP_TIMEPERFRAME  (the frame skipping/repeating controlled by the timeperframe field is supported.

                              capuremode;     //  V4L2_MODE_HIGHQUALITY (intended for still imaging application).

               struct v4l2_fract timeperframe;  //  for GET, it is the nominal frame period determined by the current video standard; for SET, it is the desired period, in seconds. Driver might skip frames, to save I/O bandwidth. Only effective when V4L2_CAP_TIMEPERFRAME is set; a “zero” will reset to nominal frame period.

                              extendedmode;

                              readbuffers;

}