I've written media-related Linux drivers for consumer and embedded devices since 2001 and I'm one of the Video4Linux core developers. I've given Linux media-related talks at the FOSDEM, LPC and ELC. I currently work as an embedded Linux consultant through my company Ideas on Board where I develop a wide range of embedded drivers including DRM/KMS and V4L2.
The Media Controller kernel API has been designed to exposing detailed information about media devices, and video capture devices in particular, to userspace. While the API is not limited to a particular type of device, it has so far been used mostly to expose the topology of complex embedded media devices to applications, allowing a fine-grained control over the device that wasn't possible through the V4L API alone.
Handling those devices in userspace isn't simple due to the hardware complexity. Careful testing of applications and userspace libraries is thus needed, especially when they aim at supporting a wide range of devices. The fact that the devices with which code needs to be tested are embedded and not always easily available to developers further increases testing issues.
A similar issue exists with the V4L API and has been addressed by a kernel driver that emulates a V4L device without requiring hardware. The goal of this project is to create a similar driver for the Media Controller API.
The Media Controller API documentation is available at http://linuxtv.org/downloads/v4l-dvb-apis/media_controller.html (built from the DocBook documentation in the kernel source tree).
The Media Controller development mailing list is email@example.com.
Tasks for the Application Period
Building the Media Controller Test Tool
A userspace utility named media-ctl is available as part of the v4l-utils package. In order to test the code that will be developed for this project the latest version of media-ctl will be needed. Furthermore, as changes to media-ctl might need to be performed as well, the utility should be compiled from its sources.
Clone the v4l-utils git tree. The git web overview page is available at http://git.linuxtv.org/cgit.cgi/v4l-utils.git/ and lists the git URL.
The build system is based on autotools. Configure and build the software. This should produce a media-ctl wrapper script in the utils/media-ctl/ directory that executes the compiled binary in utils/media-ctl/.libs. You can either install the media-ctl binary locally using 'make install' or run it from the source tree using the wrapper script.
Open-source development works based on trust. A proof that you have complete the first task properly isn't stricly required. If you don't complete it you will likely have problems later, so this is really a task designed to help you getting started with the project. Of course if you're not sure whether your results are correct and would like guidance, or have any other question, please ask on the outreachy-kernel mailing list and CC me.
Testing the Test Tool with a Hardware Device
While no access to hardware is needed to implement a virtual Media Controller driver, understanding the API is easier done when testing on a real device. All USB cameras compatible with the USB Video Class can be used to first experiment with the Media Controller API.
Clone the mainline kernel git tree, configure and build it. You can use your distribution's running kernel configuration file as a default. Make sure to enable both the CONFIG_MEDIA_CONTROLLER and the CONFIG_USB_VIDEO_CLASS options (the latter can be enabled as a module).
Install the kernel you have built. Make sure not to overwrite your distribution's kernel, but instead install the new kernel as a multi-boot option. This will allow you to reboot into a working system should the newly built kernel fail to boot properly.
Run the media-ctl tool with the --print-dot option and redirect the output to a file named uvc.dot. Process that file using the dot utility (part of the graphviz package) with 'dot -Tps -o uvc.ps uvc.dot' to generate a graph representing the device topology in postscript form. Open the generated postscript file and study how it relates to the text form of the device topology.
Testing the Virtual V4L Driver
Use your favourite webcam application to open the device and display video. The driver will generate a colour bar test pattern by default. If you have no favourite webcam application use qv4l2 from the v4l-utils package.
Creating a Skeleton Driver
Follow the Hello World kernel module example from chapter 2 of the Linux Device Drivers, Third Edition to create the skeleton driver. Note that the book has been written based on the 2.6.10 kernel, you might thus need to adapt the code to kernel API changes that have occured since then. The module should be placed in drivers/media/.
Create a Media Controller Device in the Skeleton Driver
Add Entities to the Media Controller Device
Create links between the entities to build a graph. You can mimic one of the graphs available at http://www.ideasonboard.org/media/ or create your own.
Tasks for the Internship Period
The goal of the Virtual Media Controller device is to expose a graph of media entities to userspace. To make this trully useful, the graph topology needs to be configurable, ideally from userspace. Tasks for this part are
With a virtual entities graph in place, support for virtual video sources and video processing entities would allow more elaborate usage of the driver by userspace video applications. This part extends the driver by adding virtual V4L2 video devices and sub-devices to the graph and implementing the corresponding V4L2 API.
V4L2 video capture and output devices.
As this could require more time than available for the internship, the internship target is a pipeline consisting of one sensor subdevice and one V4L2 video capture device.
V4L2 Image Source and Image Processing controls support.
Scope and Schedule
The two parts are mostly independent from each other. Once the graph topology configuration data structure is implemented support for dynamic graph modifications and for the video pipeline can be developed separately. As the three months internship period will likely not be long enough to implement all proposed features, you have the freedom to pick the tasks you find the most interesting (this should of course be discussed and agreed upon) and propose a time line. The main goal for the internship period is to get a driver merged (or on its way to being merged) in the mainline kernel with enough features to make it useful.