View source for Video codec

{{Short description|Digital video coder/decoder}}
{{More citations needed|date=May 2023}}
[[File:Video Codecs 101.webm|right|thumb|A short video explaining the concept of video codecs.]]

A '''video codec''' is [[software]] or [[Computer hardware|hardware]] that [[data compression|compresses]] and [[Uncompressed video|decompresses]] [[digital video]]. In the context of video compression, ''[[codec]]'' is a [[portmanteau]] of ''encoder'' and ''decoder'', while a device that only compresses is typically called an ''[[Encoder (digital)|encoder]]'', and one that only decompresses is a ''decoder''.

The compressed data format usually conforms to a standard [[video coding format]]. The compression is typically [[lossy]], meaning that the compressed video lacks some information present in the original video. A consequence of this is that decompressed video has lower quality than the original, uncompressed video because there is insufficient information to accurately reconstruct the original video.

There are complex relationships between the [[video quality]], the amount of data used to represent the video (determined by the [[bit rate]]), the complexity of the encoding and decoding algorithms, sensitivity to data losses and errors, ease of editing, random access, and end-to-end delay ([[Latency (engineering)|latency]]).

==History==
{{Further|Video coding format#History}}

Historically, video was stored as an analog signal on [[magnetic tape]]. Around the time when the [[compact disc]] entered the market as a digital-format replacement for analog audio, it became feasible to also store and convey video in digital form. Because of the large amount of storage and bandwidth needed to record and convey raw video, a method was needed to reduce the amount of data used to represent the raw video. Since then, [[engineers]] and [[mathematicians]] have developed a number of solutions for achieving this goal that involve compressing the digital video data.

In 1974, [[discrete cosine transform]] (DCT) compression was introduced by [[N. Ahmed|Nasir Ahmed]], T. Natarajan and [[K. R. Rao]].<ref name="pubDCT">{{Citation |first1=Nasir |last1=Ahmed |author1-link=N. Ahmed |first2=T. |last2=Natarajan |first3=K. R. |last3=Rao |title=Discrete Cosine Transform |journal=IEEE Transactions on Computers |date=January 1974 |volume=C-23 |issue=1 |pages=90–93 |doi=10.1109/T-C.1974.223784 }}</ref><ref name="pubRaoYip">{{Citation |last1=Rao |first1=K. R. |author-link1=K. R. Rao |last2=Yip |first2=P. |title=Discrete Cosine Transform: Algorithms, Advantages, Applications |publisher=Academic Press |location=Boston |year=1990 |isbn=978-0-12-580203-1}}</ref><ref name="t81">{{cite web |title=T.81 – DIGITAL COMPRESSION AND CODING OF CONTINUOUS-TONE STILL IMAGES – REQUIREMENTS AND GUIDELINES |url=https://www.w3.org/Graphics/JPEG/itu-t81.pdf |publisher=CCITT |date=September 1992 |access-date=12 July 2019}}</ref> During the late 1980s, a number of companies began experimenting with DCT [[lossy compression]] for video coding, leading to the development of the [[H.261]] standard.<ref name="Ghanbari"/> H.261 was the first practical video coding standard,<ref name="history">{{Cite web|url=http://www.real.com/resources/digital-video-file-formats/|title=The History of Video File Formats Infographic — RealPlayer|date=22 April 2012}}</ref> and was developed by a number of companies, including [[Hitachi]], [[PictureTel]], [[Nippon Telegraph and Telephone|NTT]], [[BT plc|BT]], and [[Toshiba]], among others.<ref name="h261-patents">{{cite web |title=ITU-T Recommendation declared patent(s) |url=https://www.itu.int/ITU-T/recommendations/related_ps.aspx?id_prod=1088 |website=ITU |access-date=12 July 2019}}</ref> Since H.261, DCT compression has been adopted by all the major video coding standards that followed.<ref name="Ghanbari">{{cite book |last1=Ghanbari |first1=Mohammed |title=Standard Codecs: Image Compression to Advanced Video Coding |date=2003 |publisher=[[Institution of Engineering and Technology]] |isbn=9780852967102 |pages=1–2 |url=https://books.google.com/books?id=7XuU8T3ooOAC&pg=PA1}}</ref>

The most popular [[video coding standard]]s used for codecs have been the [[MPEG]] standards. [[MPEG-1]] was developed by the [[Motion Picture Experts Group]] (MPEG) in 1991, and it was designed to compress [[VHS]]-quality video. It was succeeded in 1994 by [[MPEG-2]]/[[H.262]],<ref name="history"/> which was developed by a number of companies, primarily [[Sony]], [[Technicolor SA|Thomson]] and [[Mitsubishi Electric]].<ref name="mp2-patents">{{cite web |title=MPEG-2 Patent List |url=https://www.mpegla.com/wp-content/uploads/m2-att1.pdf |website=[[MPEG LA]] |access-date=7 July 2019 |archive-date=29 May 2019 |archive-url=https://web.archive.org/web/20190529164140/https://www.mpegla.com/wp-content/uploads/m2-att1.pdf |url-status=dead }}</ref> MPEG-2 became the standard video format for [[DVD]] and [[SD digital television]].<ref name="history"/> In 1999, it was followed by [[MPEG-4 Visual|MPEG-4]]/[[H.263]], which was a major leap forward for video compression technology.<ref name="history"/> It was developed by a number of companies, primarily Mitsubishi Electric, Hitachi and [[Panasonic]].<ref name="mp4-patents">{{cite web |title=MPEG-4 Visual - Patent List |url=https://www.mpegla.com/wp-content/uploads/m4v-att1.pdf |website=[[MPEG LA]] |access-date=6 July 2019 |archive-date=6 July 2019 |archive-url=https://web.archive.org/web/20190706184814/https://www.mpegla.com/wp-content/uploads/m4v-att1.pdf |url-status=dead }}</ref>

The most widely used video coding format, as of 2016, is [[H.264/MPEG-4 AVC]]. It was developed in 2003 by a number of organizations, primarily Panasonic, [[Godo kaisha|Godo Kaisha IP Bridge]] and [[LG Electronics]].<ref name="avc-patents">{{cite web |title=AVC/H.264 {{ndash}} Patent List |url=https://www.mpegla.com/wp-content/uploads/avc-att1.pdf |website=MPEG LA |access-date=6 July 2019 |archive-date=25 January 2023 |archive-url=https://web.archive.org/web/20230125102953/https://www.mpegla.com/wp-content/uploads/avc-att1.pdf |url-status=dead }}</ref> H.264 is the main video encoding standard for [[Blu-ray Disc]]s, and is widely used by streaming internet services such as [[YouTube]], [[Netflix]], [[Vimeo]], and [[iTunes Store]], web software such as [[Adobe Flash Player]] and [[Microsoft Silverlight]], and various [[HDTV]] broadcasts over terrestrial and satellite television.

AVC has been succeeded by [[HEVC]] (H.265), developed in 2013. It is heavily patented, with the majority of patents belonging to [[Samsung Electronics]], [[GE]], NTT and [[JVC Kenwood]].<ref name="hevc-patents">{{cite web |title=HEVC Patent List |url=https://www.mpegla.com/wp-content/uploads/hevc-att1.pdf |website=[[MPEG LA]] |access-date=6 July 2019 |archive-date=10 April 2021 |archive-url=https://web.archive.org/web/20210410171930/https://www.mpegla.com/wp-content/uploads/hevc-att1.pdf |url-status=dead }}</ref><ref name="hevcadvance">{{cite web|url=https://www.hevcadvance.com/licensors/|title=HEVC Advance Patent List|website=[[HEVC Advance]]|access-date=6 July 2019|archive-date=24 August 2020|archive-url=https://web.archive.org/web/20200824174620/https://www.hevcadvance.com/licensors/|url-status=dead}}</ref> The adoption of HEVC has been hampered by its complex licensing structure. HEVC is in turn succeeded by [[Versatile Video Coding]] (VVC).

There are also the open and free [[VP8]], [[VP9]] and [[AV1]] video coding formats, used by YouTube, all of which were developed with involvement from [[Google]].

== Applications ==
Video codecs are used in DVD players, [[Internet video]], [[video on demand]], [[digital cable]], [[digital terrestrial television]], [[videotelephony]] and a variety of other applications. In particular, they are widely used in applications that record or transmit video, which may not be feasible with the high data volumes and bandwidths of uncompressed video. For example, they are used in [[operating theater]]s to record surgical operations, in [[IP camera]]s in security systems, and in [[remotely operated underwater vehicle]]s and [[unmanned aerial vehicle]]s. Any video stream or file can be encoded using a wide variety of live video format options. Here are some of the H.264 encoder settings that need to be set when streaming to an HTML5 video player.<ref>{{Cite web|date=2021-06-18|title=What is the Best Video Codec for Web Streaming? (2021 Update)|url=https://www.dacast.com/blog/best-video-codec/|access-date=2022-02-11|website=Dacast|language=en}}</ref>

== Video codec design ==
{{Further|Video coding format}}

Video codecs seek to represent a fundamentally analog data set in a digital format.  Because of the design of analog video signals, which represent [[Luminance (video)|luminance]] (luma) and [[chrominance|color information]] (chrominance, chroma) separately, a common first step in image compression in codec design is to represent and store the image in a [[YCbCr]] color space.  The conversion to YCbCr provides two benefits: first, it improves compressibility by providing decorrelation of the color signals; and second, it separates the luma signal, which is perceptually much more important, from the chroma signal, which is less perceptually important and which can be represented at lower resolution using [[chroma subsampling]] to achieve more efficient data compression.  It is common to represent the ratios of information stored in these different channels in the following way Y:Cb:Cr. Different codecs use different chroma subsampling ratios as appropriate to their compression needs.  Video compression schemes for Web and DVD make use of a 4:2:1 color sampling pattern, and the [[DV (video format)|DV]] standard uses 4:1:1 sampling ratios.  Professional video codecs designed to function at much higher bitrates and to record a greater amount of color information for post-production manipulation sample in 4:2:2 and 4:4:4 ratios.  Examples of these codecs include Panasonic's DVCPRO50 and DVCPROHD codecs (4:2:2), Sony's HDCAM-SR (4:4:4), Panasonic's HDD5 (4:2:2), [[Apple Inc.|Apple]]'s Prores HQ 422 (4:2:2).<ref>{{Cite book|last=Hoffman|first=P.|title=Requirements for Internet-Draft Tracking by the IETF Community in the Datatracker |date=June 2011|doi=10.17487/rfc6293|doi-access=free}}</ref>

It is also worth noting that video codecs can operate in RGB space as well.  These codecs tend not to sample the red, green, and blue channels in different ratios, since there is less perceptual motivation for doing so—just the blue channel could be undersampled.

Some amount of spatial and temporal [[downsampling]] may also be used to reduce the raw data rate before the basic encoding process. The most popular encoding transform is the 8x8 DCT.  Codecs that make use of a [[wavelet]] transform are also entering the market, especially in camera workflows that involve dealing with [[Raw image format|RAW]] image formatting in motion sequences. This process involves representing the video image as a set of [[macroblocks]].  For more information about this critical facet of video codec design, see [[B-frames]].<ref>{{cite book |last1=Richardson |first1=Iain E. G. |title=Video Codec Design |date=2002 |doi=10.1002/0470847832 |isbn=978-0-471-48553-7 }}{{pn|date=August 2025}}</ref>

The output of the transform is first [[Quantization (image processing)|quantized]], then [[entropy encoding]] is applied to the quantized values.  When a DCT has been used, the coefficients are typically scanned using a [[zig-zag scan]] order, and the entropy coding typically combines a number of consecutive zero-valued quantized coefficients with the value of the next non-zero quantized coefficient into a single symbol and also has special ways of indicating when all of the remaining quantized coefficient values are equal to zero.  The entropy coding method typically uses [[Variable-length code|variable-length coding tables]].  Some encoders compress the video in a multiple-step process called ''n-pass'' encoding (e.g. 2-pass), which performs a slower but potentially higher quality compression.

The decoding process consists of performing, to the extent possible, an inversion of each stage of the encoding process.{{fact|date=August 2025}} The one stage that cannot be exactly inverted is the quantization stage.  There, a best-effort approximation of inversion is performed.  This part of the process is often called ''inverse quantization'' or ''dequantization'', although quantization is an inherently non-invertible process.

Video codec designs are usually standardized or eventually become standardized—i.e., specified precisely in a published document.  However, only the decoding process need be standardized to enable interoperability. The encoding process is typically not specified at all in a standard, and implementers are free to design their encoder however they want, as long as the video can be decoded in the specified manner.  For this reason, the quality of the video produced by decoding the results of different encoders that use the same video codec standard can vary dramatically from one encoder implementation to another.

== Commonly used video codecs ==
{{Main article|List of codecs#Video compression formats}}

A variety of video compression formats can be implemented on PCs and in consumer electronics equipment. It is therefore possible for multiple codecs to be available in the same product, reducing the need to choose a single dominant video compression format to achieve [[interoperability]].

Standard [[video compression format]]s can be supported by multiple encoder and decoder implementations from multiple sources. For example, video encoded with a standard [[MPEG-4 Part 2]] codec such as [[Xvid]] can be decoded using any other standard [[MPEG-4 Part 2]] codec such as [[FFmpeg MPEG-4]] or [[DivX Pro Codec]], because they all use the same video format.

Codecs have their qualities and drawbacks. [[Comparison of video codecs|Comparisons]] are frequently published. The trade-off between compression power, speed, and fidelity (including [[Compression artifact|artifacts]]) is usually considered the most important figure of technical merit.

==Codec packs==
Online video material is encoded by a variety of codecs, and this has led to the availability of codec packs&nbsp;— a pre-assembled set of commonly used codecs combined with an installer available as a software package for PCs, such as [[K-Lite Codec Pack]], [[Perian]] and [[Combined Community Codec Pack]].

== See also ==
* [[Bit rate]]
* [[Comparison of video codecs]]
* {{section link|Data compression|Video}}
* [[Display resolution]]
* [[Frame rate]]
* [[List of open-source codecs]]
* [[Multiplexing]]
* [[Sampling rate]]
* [[Subjective video quality]]
* [[Transcoding]]
* [[Video quality]]

== References ==
{{reflist}}

== External links ==
* [http://ivms.stanford.edu/~dsc/wzcodingvideo Wyner-Ziv Coding of Video] {{Webarchive|url=https://web.archive.org/web/20110930024946/http://ivms.stanford.edu/~dsc/wzcodingvideo |date=2011-09-30 }} describes another algorithm for video compression that performs close to the [[Slepian–Wolf bound]] (with links to source code).
* [https://web.archive.org/web/20120828085725/http://support.amd.com/us/gpudownload/windows/Pages/eyespeed_downloads.aspx AMD Media Codecs]—optional download (formerly called [[ATI Avivo]])

{{Compression formats}}
{{Compression Methods}}

{{DEFAULTSORT:Video Codec}}
[[Category:Video codecs| ]]
[[Category:Videotelephony]]
[[Category:Data compression]]