The promise of HTML5's <video> tag was a simple one: to allow web pages to contain embedded video without the need for plugins. With the decision to remove support for the widespread H.264 codec from future versions of Chrome, Google has undermined this widely-anticipated feature. The company is claiming that it wants to support "open codecs" instead, and so from now on will support only two formats: its own WebM codec, and Theora.
Google's justification doesn't really add up, and there's a strong chance that the decision will serve only to undermine the use of the <video> tag completely. This is not a move promoting the open web. If anything, it is quite the reverse.
The <video> tag in HTML5 has been contentious since its inception due to question of codecs. Should the HTML5 specification mandate support for specific codecs, and if so, which should be required? Originally, the specification chose a particular compression algorithm for the <video> tag: the Theora algorithm. This decision was opposed by many parties involved in web standards for a range of reasons; as a result, the language of the specification was changed, so that it avoided specifying any particular algorithm. While this didn't make everyone involved particularly happy, it allowed work on the specification to proceed without endless arguments about video codecs. Though some opposed the decision, in truth it was hardly unprecedented: the HTML specification for the preexisting <img> tag for embedding still images does not mandate any particular format, either.
This meant that it was up to different browser vendors to pick their own preferred codecs. Apple and Microsoft (for the as-yet unreleased Internet Explorer 9) picked H.264 and H.264 alone. Firefox picked Theora and, in Firefox 4, WebM. Until this latest announcement, Chrome supported Theora, WebM, and H.264; in the future, it will support only Theora and WebM, just like Firefox. The reason Google has given for this change is that WebM (which pairs VP8 video with Vorbis audio) and Theora are "open codecs" and H.264 apparently isn't.
Openness can't be the issue
This explanation is lacking, to say the least. It appears to be a conflation of several issues: openness, royalty-freedom, and source code availability, among others. In the traditional sense, H.264 is an open standard. That is to say, it was a standard designed by a range of domain experts from across the industry, working to the remit of a standards organization. In fact, two standards organizations were involved: ISO and ITU. The specification was devised collaboratively, with its final ratification dependent on the agreement of the individuals, corporations, and national standards bodies that variously make up ISO and ITU. This makes H.264 an open standard in the same way as, for example, JPEG still images, or the C++ programming language, or the ISO 9660 filesystem used on CD-ROMs. H.264 is unambiguously open.
In contrast, neither WebM's VP8 nor Theora were assembled by a standards body such as ISO. VP8 was developed independently and entirely in secret by the company On2, prior to the company's purchase last year by Google. Theora was created by a group of open-source developers based on early work also done by On2. Though Theora's development can be described as an open, community process (albeit different in nature and style to the more formal processes and procedures used by the standards bodies), no such claim can be made of VP8. At the time of its development, VP8 was a commercial product, licensed by On2. Keeping the specifics of its codec secret was a deliberate goal of the company. Though it has since been published and to some extent documented, the major design work and decision-making was done behind closed doors, making it at its heart quite proprietary.
Google is now building a community around WebM (similar to that around Theora), but it hasn't taken any steps to submit WebM to ISO, ITU, or SMPTE for formal open standardization. The company is preferring to keep it under its own sole control.
For Google to claim that it is moving to "open codecs" is quite absurd: H.264 is very much an open codec. WebM is not.
It's about (cost) freedom
What H.264 isn't, however, is royalty-free. ISO and ITU do not require the members working on their various standards and specifications to give up any specific patent claims that may cover the technology that they define. As such, while H.264 is an open standard, there are several hundred different patents that cover the various techniques that it uses to achieve high quality compression—for example, estimating motion from frame to frame, removal of the block artifacts that result from the compression, and the final stage of lossless compression applied to the encoded video.
The result is that anyone wanting to distribute an implementation of H.264 must obtain licenses for all of the different patented techniques that they use, and these licenses typically come at some cost. To ease this burden, licenses for the full set of patents are available from the company MPEG-LA. MPEG-LA redistributes the income it makes from licenses to the various patent holders.
MPEG-LA's license terms for H.264 set out a range of fee schedules depending on the exact nature of the H.264 implementation. Importantly to web users, video that is distributed over the web and which is, importantly, not behind any kind of a paywall, is royalty-free. This means that uploading a video to a site such as YouTube and then rebroadcasting that video to all and sundry is free. For browser developers, the situation is not quite so happy: browsers include H.264 decoders, and these are subject to royalties. The size of the necessary payment depends on the number of units shipped—browsers with fewer than 100,000 users would likely not need to pay a royalty at all—but in any case is capped at $6.5 million (equivalent to about 65 million users), annually, until 2015.
Both VP8 and Theora are, however, royalty-free. Both were designed to avoid existing video patents. Theora was designed to use no patented techniques at all. VP8 does include patented techniques, but these techniques were developed and patented by On2. Google, as present owner of those patents, is permitting their use, in any application, without payment of any royalty.
At least to a point: the threat with both of those codecs is that they may, in fact, infringe on one or more patents, in spite of efforts to the contrary. If this turns out to be the case, one or both of the codecs might end up in a very similar position to H.264, as far as royalties are concerned.
Is freedom all it's cracked up to be?
W3C, the consortium creating HTML5, mandates that its specifications are royalty-free. Being covered by patents is tolerable, just as long as the technology can be implemented by anybody without payment. The intent is to ensure that anybody can implement web standards without hindrance.
Whether royalties actually stand in the way of adoption and implementation, however, is far from obvious. Patent and royalty restrictions have also done little to prevent the development of high quality open source H.264 implementations, after all. In principle, distribution of binaries (but not source code) that implement the patented techniques of H.264 requires a license, but while many Linux distributions strive to avoid such binaries, the reality is that they are freely distributed without anyone paying a cent to MPEG-LA. As long as developers stick to distributing source code (which describes the algorithms in question, but which does not actually function), they can operate unhindered by the patents and subsequent royalty demands.
Similarly, the GIF image format was at one time patent protected, and the company that held the patents, Unisys, did go after certain implementers of its patent, successfully extracting royalty fees from some of them. But all the while, GIF was widely used across the web, and there's no real sense that the web was held back by the GIF patents. For most purposes, GIF has been supplanted by the patent-free PNG, but for many users the reason for the switch was not patents at all, but PNG's technical superiority: PNG supports 32-bit color with transparency, as opposed to GIF's 256-color restriction. WebM has no such technical superiority over H.264.
This is not to say that these efforts are necessarily legal in those jurisdictions that permit software patents, and it's not to express approval or disapproval of their actions. Rather, the point is that in practice developers don't let royalties impede their implementations.
W3C's position is reasonable enough—mandating support for H.264 would require certain open-source browsers to break the rules, which certainly shouldn't be compulsory just to support HTML5. But to use W3C's position to justify not implementing it as an option in a web browser is harder to justify. It's not as if Google can't afford the $6.5 million a year, and by paying that money the company would enable web users to view open, standards-compliant, H.264 video.
The removal of H.264 for being insufficiently "open" leaves Google open to quite reasonable accusations of hypocrisy. Chrome has for some time now included a bundled copy of Adobe's Flash plugin. Google and Adobe have worked to improve the integration between the two, to improve Flash's performance and reliability, and one result of this is that Flash ships with Chrome. Now, OK, some may suggest that H.264's openness "doesn't count" due to its royalties, but even the most hardened H.264 critic must acknowledge that H.264 is orders of magnitude more open and less proprietary than Flash. Since 2009, Adobe has made efforts to make the Flash file specification freely available, but the plugin itself remains closed-source, and its development and design are entirely under Adobe's control.
If openness is so important that Google is willing to remove features from Chrome, there is no way that the company should be shipping Flash in Chrome.
Other features, too, should be culled. Chrome (currently) supports MP3 and AAC audio when used with HTML5's <audio> tag. Both of these compression algorithms are patent-encumbered, and neither is royalty-free (though both are, like H.264, open standards developed by industry consortia). They should be no more acceptable to Google than H.264. But if the company plans to remove them, it certainly hasn't said so.
At the very least, there appears to be a significant inconsistency between the company's actions regarding video support, and the rest of its browser. If it's going to remove features for poorly-articulated ideological reasons, it would surely make sense to apply that ideology consistently.
Web video doesn't exist in a vacuum
In the domain of web graphics, PNG never entirely displaced GIF. GIF does include an important ability that PNG lacks—support for animations—but even in the world of static images, GIF never quite vanished. This is in spite of both formats having relatively few other uses—though both formats were in widespread usage on the Web, neither was used in other mass media. This should have made wholesale replacement easier for PNG to achieve—there were no significant industries dependent on the use of GIF to prolong its usage—but that never really happened.
In the world of video, however, H.264 is everywhere. It's in Blu-ray; it's in the latest version of the ATSC American digital TV specification, it's used with the European DVB family of digital TV specifications, it's in the 3G-324M video call specification; hardware support is found in every modern smartphone, on every modern GPU (which will sooner or later translate into every modern CPU); it's in camcorders and cameras and digital workflows. H.264 is not going away. It's already entrenched, and the incorporation into TV standards means that in all likelihood, H.264 will be in use for decades.
WebM has none of this. There are efforts to provide hardware support for WebM, but widespread distribution of such features is some way off (if it ever happens at all; it may well remain a niche technology), and inclusion of WebM into the broadcast and optical disc standards is phenomenally unlikely.
The practical repercussion of this massive industrial entrenchment is that there will always be pressure for devices and software to handle H.264. Video clips recorded on a cellphone? They'll be H.264. Rips of broadcast TV or Blu-ray discs? H.264. There will continue to be a vast quantity of H.264 content produced, and for much of this content, there will be pressure to use it on the web. Web video, unlike web images, is not some standalone niche; it is instead part of a much broader ecosystem. If I want to show a video clip from my smartphone to someone, it means using H.264. If I upload it to YouTube, YouTube is, at the very least, going to have to convert it from H.264 into something else. There's just no getting away from H.264.
Google might want WebM to become the dominant web video standard, but H.264 is set to remain the dominant video standard for years to come, and WebM is never going to achieve that same status. Web browsers can either handle that standard natively (maximizing both user convenience and quality), or they can stick their head in the sand and ignore it, forcing users and site operators alike to transcode video into other formats.
What this means for the open web and HTML5 <video>
Prior to HTML5's <video> tag, anyone wanting to distribute video over the web used Flash to do so, typically serving H.264 content via a Flash container. For some applications—real-time streaming, DRM-protected streaming—Flash (or equivalent technologies like Microsoft's Silverlight) remains the only practical option. HTML5's <video> doesn't strive to address such needs.
Supporting H.264 via <video> allowed for an effective migration path from Flash, at least for that subset of video tasks that HTML5 could handle. The same video file could be encoded once, and then delivered to users using either a Flash container or <video>, depending on what their browser supported. This allowed for a smooth transition from proprietary Flash to open H.264 and HTML5.
That's now not an option. Video distributors wanting to support both Flash and HTML5 users will have to encode twice; once in H.264, for Flash users, and again in WebM, for HTML5 users. This doubles the computational cost, doubles the storage requirements, and as an added bonus will tend to hurt quality. This is inconvenient for a small site with one or two videos; for sites like SmugMug it's an enormous headache. They can either suffer the doubled costs and complexity, or ignore HTML5 altogether and stick with Flash.
It looks like sticking with Flash and ignoring <video> is indeed what SmugMug may end up doing. And who can blame them? Flash works for almost every Internet user, and Flash supports H.264.
There is of course a vocal minority of users who don't have Flash on their platform: iOS users. iPhones, iPod touches, and iPads have no Flash plugin. They do support HTML5 <video>... as long as it uses H.264. So they can use the same H.264 files as the Flash users get anyway. So that's what <video> will now become: the iOS fallback tag. Flash will remain the preferred solution for "real" browsers, and the only people using <video> will be those catering to iOS. Outside the limited demographic, it'll be simply too much of a crap shoot to bother with. Flash will do the same job, more reliably, at lower cost.
This hurts the open web
Prior to Google's decision, the migration from H.264-via-Flash to H.264-via-<video> looked likely. Internet Explorer 9, Safari, and Chrome were all to include native, built-in support for the codec, and even Firefox users would be able to use H.264 video through Microsoft's plugin for that browser. This would have represented great progress.
But now? That's much less clear cut. Chrome's presence will mean that there's a small but significant minority of web users for whom WebM is the only HTML5 option. Given the choice between transcoding for this minority, and just sticking with Flash for everyone except iOS users, it's likely that developers will pick the Flash option. And it's hard to fault them for this. Freely-distributed web video costs nothing, even if it uses H.264, but video storage and compression? They cost money. Picking one codec is the sensible option, and there's no way that that codec can be WebM.
In time, Flash is expected to support WebM. It doesn't right now, but Adobe has said that it will do so eventually. When this happens, and when versions of Flash that support WebM are widely distributed, a dual WebM-via-Flash plus WebM-via-<video> approach would be viable. Except the H.264 approach will still be better, because it'll still have more industry, hardware, and software support.
Google's decision—a decision to exclude support for an open codec, giving users fewer choices and an objectively inferior browser, does nothing to advance the open web. It means eschewing open standards in favor of Google-controlled proprietary standards, and it means that Flash remains the single best mechanism for delivery of web video.