Chris Kranky

Recent Posts


WebRTC impacts on battery life

WebRTC continues to advance on better battery management accessing hardware based encoders/decoders. Let's see the results.

Chris KoehnckeChris Koehncke

Things are changing with video codecs and I sat down with some of my colleagues (Christian Ferran and Sridhar Bollam) at Tokbox and got educated on video codecs and battery life. This is a non-techie summary and quick read (check out chart below).

Video codecs have the simple job of providing a mathematical formula to convert raw video bits into something more manageable. There are always trade-offs in codecs and there will always be new and improved codecs. Like encryption, codecs require something to encode them (package) and decode them. Because codecs are a formula rather than product (hardware or software), intellectual property arguments and lawsuits are a near constant about who did what to whom.

Codecs are also the fodder of lots of technical debate, arguments and not enough alcohol consumption (in my opinion). With WebRTC, Google decided to initially use a codec called VP8. The alternative was H.264. Like shark’s teeth, there are always new codecs such as AV1, VP9, and H.265 emerging. Change is constant (except for the PSTN which has been stuck on G.711 since Alexander Graham Bell).

If you’re just trying to get started with WebRTC, I suggest you simply ignore all the noise on codecs. It’s mostly, but not all, irrelevant.

Codecs take ‘energy’ to encode and decode video. In our mobile battery operated world saving energy is important.

In encoding/decoding video, you can do it via software or via hardware. The software approach is brute force and not usually very efficient but since it’s software you can install it and keep current. Hardware requires a small specialized chip or intelligent GPU that lives to decode and encode. It works super efficient but may be behind the times and usually hard coded to support specific codecs.

H.264 has been around a long time and nearly every mobile phone has a native H264 hardware accelerated encoder/decoder. In addition, nearly any computer with a decent video card will also support H.264 decoding/encoding in hardware.

Apple had native encoding/decoding of H.264 from the early days. However, iOS initially didn’t let an application access this hardware capability (though Facetime did). However, current versions of iOS do (so proud Apple trying to be supportive of the Internet community).

Some of the old timers I deal with argue that the hardware based encoder/decoders are bad (particularly on mobile). However, I now beg to differ. Top mobile manufacturers are paying attention to this capability.

Google is working very hard to adapt the Chrome browser to access the onboard hardware H.264 encoder/decoder on mobile and desktop computers. It works today, there are some bugs but great progress. Success is well in sight!

Similarly, Google has been modifying the WebRTC code blob for mobile usage. There is excellent progress on iOS. Apple put in an great hardware/encoded “chip” into the iPhone.

Android, unfortunately, is a different story. Since there are dozens of Android phone manufacturers and no common hardware design, the state of hardware video encoding/decoding isn’t standardized. You can quickly imagine a cheap Android phone cut some corners, particularly on a seldom used video encoder/decoder.

However, Google is trying hard to make WebRTC video use the hardware acceleration on Android mobile phones. You might expect Pixel and Samsung to be first in line. My Tokbox teammates tell me excellent progress is being made.

That’s all nice, I know, but what does this all mean.

First, it doesn’t sound like VP8 or 9 will have a hardware based encoder or decoder on mobile or laptops anytime soon. This may well raise the longer term viability of these codecs. But time will answer that question and there remains a fair bit of unknown.

Many mobile phones and desktops support H.264 hardware acceleration and Chrome can now access this with impressive results. I fired up AppRTC (using parameter ?vrc=H264, add this to end of URL to attempt to use H.264) and did a quick test on a Macbook Pro (which supports hardware accelerated H.264), here are the results:

Comparison of WebRTC video codec CPU usageVP8VP9H.264
Idle CPU Utilization0%0%0%
CPU Utilization P2P 2-party video call150%165%89%
Avg Energy Impact (Mac)270-300385-420200
Lower numbers are better. Rough guideline CPU usage: H264 1X , VP8 1.5X, VP9 1.8X

01/16/2017 – thanks to Gustavo García, my stupid was originally testing VP9 against H.264, updated chart above.

download-22

In short, a WebRTC session using H.264 required 50% less CPU power! This is a pretty big deal as we do more video sessions on mobile devices dependent upon battery life. This is a big plus for WebRTC versus a ‘roll your own’ strategy as any app developed using the core WebRTC elements will automatically benefit.

 

Comments 7
  • Michael Graves
    Posted on

    Michael Graves Michael Graves

    Author

    There was also some noise about hardware support for VP8. It was supposedly happening, but I’m not sure what chipsets. One thing is certain, with defined functions like video encoding/decoding, hardware support is key to delivering battery life.


  • Lawrence Byrd
    Posted on

    Lawrence Byrd Lawrence Byrd

    Author

    It should be noted that you are testing the old(er) VP8 and H.264 codecs. I see a lot of “webRTC futures” discussion that just assume that “of course”we need to get to VP9 and H.265. This seems to me to be a “power user with good laptop” perspective since these will require even more CPU, especially on encode, and will struggle on low-end mobile devices and chew up mobile batteries at a stupendous rate. I have not yet seen any convincing analysis that shows that these new codecs will be a good idea for mobile device live multi-way video applications. Have you?

    Yes the great new codecs use less bits and therefore provide better bandwidth use, but at what cost? And how important is this for small format mobile screens, particularly with group use cases with many even smaller panes – versus better battery and CPU usage? Add in the confusion, cost and licensing complexities of H.265 and I end up concluding that H.264 may well have a good life ahead as a “local maxima” balancing all these battery/CPU/performance/cost/licensing constraints for everyday mobile apps.


  • Chris Koehncke
    Posted on

    Chris Koehncke Chris Koehncke

    Author

    I had done some digging with the OEM of the chips. Their challenge they need to sell millions of copies of anything they do for at least several years to make any money. Tsahi had a blog article https://bloggeek.me/vp9-hardware-acceleration/ from middle of last year on this. But progress has been elusive.


  • Chris Koehncke
    Posted on

    Chris Koehncke Chris Koehncke

    Author

    Good point, clearly VP9/H265 will consume more power. I find it ironic when a user asked me about 4K video for their mobile phone, however, I/we could be wrong. Perhaps a real life fluid motion video on whatever size screen is the future. Whatever it is, hardware encoding is going to be needed.


  • toto
    Posted on

    toto toto

    Author

    I’m not the author but it can be interesting
    RADE: Resource-aware Distributed Browser-tobrowser 3D Graphics Delivery in the Web
    http://ubicomp.oulu.fi/files/wimob15.pdf


  • Chris Koehncke
    Posted on

    Chris Koehncke Chris Koehncke

    Author

    The whole notion of P2P, I don’t think, has been fully exploited. The diehards cling to a client-server model. For ‘today’ there are valid reasons for this model, however I do think to scale P2P with some type of “light” server is the ultimate winner.


  • Chris Koehncke
    Posted on

    Chris Koehncke Chris Koehncke

    Author

    I think Google has thrown in the towel on VP8. Chipset mfg think in terms of years and millions of copies and the argument of VP8 vs H.264 hasn’t been significant enough to warrant a ‘war’. As Google is clearly shifting to a enterprise sales model, they will continue to argue for what’s right but aren’t going to dig in purely on technical grounds. No codec ever wins the whole battle and new ones emerge as technology shifts.