Spotty battery life costs Apple's MacBook Pro its gold-star rating
Consumer Reports nerd baffled by bizarre results
Updated Geeks at Consumer Reports have, for the first time, declined to award a "recommended" status to an Apple laptop – after the latest MacBook Pro proved unreliable during testing.
The testers tried out the 13-inch and 15-inch MacBooks with the Touch Bar, and the 13-inch without Ive's new big idea in laptop design. The results were frankly bizarre.
"In a series of three consecutive tests, the 13-inch model with the Touch Bar ran for 16 hours in the first trial, 12.75 hours in the second, and just 3.75 hours in the third," said Jerry Beilinson, Consumer Reports electronics editor..
"The 13-inch model without the Touch Bar worked for 19.5 hours in one trial but only 4.5 hours in the next. And the numbers for the 15-inch laptop ranged from 18.5 down to 8 hours."
The testing methodology is to power up each laptop, download a series of 10 web pages sequentially in Safari until the device shuts down, and then repeat. Display brightness is set at 100 nits and the automatic brightness adjuster is turned off.
It's not the best testing methodology in the world, but it's not fatally flawed either and shouldn't account for such a wide variation in results. Beilinson said they had submitted the test logs to Apple but hadn't heard back on a cause.
Curiously, when a couple of the same tests were performed using Chrome instead of Safari then battery life improved considerably. Beilinson said the Chrome tests were insufficient to quantify the difference but that it might be something to consider for owners looking to escape the power cord.
The testers used store-bought laptops for the testing, rather than those provided by Cook & Co themselves. There have been a number of reports from Reg readers about dodgy battery times and Apple's response has been to turn off the estimated battery life monitor in the latest build of macOS Sierra.
We've asked Apple for comment and a laptop to try our own tests on. Unsurprisingly there has been no response. ®
Updated to add on January 10, 2017
A spokesperson for Apple has been in touch to say Consumer Reports' benchmarks uncovered a web browser bug that drains the MacBook Pro's battery charge. That programming flaw has now been fixed and released via the Safari beta program. Apple also claims the magazine ran its tests in Safari's developer mode which produces unfair results due to it disabling the web cache.
Consumer Reports counters that it performs the same tests across all laptops to get consistent results and give batteries a thorough workout.
Here's what Apple had to say:
We appreciate the opportunity to work with Consumer Reports over the holidays to understand their battery test results. We learned that when testing battery life on Mac notebooks, Consumer Reports uses a hidden Safari setting for developing web sites which turns off the browser cache. This is not a setting used by customers and does not reflect real-world usage. Their use of this developer setting also triggered an obscure and intermittent bug reloading icons which created inconsistent results in their lab. After we asked Consumer Reports to run the same test using normal user settings, they told us their MacBook Pro systems consistently delivered the expected battery life.
We have also fixed the bug uncovered in this test. This is the best pro notebook we’ve ever made, we respect Consumer Reports and we’re glad they decided to revisit their findings on the MacBook Pro.
Meanwhile, Consumer Reports says:
Modern laptops have a variety of sophisticated battery management techniques and settings built into both their hardware and operating system software ... Many of these settings are set by default to extend battery life. That’s generally a good thing. But because these settings are so variable and situation-dependent, we turn several of them off during testing.
We also turn off the local caching of web pages. In our tests, we want the computer to load each web page as if it were new content from the internet, rather than resurrecting the data from its local drive. This allows us to collect consistent results across the testing of many laptops, and it also puts batteries through a tougher workout.