Munich (Germany) and Westlake Village (CA) – Two weeks ago, TG Daily reported that external USB 2.0 devices may cause certain notebook computers to consume power excessively. In the interest of tracking down the source of the problem, Tom’s Hardware Guide worked with engineers from both Intel and Microsoft. Through our extensive collaborative research, we can report that an Advanced Configuration and Power Interface (ACPI) driver in Windows XP Service Pack 2 is the sole cause of the unusually high power drainage observed by all three teams. Today, Microsoft confirmed to TG Daily that it is developing a new patch to fix this particular driver bug, whose nature is somewhat different of those addressed by earlier Microsoft Knowledgebase postings.
“The TG Daily article brought to our attention that our existing workaround is difficult to implement,” said Michael Burk, PR manager for Microsoft’s client team. “We also learned that we will need to issue a BIOS update in addition to a Registry modification, to make sure that the fix will be stable.” While the existing workaround, which we reported on earlier, is somewhat efficient in regaining substantial battery time for notebooks connected to USB 2.0 devices, Burk told us this new patch will go even further to address these new issues.
In the meantime, Microsoft does not recommend that customers install the existing workaround, due to potential instability issues. Instead, Burk told us, the company recommends they wait for the patch as well as a BIOS update, both of which are currently under evaluation in Microsoft’s labs.
The power drain bug which Tom’s Hardware Guide engineers isolated is different from similar power drain issues whose causes and solutions have been made public by Microsoft since 2004. In the intervening weeks since our original story was published, our teams have been evaluating a non-public preliminary workaround, that Microsoft published internally and among selected partners in July of 2005. Our collaborative tests revealed that the new problem’s root cause is not – as we already had indicated in our initial article – Intel’s Core Duo platform itself, even though the problem crops up on Core Duo notebook systems. Instead, the root cause is a Microsoft ACPI driver that leaves the asynchronous task scheduler component continuously running.
Although the contents of Microsoft Knowledgebase article KB899179, which listed the company’s preliminary workaround, have been published on several Web sites, Burk noted that the article remains confidential, and its publication by outside sources has not been officially authorized. While we question Microsoft’s reasons for not having published the KB article until today, we have to respect the company’s guidelines.
Industry sources told us that Intel apparently also completed development of its own solution. Intel spokesperson Connie Brown could not confirm the availability of this fix, although she mentioned that Intel has been evaluating possible solutions “for some time.”
At the conclusion of our initial article, we promised we would find a viable solution to the power drain issue, that regains the battery time that is consumed, whether the final cause turned out to be a driver bug or some undiscovered chipset issue. Our findings jump-started a healthy public discussion, which prompted other publications to follow up. At the same time, they lit a fire under Intel and Microsoft, which fueled our collaborative search for the solution.
Although Microsoft responded to our initial investigation by releasing the preliminary fix it had been working on, we believed that simply publishing our preliminary conclusions and leaving them dangling in the air while we moved on to other matters, would be the same as publishing an incomplete account and expecting that alone to inspire others to find the solution. No amount of numbers, no matter how well researched, would compensate for thousands of users losing battery life.
Despite that dual-core notebooks are still virtually non-existent, at least in retail channels, and even Intel’s engineers had to scramble to get their hands on a production ready Napa notebook, both firms were able to replicate our test environment.
Microsoft’s six-month-old workaround
According to KB899179, the USB 2.0 driver installed by Windows XP SP2 “leaves the asynchronous scheduler component continuously running.” As a result, continuous memory accesses prevent the system from falling into deeper ACPI sleep states, commonly referred to as “C-states.” Precluded from powering down, USB 2.0 devices continue to draw power from the notebook battery.
The new fix released to us outlines a simple, seven-step modification to the Windows Registry, that can be applied within two or three minutes. As mentioned earlier, we did not receive permission from Microsoft to publish the patch, which – at least officially – is still only available to Microsoft partners under a confidentiality agreement. However, at the time of the publication of this article, the details of workaround were still available in a discussion thread on Slashdot.
Since our initial report, several readers pointed us to a public advisory from Microsoft, stating that computers running Windows XP SP2 on multiple processors with power management features “may experience decreased performance.” While Microsoft recommends to install this hotfix, this particular issue is not related to the USB 2.0 problem we discovered.
For the second phase of our testing, Asus – the manufacturer of the near-production-ready Core Duo notebooks we originally tested – was able to replace our near-production notebook with a regular production notebook.
The test setup:
In consultation with Intel’s engineers, we planned a new battery of benchmarks with the following scenarios:
Benchmarking Napa and Sonoma
Once again, we used MobileMark05 Reader. Instead of using a DVD Combo drive as our USB 2.0 device, we switched to an external hard drive, equipped with its own power supply. To eliminate all possible sources of additional or unusually high power consumption, the hard drive was left idle. According to our measurements, the HDD drew less than 1 watt of power from the notebook to which it was connected.
The first test confirmed our initial test results from the first article. The Asus notebook, whose normal battery time was 257 minutes, dropped to 183 minutes with the USB 2.0 drive connected – a loss of 74 minutes.
The addition of Microsoft’s Registry key did enable the Asus notebook to regain a significant portion of this battery time, as our benchmarks prove. With the Registry patched, battery time only dropped from 254 minutes to 234 minutes. While reclaiming 53 minutes indicated to us that Microsoft’s patch addressed the “ballpark” of the issue, our experience with playing in Microsoft’s ballparks told us that something which accomplishes a partial fix isn’t necessarily the same something which could accomplish a complete fix.
Disabling the USB camera, no patch installed and re-running the battery of tests, we discovered, did not positively impact battery life: Quiote the contrary was the case: Batterylife decreased by 15 Minutes to 242 minutes (4:02 hours).
So, what about single-core notebooks?
One of the leftover concerns from our initial investigation was, why did our Sonoma notebook appear to resist the driver bug? As Intel had indicated to us during one of our conference calls last week, all notebooks – not just Core Duo – may be affected by this same bug; and Microsoft’s existing Knowledgebase article also clearly stated that every notebook with power management capability may be affected.
So we decided to modify our test battery a little bit. Instead of copying data over the USB interface, we merely connected it to the notebook, so it would draw as little power as possible. Then we ran our complete battery of scenarios once again on the Sonoma notebook. To be as consistent as we could with our previous tests, we used an Asus W5000 notebook that integrated the identical components of the Core Duo notebook, with the exception of the processor and chipset (Pentium M 750, 915GME). The batteries for both the Sonoma and Napa notebooks have the same capacity.
The single-core notebook posted a running time of 191 minutes without the patch, and without a USB 2.0 drive connected. While both Napa and Sonoma notebooks use batteries with equivalent capacities, the Core Duo platform is able to extend battery life by as much as 35% over its predecessor, highlighting the progress Intel has made with its Napa platform. Connecting our USB 2.0 hard drive decreased the battery running time by 12 minutes, to a total of 179 minutes.
Even more interesting, however, we discovered the Microsoft workaround increased the overall battery time of our test notebook to 223 minutes without a USB 2.0 hard drive connected. With the HDD connected, the device topped its original battery running time: 202 minutes with a USB 2.0 hard drive connected, an increase of 17% over the running time without USB 2.0 drive and 13% with connected drive.
So previous platforms like Sonoma, Intel’s second generation Centrino Mobile technology platform, appear to be impacted by this issue.
However, the HDD’s impact on battery life was minor in both scenarios. Without the patch installed, battery time dropped 12 minutes, or 6.3%; with the patch installed, it declined by 21 minutes, or 9.5%. Overall, the patch provides the user with a gain in battery time of 23 minutes, which is reason enough to consider deploying Microsoft’s patch for single-core notebooks.
What about notebooks with integrated USB 2.0-Devices?
Asus’ integration of webcams in their W5F and W5000 systems raise the question of their impact on the system. Can users squeeze more battery life out of a notebook if they remove the camera driver and disable all USB ports? Both the W5F and W500a come with an option that lets users disable all USB ports through their BIOS.
So of course, we decided to also run the notebooks through this test scenario. What we discovered remains puzzling: Disabling the USB ports reduced the battery time for both our Asus notebooks.
Where is the battery leak?
The key result of our investigations of the power drain bug certainly is the fact that there currently is an opportunity for users to regain battery time they otherwise would have lost.
Users interested in technical details, however, may be interested in how energy is used up in the new as well as previous generation notebooks. Intel and a report on a different enthusiast site claim that the driver bug prevents the processor to reach a power saving “C3” state. In other words, the processor consumes more power in its idle mode than it should.
Initially, we researched this issue with tools that are integrated in Windows. For example, we followed the data generated recorded by the Performance Monitor of Windows XP, which we expected to deliver some indication of how both systems react when USB 2.0 devices are connected. The following graphs detail which energy states, C2 and C3, are reached when a USB 2.0 device is connected, during its operating and when it is removed.
Performance monitor recording of the Sonoma notebook. Click here to see the enlarged view.
Performance monitor recording of the Napa notebook. Click here to see the enlarged view.
According to these measurements, the power management of Core Duo works just as it should. When idle, the CPU rests in C3. With minor load, the absolute C3 time decreases. When the USB drive is disconnected, the CPU remains in C3 for nearly 100% of the time. Similar observations can be made, when these measurements are repeated with a Sonoma (Pentium M) notebook. So, where is then the battery leak?
During one of our phone conferences with Intel last week, engineers hinted that the Windows’ Performance Monitor may not be exact enough to record actual energy states. However, Intel did not provide us with a tool that could prove this claim
In such cases, we typically rely on third party sources. As we learned, there is an Intel-branded, but non-public, tool, which can log the different energy states. We recorded two videos here (with patch) and here (without patch), which prove that the Windows Performance Monitor does not show correct transitions of the CPU’s energy states. Also, the tool reveals that the Napa system in fact does reach its C3 state – with and without the secret patch. The difference simply is that the CPU is able to spend more time in C3 when the patch is applied. To conclude that C3 is never achieved by the system, however, is not accurate.
A brief explanation of the timeline of our videos: After about 10 seconds, the device is disconnected from the power outlet. The CPU moves from C2 into C3 immediately. After 20 seconds, the USB 2.0 device is connected, which is recognized after a 5-second delay. 20 more seconds into the recording, we the polling interval is reduced from 500 ms to 170 ms. As a result, the switches between C2 and C3 get more active.
Conclusion: Everyone wins, sort of
While solutions for certain problems require time to be tracked down and discovered, when our hearts and minds are in the right place, the user always benefits in the end. This power drain bug is one example: While we suspected a power problem for quite some time, and began investigating possible sources several weeks ago, simply observing, re-testing, and re-explaining certain behaviors fails to address our core objective: eliminating the problem for the user. There’s more to this story.
The final credit for discovering this power consumption issue does not go to Tom’s Hardware, but to a conscientious industry source who felt that it was necessary to publicize an issue that may harm the interests of notebook users. Intel told us very little in our initial discussions, confining its responses to the most conservative language, saying that the company believes to have done a “great job” implementing its USB 2.0 host into the ICH7-M controller, and pointing to “other components” that could be culpable. Technically, this was an unacceptable dead end, not only for us but for all notebook owners.
Thanks to our industry source, we had additional information that indicated to us that Intel knew more than it revealed at the time. What we did not know at that time was that Intel was restrained by a confidentiality agreement with Microsoft. But we believe our research, triggered by our industry source and shared with you along the way, compelled Intel to take the power drain issue more seriously, and to elevate the level of its communication with Microsoft from the back basement to the front office. According to the now famous KB posting, Microsoft has known about power drain issues related to USB 2.0 for more than half a year, but still declined to release a patch to correct the bug. Today, Microsoft is on track to make a final patch available that can be installed by all users, regardless of their knowledge of the System Registry. This outcome is reward enough to make our entire experience these last few weeks worthwhile. The reasons for companies’ secrecy may be a “B-level” story for another day, if anyone’s interested, once we’ve completed Job One: making your work easier and more productive.
Which notebooks require patching?
Axel Mino, a test engineer for German computer magazine PC Professionell, was benchmarking an Acer Core Duo notebook at the time we published our first article and made an interesting observation. His evaluation unit came pre-installed with the Registry modification outlined in KB899179, while our Asus unit did not include the workaround in the shipping production version.
Needless to say, we were curious whether there were more notebook manufacturers making use of the confidential patch. We asked representatives from Dell, Fujitsu, Hewlett-Packard, Lenovo, Sony and Toshiba last week, but did not receive answers until today. Our typically reliable industry sources, however, indicate that there may be very few notebook builders out there that have integrated the patch into their production units.
While we’ve communicated with several notebook manufacturers over the course of our research, only Asustek has offered a response to our inquiries. Its German division told our engineers, “Asustek has been informed about this issue, two weeks ago, in the first week of February. Before we are able to use the patch, we have to evaluate it in thorough quality testing and for possible side effects. In addition to that, we only can offer a patch if it has been officially released by Microsoft. In the meantime, customers can contact their Asus service center and partners. The patch will be provided to our customers by e-mail, as soon as the serial number of a device has been verified.”
Users who are interested in checking whether their notebook is patched, can simply use the guidelines outlined in the KB posting to locate the Registry key in question. At this time, we do not recommend installing the fix reported in this posting, even if we found the workaround to significantly improve the battery life of our test systems, and even though we didn’t notice any instabilities during our own tests thus far. Microsoft’s comprehensive fix, which will include a BIOS update, will certainly be worth the wait.