StarCraft II Wings of Liberty - Beta Performance
Posted on: 02/20/2010 02:51 AM

Today we are going to run some preliminary performance tests using Blizzards upcoming game, StarCraft II Wings of Liberty, which has just entered the beta testing phase. Although the results will be based on a beta version of the game, they should give you a fairly good idea of how your gaming system will handle this exciting real-time strategy game...

StarCraft II is a game that needs no introduction, largely because you have all been hearing about it for many years. The game has been in the making for what seems like forever, and has been delayed almost as long as Nvidia’s upcoming Fermi architecture (sarcasm). Still the delays do not seem to have discouraged anyone from wanting to play this game, and if anything, it has made us desperate to get a taste of what this new real-time strategy game is all about.

A handful of lucky gamers are getting that taste as we speak, with the beta version of StarCraft II Wings of Liberty. Although a release date has not officially been announced, Blizzard are claiming to release the game this year and honestly, they had better do so! Still, with the game currently in beta testing stages, this is a good sign that the release date is not all that far away.

Therefore, after a long wait, it is now time to work out if your gaming rig requires any last minute updates to help you enjoy StarCraft II to its fullest. Although real-time strategy games are often not nearly as visually demanding as first person shooters, there have been games that have been able to bring even the most capable rigs to their knees, Supreme Commander springs to mind as an example.

Given that there will be a massive audience for StarCraft II, we suspect that it will be optimized for a range of hardware. After all, the original game became the best-selling PC game for that year, selling over 1.5 million copies worldwide. This was an impressive record to set 12 years ago, back in 1998, and if that was not impressive enough, StarCraft sold 9.5 million copies across the globe within a decade of its release.

Naturally then, Blizzard has high hopes for the second installment of the StarCraft franchise and so they should, since the initial release of the original StarCraft game the Battle.net online multiplayer service has grown by a staggering 800%. Believe it or not, after all this time StarCraft remains one of the most popular online games in the world.

That is saying quite a bit, as other popular real-time strategy games, such as Command & Conquer: Red Alert 3, are lucky to survive more than a couple of years with a significant amount of players logged on. Of course we are not going to dare compare the quality of a half baked game like Red Alert 3 to StarCraft II, at least not yet anyway. With that said, let’s move on to discuss the testing process...


Printed from Legion Hardware (http://www.legionhardware.com/articles_pages/starcraft_ii_wings_of_liberty_beta_performance,1.html)


Testing Methodology
StarCraft II Wings of Liberty - Beta Performance
Posted on: 02/20/2010 02:51 AM

In total we tested 18 different graphics card configurations using both ATI and Nvidia chips that ranged from the ultra expensive models to budget-minded offerings. For testing we have only used the “ultra” quality preset, without the anti-aliasing or anisotropic filtering quality settings enabled.

Please note that the beta does not include the option to enable anti-aliasing or anisotropic filtering in-game. Furthermore, it was not possible to force these settings when using any of the ATI graphics cards, they would simply ignore the settings in-game. However, the Nvidia based cards were able to use AA/AF quality settings with the latest drivers.

Despite this, we opted not to run any anti-aliasing or anisotropic filtering quality settings just yet, as we would have no comparison for the Nvidia cards. On a side note, in terms of performance it appears that the Nvidia driver is further along than the current ATI drivers.

The latest official drivers were used for all graphics cards, which saw both Crossfire and SLI working correctly. We did also try some unreleased drivers from ATI, but this did not allow us to use AA/AF in-game and it did not improve performance in any way.

For measuring frame rates we relied on Fraps, where we recorded five minutes of game play using a replay of a 4-player online battle. The Intel Core i7 965 EE processor was overclocked to 3.7GHz in an attempt to remove any CPU bottlenecks that could influence high-end graphics cards scores.

Please note that each test was conducted 6 times, with a pre-cache test as well. First we ran a pre-cache test, which often produced very low minimum frame rate results. After that the test was run 6 more times and the average and minimum frame rates were recorded. This gives us very accurate results of both the minimum and average frame rates.

However, we know that many of you will be interested in CPU scaling performance as well, and therefore we down clocked the Intel Core i7 965 EE processor to see what kind of impact this had on performance. Ideally we would like to test a number of platforms, though given this is only a limited beta version of StarCraft II we will hold off on the full scale testing until we have the final code.

Finally, we will be looking for a minimum of 30fps for stutter-free game play.

Test System Specs
Hardware
- Intel Core i7 965 @ 3.70GHz

- x3 2GB G.Skill DDR3 PC3-12800 (CAS 9-9-9-24)

- Seagate 500GB 7200-RPM (Serial ATA300)

- Radeon HD 5870 (1GB) Crossfire
- Radeon HD 5870 (1GB)
- Radeon HD 5850 (1GB)
- Radeon HD 5770 (1GB)
- Radeon HD 5750 (1GB)
- Radeon HD 5670 (512MB) Crossfire
- Radeon HD 5670 (512MB)
- Radeon HD 5570 (1GB)
- Radeon HD 4890 (1GB)
- Radeon HD 4870 (1GB)
- Radeon HD 4850 (1GB)
- Radeon HD 4830 (512MB)
- Radeon HD 4770 (512MB)
- GeForce GTX 285 (1GB)
- GeForce GTX 275 (896MB)
- GeForce GTX 260 (896MB)
- GeForce GT 240 (512MB)
- GeForce 9800 GT (512MB)
- GeForce 9600 GT (512MB)

- Asus P6T Deluxe

Software
- Microsoft Windows 7 Ultimate (64-bit)
- ATI Catalyst 10.2
- Nvidia Forceware 196.21




Printed from Legion Hardware (http://www.legionhardware.com/articles_pages/starcraft_ii_wings_of_liberty_beta_performance,2.html)


2560x1600 Ultra Performance
StarCraft II Wings of Liberty - Beta Performance
Posted on: 02/20/2010 02:51 AM

Starting at 2560x1600, we have good news for owners of mid-range graphics cards. Even at this extreme resolution the Radeon HD 4770 and 4850 were able to deliver perfectly playable performance. Again keep in mind that we are using maximum in game quality settings, while AA/AF options are not available. The GeForce GTS 250 and Radeon HD 5750 were also very impressive, with minimum frame rates of 40fps which will ensure constantly smooth game-play.

Basically any graphics card above the GeForce GTS 250 or Radeon HD 5750 in this graph is going to easily play StarCraft II using maximum quality settings based on this beta performance. The Radeon HD 5770 for example, delivered a minimum of 48fps with an average of 54fps. The GeForce GTX 260 was even more impressive with a minimum and average of 55/61fps.

The GeForce GTX 285 matched the Radeon HD 5850 with just over 70fps as the minimum and just over 80fps for the average. The Radeon HD 5870 was faster again with a minimum of 87fps, which is clearly overkill. That said, once the AA/AF settings can be enabled, this headroom will be greatly desired. As you can see Corssfire is working perfectly, with a pair of Radeon HD 5870 graphics cards spitting out an average of 135fps!


Printed from Legion Hardware (http://www.legionhardware.com/articles_pages/starcraft_ii_wings_of_liberty_beta_performance,3.html)


1920x1200 Ultra Performance
StarCraft II Wings of Liberty - Beta Performance
Posted on: 02/20/2010 02:51 AM

As we reduce the resolution to 1920x1200 we see that frame rates have increased quite significantly and now even the Radeon HD 5670 is able to deliver perfectly playable performance with 35fps. The Radeon HD 5570 still struggles here and will likely require a further reduction in resolution. We were surprised to find so many graphics cards now considered to be low-end, rendering a minimum of 40fps.

The old GeForce 9800 GT averaged 55fps with a minimum of 49fps, which is again more than enough for perfectly playable performance. The Radeon HD 5750 also does very well with a minimum of 53fps, while the 5770 rendered a minimum of 62fps. Anything faster than this can be again considered overkill and that includes the GeForce GTX 260.


Printed from Legion Hardware (http://www.legionhardware.com/articles_pages/starcraft_ii_wings_of_liberty_beta_performance,4.html)


1680x1050 Ultra Performance
StarCraft II Wings of Liberty - Beta Performance
Posted on: 02/20/2010 02:51 AM

Now at 1680x1050 the low-end/mid-range graphics cards see around a 5fps performance boost, while the higher end cards only gain a few frames per second. At this resolution virtually all the tested graphics cards were capable of delivering perfectly playable performance in StarCraft II. While the Radeon HD 5570 will deliver acceptable performance most of the time, it may suffer from lag during intense battles.

Update: Please be aware that we just updated this graph. The original had the GeForce GTX 260 and GTS 250 muddled up and the resolution of the graph was wrong, sorry for the confusion.


Printed from Legion Hardware (http://www.legionhardware.com/articles_pages/starcraft_ii_wings_of_liberty_beta_performance,5.html)


CPU Scaling – Core i7 9xx
StarCraft II Wings of Liberty - Beta Performance
Posted on: 02/20/2010 02:51 AM

CPU Scaling is not something we always find necessary to test when dealing with first person shooters. However when it comes to real-time strategy games it’s a must, and while we would have really liked to have conducted far more testing here, there simply is not enough time. Furthermore, we feel it would make more sense to wait for the game to exit the beta stage before conducting extensive testing.

That said, the results above clearly show that the CPU is an important factor for StarCraft II performance. Nonetheless, keep in mind that these results do show extreme scenarios, as we are using a Radeon HD 5870 at 1680x1050 without any AA/AF quality settings enabled. Here the GPU is not working nearly as hard as it could. Using a Radeon HD 5750, for example, would provide far less extreme results, and this is something we will show in future StarCraft II performance articles.


Printed from Legion Hardware (http://www.legionhardware.com/articles_pages/starcraft_ii_wings_of_liberty_beta_performance,6.html)


Multi-core Performance
StarCraft II Wings of Liberty - Beta Performance
Posted on: 02/20/2010 02:51 AM

Next we decided to test multi-core performance by measuring the impact disabling cores on the Core i7 processor had. Not only have we measured the impact this had on performance, but we have also taken screen shots of the CPU utilization levels from the Windows Task Manager with the processor clocked at 2.70GHz. Finally, please note that the dual core configuration still had 4 threads at its disposal and the single core configuration 2 threads, as Hyper-Threading was enabled.

Although we do see a nice 25% performance increase when going from 2 to 4 cores in StarCraft II with the Core i7 processor at 2.70GHz, the game does only appear to be dual-core optimized when looking at the Windows Task Manager CPU utilization data.

That said, we saw just a 9% increase for the average frame rate when going from a single core (with HT) to a dual-core configuration with Hyper-Threading. While the average frame rate only increased by 9%, the minimum frame rate was boosted by 43%, and an increase of just 20% was seen when going from a dual-core to a quad-core configuration.

With 8 threads (4 cores) enabled, half the threads are seen to be doing absolutely nothing, these are the threads enabled by Hyper-Threading. However if you were to think that Hyper-Threading does nothing for StarCraft II you would be wrong. The single core data with Hyper-Threading enabled shows the second thread to be working almost as hard as the first, with a total CPU utilization level of over 85%. The problem with Hyper-Threading enabled on four dedicated cores is that there is nothing left for it to do.

With 4 threads enabled the CPU utilization drops to around 30% and here we see just the first thread working hard. The Hyper-Threading enabled threads are doing very little work, while only about 10-20% of the second core is being used. With 8 threads enabled the data is much the same, though because there are 4 threads doing nothing the utilization drops below 20%.

For the sake of finding out we disabled Hyper-Threading with just a single 2.70GHz core enabled. The CPU utilization was locked at 100% the entire time and this is why we see spikes in the minimum frame rate when using less cores or threads. As it stands StarCraft II appears only to be dual-core optimized, no surprises there really.

That said, while the game only appears to heavily use two cores, we still saw some activity across four when enabled. Moreover, the performance was considerably better when enabling all four cores with Hyper-Threading, not that we think Hyper-Threading is making any tangible difference here. Bottom-line, you are going to want a dual-core processor at the very least to play StarCraft II, while we recommend getting your hands on a quad-core processor for the best performance.


Printed from Legion Hardware (http://www.legionhardware.com/articles_pages/starcraft_ii_wings_of_liberty_beta_performance,7.html)


Conclusion
StarCraft II Wings of Liberty - Beta Performance
Posted on: 02/20/2010 02:51 AM

StarCraft II is not nearly as demanding as I imagined it might have been, and despite looking quite impressive, it was very playable on mid-range graphics cards using maximum in-game quality settings at extreme resolutions such as 2560x1600. Although we have only given you a preview of the performance that is to be expected from StarCraft II by using a beta copy, this should give gamers a good idea of how their gaming system will handle this new title.

At this stage we are not going to discuss the ATI vs. Nvidia comparison in detail, as we feel the drivers are not quite up to speed, particularly on ATI’s behalf. Still, the Radeon HD 5870 was the fastest single GPU graphics card available when testing with StarCraft II, while the GeForce GTX 285 was able to match the Radeon HD 5850.

The GeForce GTX 260 was on the other hand much faster than the Radeon HD 4870, and it was a similar case when comparing the GeForce GTX 275 to the Radeon HD 4890. The Radeon HD 4850 was slightly slower than the GeForce 9800 GT, while the Radeon HD 5670 was also slightly slower than the GeForce 9600 GT.

Still, in terms of performance things are just about where they should be, and I have to admit to being a little surprised that Crossfire worked so well. Something we noticed right away with benchmarking with StarCraft II was how stable the game is already. With a wide range of graphics cards we did not encounter a single issue, no graphical glitches and no crashes - quite impressive really.

Although we were not testing with anti-aliasing or anisotropic filtering quality settings enabled, the game still looked very sharp. That said, most gamers are going to want to enable these settings and once it is possible to use AA in-game with the ATI graphics cards, we will certainly re-test using the AA/AF quality settings.

When reading through the descriptions of the various quality settings, we noticed that Blizzard does state that the Reflections and Effects rely on the CPU and not the GPU. Therefore it makes perfect sense that we saw a 27% performance increase when clocking our Core i7 processor from 2.70GHz to 3.20GHz. There is still a great deal of testing to be done here and we plan to test a range of processors, both dual and quad-core models.

For now however it would seem that the more you can squeeze out of your processor the better, assuming you are not happy with the level of performance you are currently receiving. That said, even with the Core i7 processor under-clocked to just 1.70GHz we still received playable performance, despite dropping a significant amount of frames.

When it comes down to which graphics card is best to use with StarCraft II, there is any number of $200 US plus options to choose from. That said, without AA/AF enabled even affordable graphics cards, such as the Radeon HD 5750 ($140 US), can deliver perfectly playable performance at resolutions up to 2560x1600.


Printed from Legion Hardware (http://www.legionhardware.com/articles_pages/starcraft_ii_wings_of_liberty_beta_performance,8.html)