EXTREME Overclocking Forums
Home | Reviews | Forums | Downloads | $ EXTREME Deals $ | RealTime Pricing | Free Magazines | Gear | Folding Stats Newsletter | Contact Us


Go Back   EXTREME Overclocking Forums > Website Related Info (No Post Increase) > Tech News & Press Releases > The Front Page Headlines
Register Forum Rules FAQ Search Today's Posts Mark Forums Read

Welcome Guest Visitor! Please Register, It's Free and Fun To Participate!
The EXTREME Overclocking Forums are a place for people to learn how to overclock and tweak their PC's components like the CPU, memory (RAM), or video card in order to gain the maximum performance out of their system. There are lots of discussions about new processors, graphics cards, cooling products, power supplies, cases, and so much more!

You are currently viewing our boards as a "guest" which gives you limited access to view most discussions. You need to register before you can post: click the register link to proceed. Before you register, please read the forum rules. By joining our free community you will have access to post topics, communicate privately with other members (PM), respond to polls, upload your own pictures, and access many other special features. Registration is fast, simple, and absolutely free! To start viewing messages, select the forum that you want to visit from the selection below.

After you have registered and read the forum rules, you can check out the FAQ for more information on using the forum. We hope you enjoy your stay here!

Note To Spammers: We do not allow unsolicited advertising! Spam is usually reported & deleted within minutes of it being posted, so don't waste your time (or ours)!


Please Register to Post a Reply
 
Thread Tools
Old 09-14-2011, 03:56 PM   #1
SSPrncVegeta
Speak softly
SSPrncVegeta's Avatar
Senior Member
 
Posts: 7,103
Last Seen: 10-28-2014
Age: 29
From: Northern Va
iTrader: 1 / 100%
A new look at game benchmarking: Measuring multi-GPU micro-stuttering

Why FPS fails
As you no doubt know, nearly all video game benchmarks are based on a single unit of measure, the ubiquitous FPS, or frames per second. FPS is a nice instant summary of performance, expressed in terms that are relatively easy to understand. After all, your average geek tends to know that movies happen at 24 FPS and television at 30 FPS, and any PC gamer who has done any tuning probably has a sense of how different frame rates "feel" in action.
Of course, there are always debates over benchmarking methods, and the usual average FPS score has come under fire repeatedly over the years for being too broad a measure. We've been persuaded by those arguments, so for quite a while now, we have provided average and low FPS rates from our benchmarking runs and, when possible, graphs of frame rates over time. We think that information gives folks a better sense of gaming performance than just an average FPS number.
Still, even that approach has some obvious weaknesses. We've noticed them at times when results from our FRAPS-based testing didn't seem to square with our seat-of-the-pants experience. The fundamental problem is that, in terms of both computer time and human visual perception, one second is a very long time. Averaging results over a single second can obscure some big and important performance differences between systems.

We didn't set out to hunt down multi-GPU micro-stuttering. We just wanted to try some new methods of measuring performance, but those methods helped us identify an interesting problem. I think that means we're on the right track, but the micro-stuttering issue complicates our task quite a bit.

In fact, in a bit of a shocking revelation, Petersen told us Nvidia has "lots of hardware" in its GPUs aimed at trying to fix multi-GPU stuttering. The basic technology, known as frame metering, dynamically tracks the average interval between frames. Those frames that show up "early" are delayed slightly—in other words, the GPU doesn't flip to a new buffer immediately—in order to ensure a more even pace of frames presented for display. The lengths of those delays are adapted depending on the frame rate at any particular time. Petersen told us this frame-metering capability has been present in Nvidia's GPUs since at least the G80 generation, if not earlier. (He offered to find out exactly when it was added, but we haven't heard back yet.)

Measuring performance in frame-time allows for several unique advantages over frame rate. You can explain why performance can feel sluggish at higher framerates because of alternating quick and slow frames. You can also compare stuttering between cards.

Would you guys prefer this method of measuring performance to the standard FPS?


Source: TechReport

Also check out this follow-up article with video.

Last edited by SSPrncVegeta : 09-15-2011 at 04:43 PM.
United States  Offline
    Register to Reply to This Post
Old 09-15-2011, 09:56 AM   #2
vai0
><>
vai0's Avatar
Senior Member
 
Posts: 3,000
Last Seen: Yesterday
Age: 23
From: Florida
iTrader: 0 / 0%
Very nice article micro stutter captured on graph and video.
United States  Offline
    Register to Reply to This Post
Old 09-15-2011, 11:04 AM   #3
Czarspuppet
I get around
Czarspuppet's Avatar
Senior Member
 
Posts: 402
Last Seen: 02-29-2012
Age: 33
From: Earth
iTrader: 0 / 0%
Could he be looking at it the wrong way? Frames per second is a statistic that doesn't reflect standard deviation (error, distribution, whichever you prefer).

The problem with FPS is that at 300 it doesn't rule out the possibility of the machine having spent half a second on one frame, which humans perceive as 2 FPS.
United States  Offline
    Register to Reply to This Post
Old 09-15-2011, 12:03 PM   #4
SSPrncVegeta
Speak softly
SSPrncVegeta's Avatar
Senior Member
 
Posts: 7,103
Last Seen: 10-28-2014
Age: 29
From: Northern Va
iTrader: 1 / 100%
Quote:
Originally Posted by Czarspuppet View Post
Could he be looking at it the wrong way? Frames per second is a statistic that doesn't reflect standard deviation (error, distribution, whichever you prefer).

The problem with FPS is that at 300 it doesn't rule out the possibility of the machine having spent half a second on one frame, which humans perceive as 2 FPS.
That's why the article argues against FPS in favor of frame time, with being able to measure variation being one of the advantages mentioned in the article.
United States  Offline
    Register to Reply to This Post
Old 09-15-2011, 02:59 PM   #5
Sephious
"Doctor"
Sephious's Avatar
Senior Member
 
Posts: 3,362
Last Seen: 12-17-2013
Age: 25
From: Saskatchewan
iTrader: 8 / 100%
I love this. I, for one, can see micro-stuttering so I welcome this benchmarking type.
Canada  Offline
    Register to Reply to This Post
Old 09-15-2011, 05:21 PM   #6
SSPrncVegeta
Speak softly
SSPrncVegeta's Avatar
Senior Member
 
Posts: 7,103
Last Seen: 10-28-2014
Age: 29
From: Northern Va
iTrader: 1 / 100%
I'm testing this out right now. Info in a bit.
United States  Offline
    Register to Reply to This Post
Old 09-15-2011, 06:24 PM   #7
K31TH3R
Reality in 01101011's
K31TH3R's Avatar
Senior Member
 
Posts: 4,480
Last Seen: 06-15-2014
Age: 27
From: Iowa
iTrader: 22 / 100%
FINALLY someone who has accurately captured and shown microstuttering.

Someone give these people a cookie, this is great. Frame-time measurements is THE WAY TO GO. FPS cannot show this, and this is genius.

Time to adapt to a new form of measurement, this makes loads of sense.

Good find!
United States  Offline
    Register to Reply to This Post
Old 09-15-2011, 06:34 PM   #8
SSPrncVegeta
Speak softly
SSPrncVegeta's Avatar
Senior Member
 
Posts: 7,103
Last Seen: 10-28-2014
Age: 29
From: Northern Va
iTrader: 1 / 100%
Well, Batman:AA was a poor choice because I get 200fps (usually <6ms time frame). I'll do what they did in that video and restrict the framerate with vsync.

Quote:
Originally Posted by K31TH3R View Post
FINALLY someone who has accurately captured and shown microstuttering.

Someone give these people a cookie, this is great. Frame-time measurements is THE WAY TO GO. FPS cannot show this, and this is genius.

Time to adapt to a new form of measurement, this makes loads of sense.

Good find!
I love messing with statistics, so I'm enjoying myself interpreting the data.
United States  Offline
    Register to Reply to This Post
Old 09-15-2011, 07:13 PM   #9
SSPrncVegeta
Speak softly
SSPrncVegeta's Avatar
Senior Member
 
Posts: 7,103
Last Seen: 10-28-2014
Age: 29
From: Northern Va
iTrader: 1 / 100%
I chose these specific part of the benchmark (near the end when the camera flies by the skull pile) before seeing the data because I could visibly detect the stuttering. And... there it is.

Attached Thumbnails
Click image for larger version

Name:	microstuttering.png
Views:	1036
Size:	35.7 KB
ID:	160109  
United States  Offline
    Register to Reply to This Post
Old 09-15-2011, 07:23 PM   #10
vai0
&gt;&lt;&gt;
vai0's Avatar
Senior Member
 
Posts: 3,000
Last Seen: Yesterday
Age: 23
From: Florida
iTrader: 0 / 0%
Great graph Vegeta.

Could you feel micro-stuttering? I wonder how far apart each frame must be for the average person to feel it.
United States  Offline
    Register to Reply to This Post
Old 09-15-2011, 07:26 PM   #11
SSPrncVegeta
Speak softly
SSPrncVegeta's Avatar
Senior Member
 
Posts: 7,103
Last Seen: 10-28-2014
Age: 29
From: Northern Va
iTrader: 1 / 100%
The B:AA benchmark is a flyby, so it's more like watching a movie. Let me record myself playing, stop when I notice it, and look at the data.
United States  Offline
    Register to Reply to This Post
Old 09-15-2011, 07:45 PM   #12
SSPrncVegeta
Speak softly
SSPrncVegeta's Avatar
Senior Member
 
Posts: 7,103
Last Seen: 10-28-2014
Age: 29
From: Northern Va
iTrader: 1 / 100%
Here's a playthrough of the first part of the first challenge in B:AA. My games always feel like somewhat jittering when one alternating time frame is significantly different from the other. This explains why.

Attached Thumbnails
Click image for larger version

Name:	microstuttering2.png
Views:	1044
Size:	32.3 KB
ID:	160110  
United States  Offline
    Register to Reply to This Post
Old 09-15-2011, 07:50 PM   #13
Czarspuppet
I get around
Czarspuppet's Avatar
Senior Member
 
Posts: 402
Last Seen: 02-29-2012
Age: 33
From: Earth
iTrader: 0 / 0%
Is this a SLI/XFIRE problem or do single GPU systems do it too? (perhaps in different circumstances)
United States  Offline
    Register to Reply to This Post
Old 09-15-2011, 07:57 PM   #14
SSPrncVegeta
Speak softly
SSPrncVegeta's Avatar
Senior Member
 
Posts: 7,103
Last Seen: 10-28-2014
Age: 29
From: Northern Va
iTrader: 1 / 100%
Quote:
Originally Posted by Czarspuppet View Post
Is this a SLI/XFIRE problem or do single GPU systems do it too? (perhaps in different circumstances)
The problem comes from inefficiencies of Crossfire/SLI handling multiple GPUs. A single GPU will always provide its peak performance.

This image sums it up best. Vertical variance = micro-stuttering. Notice how it's significantly less on the single-GPU 6970? Even though the 6990 is performing better, it's stuttering like crazy. A consistent, somewhat longer time frame is more important than varying, faster time frame performance.

Image is from page 4 of the article.
Attached Thumbnails
Click image for larger version

Name:	bc2-amd.gif
Views:	537
Size:	11.0 KB
ID:	160111  
United States  Offline
    Register to Reply to This Post
Old 09-26-2011, 04:40 PM   #15
webtax
Running System Stock
Forum Newbie
 
Posts: 0
Last Seen: 11-01-2011
it would be nice to see if there's a difference between nvidia and ati in this aspect
Chile  Offline
    Register to Reply to This Post
Old 09-26-2011, 05:14 PM   #16
SSPrncVegeta
Speak softly
SSPrncVegeta's Avatar
Senior Member
 
Posts: 7,103
Last Seen: 10-28-2014
Age: 29
From: Northern Va
iTrader: 1 / 100%
Someone with two Nvidia GPUs... GO!!
United States  Offline
    Register to Reply to This Post
Old 09-26-2011, 10:47 PM   #17
xdownsetx
Menior Sember
xdownsetx's Avatar
Senior Member
 
Posts: 1,366
Last Seen: 10-12-2014
Age: 28
From: Anaheim, CA
iTrader: 1 / 100%
Vegeta, give these leaked 11.9's a shot. I'm interested to see if anything is different, however unlikely.

http://www.guru3d.com/news/amd-catal...out-for-grabs/
United States  Offline
    Register to Reply to This Post
Old 09-26-2011, 10:57 PM   #18
SSPrncVegeta
Speak softly
SSPrncVegeta's Avatar
Senior Member
 
Posts: 7,103
Last Seen: 10-28-2014
Age: 29
From: Northern Va
iTrader: 1 / 100%
Why would 11.9 fixed an issue that's been in Crossfire and SLI since their debut? I'll give them a try and post my results, but I doubt they'll do anything close to fixing the problem.
United States  Offline
    Register to Reply to This Post
Old 09-27-2011, 01:26 AM   #19
nhan4
Overclocker
nhan4's Avatar
Senior Member
 
Posts: 393
Last Seen: Today
Age: 27
From: San Francisco
iTrader: 13 / 100%
The first time I used crossfire on 2 6870s(Q6600 Abit P35 system), I've encountered micro-stuttering, I was running way more than 60 FPS(using fraps) but it did not look like smooth 60 FPS. I thought it was maybe because one card was running 16x and the other was 4x.

Upgraded to sandy bridge p67 platform(8x,8x), that problem is gone.
United States  Offline
    Register to Reply to This Post
Old 09-27-2011, 01:36 AM   #20
Josie Wales
Extreme Overclocker
Josie Wales's Avatar
Senior Member
 
Posts: 2,373
Last Seen: 10-29-2014
From: St Paul
iTrader: 4 / 100%
Great article SSPrncVegeta. Thanks for hooking us up.
United States  Offline
    Register to Reply to This Post
Sponsored Links:
Please Register to Post a Reply


Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Forum Jump

All times are GMT -6. The time now is 11:41 AM.

Copyright 2000 - 2011, Jelsoft Enterprises Ltd.
Powered by vBulletin
Copyright 2000 - 2011, EXTREME Overclocking