VR Benchmarking Results, using FCAT from nVidia

Discussion in 'Technical & Support' started by MarcG, Jun 3, 2017.

  1. MarcG

    MarcG Registered

    Joined:
    Oct 4, 2010
    Messages:
    6,854
    Likes Received:
    2,234
    In this Post I shall be keeping a track of Benchmarking Results through each Build starting with Build 1861431, not only that but in the following posts/pages I'll do additional Benchmarking with various Graphical Setting changes to help better understand which areas put more strain on my Rift CV1. This may help you to find your own preference in settings and may help Studio397 in further optimisation of VR.

    The Software I am using & the User Guides are available at the link below (it's all free):
    http://www.geforce.com/hardware/technology/fcat/downloads

    My Benchmark Routine will be the same for every Build, it consists of the following:
    • Asynchronous Spacewarp (ASW) is set to AUTO
    • Pixels Per Display (PPD) is set to 1.4 in the Oculus Tray Tool. SteamVR * due to a change in SteamVR this is now set to 1.9 *
    • Folders Cbash & Shaders deleted before running RF2
    • Default player.JSON
    • Honda Civic
    • Malaysia South Loop
    • AI Control
    • 60 Second Benchmark starting on the Start/Finish Line (Rolling Start)
    • Keeping my head Still as any movement of the HMD could skew results
    I have applied the following Settings:
    • Graphics Settings.jpg
    • Video Res.jpg
    • Weather.jpg
    • Video Settings.png
    Edit: Thanks to @The Iron Wolf for pointing out at the Higher Frametime means Lower FPS, so the best FPS Results are at the bottom.
    ======================================================================

    June 3rd - Build 1861431 - First Benchmark
    June 6th - Build 1868941
    June 9th - Build 1879993 - *Removed from Results - Change in SteamVR SS Settings & possible issues with the Software may have affected this Benchmark
    June 21st - Build 1905385 - Better performance during Race Start
    June 27th - Build 1915820 - Continued improvement

    Build Benchmarks.png
     
    Last edited: Jul 12, 2017
  2. MarcG

    MarcG Registered

    Joined:
    Oct 4, 2010
    Messages:
    6,854
    Likes Received:
    2,234
    Post Effects Benchmark - Build 1861431 to see what the difference is between the Levels of None through to Ultra.

    As you can see in the first picture No & Low Post Effects are roughly the same, in fact in game with this Build (1861431) I could barely notice any visual difference.

    Build 1861431 Post Effects.png

    With the Plot Graph added you can see the difference between No PP and Ultra PP, this really gives you a look at just how many Dropped Frames there are with Ultra. Now I'm no VR Expert when it comes to fully understanding most of the jargon involved with rendering to the device, but even with my lack of knowledge it's clear to me just how much of an impact Ultra has over No Post Effects!

    Build 1861431 Post Effects - Frames.png

    ==================================================================

    Pixels Per Display Benchmark - Build 1861431

    Settings the same as the first post except:
    • AI 0
    • AA Off
    As you can see from the Graphs (the program limits 8 Plots per graph so two Graphs shown) you can see that there's a steady decrease from 1.0 to 1.5 but then there's a jump to 1.6, It then goes steady again until 2.0 which is a big jump from 1.9.

    Build 1861431 - PPD 1.0 to 1.7.png
    Build 1861431 - PPD 1.3 to 2.0.png

    In this picture I've included the FPS Numbers, pay particular notice to the Unconstrained FPS column.
    Build 1861431 PPD Numbers.png

    Finally the Plot Graph with even PPD from 1.0 to 2.0 selected, there's not a great deal of difference due to no other AI on track, but 2.0 has got Dropped Frames as you can see below.
    Build 1861431 - PPD Plot Graph 1.0 to 2.0.png

    ====================================================================

    Anti-Aliasing (AA) Benchmark - Build 1861431

    Same settings as the first post except:
    • AI 0
    • PPD 1.0
    This one is very interesting, notice that AA Off, 1 & 2 are a steady decrease in FPS, whilst 3,4 & 5 are almost all identical. There are dropped frames in AA 5 as you can see in the Plot Graph.
    Build 1861431 - AA.png
     
    Last edited: Jun 22, 2017
  3. MarcG

    MarcG Registered

    Joined:
    Oct 4, 2010
    Messages:
    6,854
    Likes Received:
    2,234
    Shadows Benchmark - Build 1861431

    Same overall settings as the first post but instead I chose Monte Carlo (Official Studio397 content) due to it's sheer volume of Shadows!

    In this first image (better FPS Graphs are the lowest) you can see there's nt a lot of difference between OFF and LOW, even MEDIUM is not that much worse. HIGH isn't too bad as it's not until you chose MAX that there's a massive jump and loss of FPS. I'd say on Shadow Friendly Tracks on my system I could possibly use MEDIUM whereas before this test I've been running on LOW.

    Build 1861431 - Shadows.png
    The Plot Graph again between OFF & MAX shows the difference

    Build 1861431 - Shadow Plot.png

    =============================================================================

    Weather Benchmark - Build 1861431

    Sunny, Default & Scripted were all the same but add in Overcast and Raining and there's a difference, not a massive difference but certainly a noticeable drop in performance with Raining.
    Build 1861431 - Weather.png

    ==============================================================================

    Environment Reflections - Build 1861431

    Pretty standard/as expected loss of FPS and dropped frames, perhaps LOW isn't too much of a loss as it stays fairly close to OFF in some areas.
    Build 1861431 - Environment Reflections.png
     
    Last edited: Jun 22, 2017
  4. The Iron Wolf

    The Iron Wolf Registered

    Joined:
    Feb 20, 2016
    Messages:
    984
    Likes Received:
    984
    Marc, super thanks for doing this. We really need structured benchmarks for VR.

    I wanted to suggest couple of things:
    * It might be better to measure with ASW off to get that out of the way
    * Depending on what you want to measure (CPU vs GPU) you may want to remove GPU bounding settings out of the equation - PPD/AA.

    In any case, this is great start. Is that single car (edit: I see 19AI, right)?
     
    vegaguy5555 and bravotangosix like this.
  5. MarcG

    MarcG Registered

    Joined:
    Oct 4, 2010
    Messages:
    6,854
    Likes Received:
    2,234
    yeah I may have to do ASW Off, locked to 45 & Auto to see if there's a big difference but if there's not much between the three then I'll stick with just one.
    I'm only Benchmarking to see if each future Build improves on performance/optimisation so I'm not fussed about CPU vs GPU results, as with all benchmarking it is solely down to the PC it's being done on so it won't be of a major help to everyone, more of a little guide perhaps as I go through the other Setting changes :)

    Edit: For reference here's the difference of ASW, the Build 1861431 is AUTO
    Build 1861431 ASW.png
     
    Last edited: Jun 3, 2017
  6. The Iron Wolf

    The Iron Wolf Registered

    Joined:
    Feb 20, 2016
    Messages:
    984
    Likes Received:
    984
    As you can see Marc, ASW inserts synthetic (fake, interpolated frames, yellow color). This is super interesting.
     
    vegaguy5555, bravotangosix and MarcG like this.
  7. MarcG

    MarcG Registered

    Joined:
    Oct 4, 2010
    Messages:
    6,854
    Likes Received:
    2,234
    Post #3 Updated with Shadow Tests, that's all for tonight, I have some time off work coming up and will do a Pixels Per Display (PPD) and AA Benchmark when time allows.
     
    Last edited: Jun 3, 2017
  8. Eddy

    Eddy Registered

    Joined:
    Jun 27, 2013
    Messages:
    1,649
    Likes Received:
    394
    Thanks Marc great stuff!!!
     
    vegaguy5555 and MarcG like this.
  9. The Iron Wolf

    The Iron Wolf Registered

    Joined:
    Feb 20, 2016
    Messages:
    984
    Likes Received:
    984
    There's a really nice outcome of this thread already - we now do know that there's FPS difference between high and full shadows. Curious to see what effect shadow filtering has on VR FPS. Turns out this tool has ability to show CPU time as well, I'll dig into that when I can :)
    Thanks Marc for doing this, thread with numbers is really useful.
     
    vegaguy5555, bravotangosix and MarcG like this.
  10. The1Dijk

    The1Dijk Registered

    Joined:
    Jun 12, 2014
    Messages:
    32
    Likes Received:
    10
    Good stuff Marc, thanks a lot. I'm using the same GPU along with a slightly better CPU, so this is very much comparable to my system. It also seems from your tests that PP on low should be possible too without having too much of an impact.
     
    vegaguy5555 and MarcG like this.
  11. MarcG

    MarcG Registered

    Joined:
    Oct 4, 2010
    Messages:
    6,854
    Likes Received:
    2,234
    Yes that and shadows on low as well I think, maybe stretch to Medium shadows on less hungry tracks. Be interesting to see the PPD results, hopefully tomorrow I'll get them done.
     
    vegaguy5555 likes this.
  12. MarcG

    MarcG Registered

    Joined:
    Oct 4, 2010
    Messages:
    6,854
    Likes Received:
    2,234
    Post #2 updated with Pixels Per Display and Anti-Aliasing (AA) Benchmarks
     
    Last edited: Jun 5, 2017
  13. MarcG

    MarcG Registered

    Joined:
    Oct 4, 2010
    Messages:
    6,854
    Likes Received:
    2,234
    A quick Weather Benchmark added to Post #3.
     
    bravotangosix likes this.
  14. MarcG

    MarcG Registered

    Joined:
    Oct 4, 2010
    Messages:
    6,854
    Likes Received:
    2,234
    Environment Reflections added to Post #3
     
    bravotangosix and The Iron Wolf like this.
  15. Clemenca

    Clemenca Registered

    Joined:
    Feb 10, 2013
    Messages:
    126
    Likes Received:
    32
    Great work ! ;)

    Only one question, why are you still on GTX970 for vr ?? This graca is now more then 4 years old and really not represenativ for state of the art vr gaming. GTX1080 is getting cheaper and cheaper... and you will get great results in vr with it. I am on a TitanXP and since the last VR Update i can drive with everything full, only shadows little lower, ppd 1.6, level 4 aa, pe high and asw works great.
     
  16. MarcG

    MarcG Registered

    Joined:
    Oct 4, 2010
    Messages:
    6,854
    Likes Received:
    2,234
    Because I can't afford an upgrade regardless of how "cheap" a 1080 may be, my GTX970 works perfectly fine in VR for my needs so there's no rush for me to upgrade yet anyway.
    I always want to upgrade my PC (!) but real life bills, rent & food get in the way of that unfortunately :p
     
    vegaguy5555 likes this.
  17. Clemenca

    Clemenca Registered

    Joined:
    Feb 10, 2013
    Messages:
    126
    Likes Received:
    32
    Hmmm... thats bad, but you spent around €600 on a vr set... is this correct ?
     
  18. MarcG

    MarcG Registered

    Joined:
    Oct 4, 2010
    Messages:
    6,854
    Likes Received:
    2,234
    Please don't bring my personal life and reasonings for not being able to upgrade in this thread, a lot has changed in my life since I got VR, thank you :)
     
    vegaguy5555 likes this.
  19. Clemenca

    Clemenca Registered

    Joined:
    Feb 10, 2013
    Messages:
    126
    Likes Received:
    32
    You are welcome ;)
     
  20. MarcG

    MarcG Registered

    Joined:
    Oct 4, 2010
    Messages:
    6,854
    Likes Received:
    2,234
    Post #1 updated with Build 1868941
     
    Clemenca likes this.

Share This Page