Logging Out-of-Realtime Physics Freezes [public-test]

Discussion in 'Technical & Support' started by Christopher Elliott, Mar 19, 2019.

  1. Marcel Offermans

    Marcel Offermans Registered

    Joined:
    Oct 4, 2010
    Messages:
    647
    Likes Received:
    2,936
    The logging is useful both in single- and multiplayer. Our focus is on multiplayer as the effects of a physics freeze there are worse than when you drive in singleplayer. In singleplayer, if the client is too slow, time simply stops for a while which is annoying but not nearly as big a problem as in multiplayer, where time only stops for your client, meaning you warp into the path of cars behind you (as server time must always go on for obvious reasons). So please feel free to also submit logs in singleplayer mode.
     
    Mindraiderz likes this.
  2. Marcel Offermans

    Marcel Offermans Registered

    Joined:
    Oct 4, 2010
    Messages:
    647
    Likes Received:
    2,936
    The point you make is a very valid one. Plugins in general can slow down the physics thread as they are executed on the same thread (in-line). This is exactly why we are now timing each individual plugin. For all developers out there, the plugin documentation has extensive warnings about things you cannot do in the callbacks: nothing that might block or do I/O can be done on this callback. If you need something like that to happen, use a separate background thread and some kind of non-blocking message queue to do the actual work.

    So the example you give here @DaVeX is indeed a disaster waiting to happen: loading a file directly from plugin code without such a mechanism in place. The good news is that from our timing in the performance log, we can now easily identify such issues and will report back our findings.
     
    Atsuki, The Iron Wolf and Krevett62 like this.
  3. Marcel Offermans

    Marcel Offermans Registered

    Joined:
    Oct 4, 2010
    Messages:
    647
    Likes Received:
    2,936
    We ship with 4 plugins, and I'll explain what each of them does:
    1. TrackIR_rF2_Plugin - is used for the TrackIR device and does head-tracking, it can safely be disabled if you don't use such a device.
    2. KnockoutQualifying - is a "rules" plugin that simulates multiple "knockout" style qualifying sessions and can safely be disabled unless you specifically want to use this format.
    3. StockCarRules - is a "rules" plugin that contains specific rules for stock car racing and can safely be disabled unless you're doing Stock Car races.
    4. SteamPlugin - interacts with Steam, keeps track of achievements, statistics, friends, dedicated server features and a few more things. I would not recommend disabling it in general and it will certainly prevent you from joining our competitions if you do, but strictly spoken, you can disable it and rFactor 2 will still run.
    So, to summarize, please leave the SteamPlugin running and the others you can disable for testing. That said, as we're now also timing specific plugins, feel free to, as a test, leave others enabled and send us the data.
     
    Ernie, Adam Helyes and The Iron Wolf like this.
  4. Viktor Jim

    Viktor Jim Registered

    Joined:
    Mar 21, 2019
    Messages:
    19
    Likes Received:
    0
  5. 4thworld

    4thworld

    Joined:
    Dec 17, 2016
    Messages:
    286
    Likes Received:
    950
    @Viktor Jim , thanks could you keep Realtime Logging enabled inside your player.json?
    In that way we can see where to act if you're experiencing some issues.
    thanks.
     
  6. Mario Bayer

    Mario Bayer Registered

    Joined:
    Apr 22, 2019
    Messages:
    4
    Likes Received:
    0
    Hello Racer,
    I also have the problem in online races that the screen freezes. However, it freezes completely for me. The picture stops and I have no sound anymore. The PC will then stop accepting commands. It only helps to restart via the reset button. Is this the same problem or is the cause different for me?

    I only drive on P1 servers and I've never had any problems in practice. Only in the long distance race I have this freeze. And since only once usually within the first hour, then it is quiet.

    I am very grateful for your help.
     
  7. Marcel Offermans

    Marcel Offermans Registered

    Joined:
    Oct 4, 2010
    Messages:
    647
    Likes Received:
    2,936
    @Mario Bayer the freezes we're looking at here are those when the computer briefly freezes up and then continues again. The performance logs are only saved when the session ends. Your problem seems to be different, if the only thing that helps is a reboot of your computer. We would still like to get a (separate) report of that with as much detail about your system and configuration as possible so we can try to reproduce it. As a first guess, are you sure you're not somehow running out of memory or diskspace or anything like that? Also make sure your computer is not overheating. Could be something totally different though.
     
  8. Viktor Jim

    Viktor Jim Registered

    Joined:
    Mar 21, 2019
    Messages:
    19
    Likes Received:
    0
    Ok, today i will do another test with that option on
     
  9. Brent

    Brent Registered

    Joined:
    Nov 5, 2015
    Messages:
    236
    Likes Received:
    111
    I'm getting really bad constant random CPU spikes in multiplayer and single player that are causing stuttering, so bad it's undriveable. I did not have this issue Friday afternoon. No matter what car or track I'm having the issue. I'm watching the magenta cpu status bar max out, it's just the bottom edge of the bar and it spikes and freezes, randomly. I'm not having any issue with any other games on my PC. I7 8700K not overclocked, AMD Radeon R9 390 not overclocked, 16GB DDR4-2400 RAM, 500GB SSD M.2 NVMe with 120GB free, single monitor at 1920x1080 and tried a lower resolution as well. Tried verifying integrity, dropped back to V1111 and back to V1112 and both stutter. Tried several cars and tracks. I completely reinstalled RF2, deleted all RF2 files from the folders after uninstalling for a total clean reinstall. Still having the same issue...

    Log and trace files of a lap around Silverstone in the C6R with the latest non beta V1.1112:
    https://www.dropbox.com/s/fpnpzs22v515qew/2019-04-23 RF2 Trace and RTL files.zip?dl=0

    Stream of the above lap with CPU GPU and FPS data on:


    I switched to the latest public beta build update ID [3755442] and I'm getting the same constant freezing, several exiting pit road. The only thing I notice is the magenta CPU bar is now fully visible and appears to be maxed out, when the bottom line of it pulses from left to right it stutters.

    Here's log and trace files of a partial lap at Silverstone with the same C6R mod with beta build 3755442:
    https://www.dropbox.com/s/d1ya3zsw85x8722/2019-04-23 RF2 Trace and RTL files ID3755442.zip?dl=0
     
    Last edited: Apr 23, 2019
  10. 4thworld

    4thworld

    Joined:
    Dec 17, 2016
    Messages:
    286
    Likes Received:
    950
    @Brent, thanks this is helping us a lot.

    It seems that your issue is in the FFB update can you answer the following questions:
    1) what wheel do you have?
    2) are your wheel firmware up to date?
    3) I am going to release a new build with some extra logging for you later, would you be able to try it, please?
    4) do you have any 3rd party plugins?
    5) do you have anything slightly unordinary on your setup like multiple wheels/gamepads attached?

    Thanks a lot
     
    Last edited: Apr 23, 2019
    Rui Santos likes this.
  11. Brent

    Brent Registered

    Joined:
    Nov 5, 2015
    Messages:
    236
    Likes Received:
    111
    1) Logitech G27
    2) I'll verify that later today, the last update was 2016 so I'm fairly certain it is the latest. I'll confirm
    3) Yes
    4) I have been running the shared memory plugin in the past for CrewChief. With this issue I turned it off, restarted and still had the issue. With the clean slate install I don't believe it's being used, I'll confirm.
    5) I had a USB space mouse connected and have had a Microsoft Xbox gamepad controller connected in the past. I made sure both were unplugged for the beta build log above.

    You're welcome!
     
  12. 4thworld

    4thworld

    Joined:
    Dec 17, 2016
    Messages:
    286
    Likes Received:
    950
    Build update ID [3761808]:
    - Additional logging information on FFB thread @Brent

    branch name: public-test
    password : UpERYPhIblEHorTiO
    verify your files via Steam after joining the branch

    1. Select rFactor 2 from steam game library, right click and select properties.
    A dialog box with a number of tabs will appear, your looking for one that says "BETAS".
    2. select the "BETAS" tab it by clicking on it.
    You will see 2 things now, a drop down menu and a text box to enter a beta access code.
    3. enter the password provided,
    4. select public-beta from the dropdown menu
     
    Rui Santos and The Iron Wolf like this.
  13. Mario Bayer

    Mario Bayer Registered

    Joined:
    Apr 22, 2019
    Messages:
    4
    Likes Received:
    0
    @Marcel Offermans
    Here are my system data:

    Windows 10 PRO 64 bit
    Intel Core i5 9600K 3.7 @ GHz
    RAM: 16 GB
    Nvidia GeForce GTX 1070ti
    1GB SSD

    The PC is new and was bought earlier this year or built by a friend.

    Of a heat problem, I tend not go out because I can play ACC, Raceroom without a problem. Also for a long time.
    I have similar problems with F1 2018. After about 10 minutes, the freeze only comes with the difference that then a blue screen but the PC does not start by itself.

    What data, etc. do you need to help me? I'm only on PC this year (previously Playstation) of so I'm really grateful if someone helps me, because I'm not so familiar with where I can find certain parameters / troubleshooting.

    I would be very happy if I can play rFactor2 without any problems, because I have already registered for a P1 event again, but it is absolutely impossible to have fun with these freezes.

    Best regards
     
  14. Marcel Offermans

    Marcel Offermans Registered

    Joined:
    Oct 4, 2010
    Messages:
    647
    Likes Received:
    2,936
    The specifications of your computer are absolutely fine. One thing you could try is to enable trace logging and send me the log file. Do you know how that works?
    Another thing to look for is the Windows event log. Sometimes it contains information on why your computer froze. Again, do you know where to find that?
     
  15. Stefan_L_01

    Stefan_L_01 Registered

    Joined:
    Nov 18, 2012
    Messages:
    595
    Likes Received:
    386
    I tried to reproduce today in Donington, but it was running smooth with replay enabled. However now there are Flowers on the gras. Both helis run smooth.
    Then I increased some Level to med, had one Freeze Right joining track starting out of the stall. But then it was smooth. Did not had realtime log enabled.
    Restartet with realtime log enabled and joined track, immediatly things went horribly wrong. Wheel shaking left Right and so on. Aborted.

    Remembers me to my colleages in buissness doing machine Control software, whenever they enable high log Level real time goes out of Bounds. Natual Problem that extended logging is producing more problemes than resolving if not done the proper way :). My Magenta bar, usually < 30% is now on ex.
    Sry, can not deliver usable data this way
    BR
    Stefan
     
  16. Brent

    Brent Registered

    Joined:
    Nov 5, 2015
    Messages:
    236
    Likes Received:
    111
    I can confirm a couple of things.

    2) I was using an older logitech driver Ver 5.08.146 date 9/16/2009. I uninstalled it, shut down, restarted and installed the latest Ver 5.10.127 3/14/2016
    4) The track IR plugin along with the steam plugin were on with the clean slate install. I turned off the track IR since I don't use it. Only the steam plugin is on now.

    I used the passcode for the public beta 3761808, confirmed I'm using that buildID and then verified the files via Steam after joining the branch. I'm still getting stuttering about the same as before.

    Log and trace files of a lap around Silverstone in the C6R with Build ID 3761808:
    https://www.dropbox.com/s/2jfwcvgj4bbisq8/2019-04-24 RF2 Trace and RTL files ID3761808.zip?dl=0

    Stream of the above lap with CPU GPU and FPS data on and replay, halfway through the lap I expanded full screen to make the replay position visible:
     
    The Iron Wolf likes this.
  17. Ari Antero

    Ari Antero Registered

    Joined:
    Jul 27, 2012
    Messages:
    1,882
    Likes Received:
    829
    @Brent You have too high CPU usage. There is something else than rF2 which is bottlenecking your CPU.

    CPU BN.jpg
     
  18. Brent

    Brent Registered

    Joined:
    Nov 5, 2015
    Messages:
    236
    Likes Received:
    111
    My stream from the other day running the non beta V1.1112 build has the CPU usage about 20-25% with spikes. With the public beta builds I notice the bar appears to stay full, not sure why that's the case, might be something to do with the beta build, but I can see the magenta line drop on the right hand side when it stutters.
     
  19. Ari Antero

    Ari Antero Registered

    Joined:
    Jul 27, 2012
    Messages:
    1,882
    Likes Received:
    829
    I think that magenta line is CPU usage to all CPU cores and when its all the way up you have 100% CPU load to all cores which I don`t understand is possible at all.
    If you look pic below CPU#0 and CPU#2 are the physical cores rF2 is using and I am wondering how your CPU is loading all Cores when rF2 is using only two cores and few threads.
    CPU Bret.jpg
     
  20. Bernd

    Bernd Registered

    Joined:
    Jul 9, 2013
    Messages:
    1,509
    Likes Received:
    755
    @Marcel Offermans and @4thworld and all other S397 member that are involved in this.
    I just want to thank you, that you continously put so much effort in trying to solve the freeze and stuttering problems, that the users have.
    I guess i can talk for all of us, when i say that this is very much appreciated, because it is not self-evident nowadays.
     

Share This Page