Hi all, It is benchmarking times these days, so I thought it could be interesting to make a thread compiling the typical things that can be easily overseen and forgotten while doing benchmarks that can affect the obtained results ruining the drawn conclusions. If publicating results for others to reproduce make sure you specify them all! It is also important to write them down if you want to use them in the future to make sure that new benchmarks are comparable to older ones. I recommend to directly edit frapslog.txt including them ahead of the benchmark figures. These are coming from my own experience. It has taught me to be very methodical in these type of procedures. 0. For sure I will initially mention that it is clear that similar, both, screen (resolution, AA...) and display (circuit detail, shadows...) settings should be used. The same applies for used vehicle and track. However, this is the part that is usually done correctly unless the one doing the benchmark is a complete moron that doesn't even bother to check this "easy" part. 1. FOV. Always use the same FOV. A bigger FOV will reduce FPS. 2. Unless especifically benchmarking car related fps issues, try to use only one car in track which will be driven by the AI. It is difficult to reproduce similar situations when multiple cars. If using multiple cars, do it in a race session placing your car at the end of the grid so that the number of vehicles in score is anyways the same. Use AI level of 105 or so to make sure your car does not overtake opponents. In any case the results will be less repetitiva compared to single car tests. 3. Do the benchmarking around a complete lap. Start finish line is the logical reference to pick. You can time accelerate until slightly before getting there to save some valuable time in these type of tedious jobs. 4. Disabling plugins is the easiest way to guarantee similarity in here between different installs or machines. 5. Beware of the mirror mode you are using. Checking differences in fps performance with and without mirrors can help detecting problems in non optimized tracks. 6. Check weather to Sunny unless specifically testing for it's effect in performance. Rain for example has a big impact in fps. 7. Check if session times are the same time. Night racing hits fps performance. When alternating between dx9 and dx11 builds, player json is maintained so most of these checks are OK by default. However if using different installs or machines there is no such guarantee. If you think I have missed any relevant variable that should also be taken into account when doing benchmarks, post it and I will add it to the list.
