The proper testing of server ahead of builds being released

Discussion in 'General Discussion' started by hardtohandle, Feb 26, 2015.

  1. hardtohandle

    hardtohandle Registered

    Joined:
    Jul 15, 2013
    Messages:
    364
    Likes Received:
    54
    I have a love/hate relationship with RF, I think a lot of us do. It's an awesome sim, and it has a lot of problems - all too often serious problems.

    There is more than one area I guess where things need to improve, but there is one area that we here in the RF2 community can perhaps collectively help with - stress testing the server BEFORE it gets released.

    Time and time again we get builds that just DO NOT allow you to stage a proper race with no serious problems. I been on RF2 for 2 years now and this happened many times, and it is still happening today.

    Here is the thing. I am guessing here, but I reckon ISI simply DO NOT test these builds with a full server of real drivers before release in anyway whatsoever. And guess what? Half of these server problems simply do not kick in when you have only 8 drivers in the server - they only raise their ugly heads when you have 28 drivers in the server. It's no good checking it out with only a handful of drivers and ticking it off as tested! Because soon as the server is full it falls to pieces - so therefore our races do too.

    Personally I reckon one thing single thing that could be done to improve this situation so much, and to improve the kind of game experience players have when they join a league is simply to PROPERLY stress test the critical server components BEFORE release with a FULL server of 30 drivers. If this could happen, we could go into a weekend of various league races with admins feeling pretty confident the server is at least going to work properly, and drivers can feel confident they will at least be able to get into the server, start the race without being disconnected, and finish the race without playing russian roulette with disappearing cars etc etc. Wouldn't that be wonderful?

    It is bad enough that these things happen to our races, but the really damaging thing is that this way that faulty build is going to stay out there in the wild then for 6 weeks generally instead of being nipped in the bud during testing. We get several bad builds a year so therefore several months of the year RF is crippled with serious faults - leagues have to struggle through a significant chunk of their season with a faulty build, the all important new players coming in are soon turned away when they have to struggle like hell just to keep the sim running properly only to find when they do eventually join a league the races fall apart anyway because of serious faults in the server.

    MY SUGGESTION

    Is that all of us here, and especially league admins who can appeal to their drivers, get together 30 drivers purely for a test race to test the build with a full server. If it works when it is full, then the build is good. Trying it with 6 tests nothing. If 6 leagues can get 5 drivers each to show up to some ad-hoc test race, that will do it. Also we can use the ISI forum here to attract people in too.

    The goal is simply to test the server when full, with some mod or other and track. Nobody needs to practice or sort a set, it don't matter if everybody is crashing and going in the wall, all that matters is that 30 people get in that server and drive on track simultaneously. We will see then if it works, or if it does not.

    At present I suspect the leagues that have big grids are the first guys to run these builds with full servers, then it is too late if there are serious problems and we a all stuck with a lemon build for 6 weeks which is a double whammy and hurts the game badly as a whole.

    Of course ISI will need to provide an unstable build as they did in the past. A good idea suggested to me last night could be for ISI to put out unstable builds on a Friday at close of business US time, so we can test that Friday night and the whole weekend, and if it works properly it can be released the following Friday and we can all look forward to 6 weeks of simracing on a solid build, new players have double the chance of staying. Crucial thing is simply to get 30 guys in a server for 1 hour and see if it works before release.

    What do you think?
     
    Last edited by a moderator: Feb 26, 2015
  2. Noel Hibbard

    Noel Hibbard Registered

    Joined:
    Oct 5, 2010
    Messages:
    2,744
    Likes Received:
    40
    I take it you haven't seen the dedicated servers for the test team on the MM?

    Code:
    Server Name         build Mod               Track    Players Max
    -----------         ----- ---               -----    ------- ---
    ISI hoover kvm        934 vmod20150225v1    Malaysia       0  25
    ISI Test Team 934 1   934 Eurotest022620152 Malaysia       1  30
    LCD-918-TEST          934 mpVmod_62028_0    Malaysia       0 103
    
    Take a look at the MM at 10PM (EST) on Wednesday nights, 8PM (GMT) on Thursday nights or 3PM (GMT) on Saturdays.

    I think a wider test would be helpful though. The unstable testing seemed helpful but that's been dropped now for some reason.
    http://rfactor.net/web/rf2/devscorner/unstable-build-testing/
     
  3. hardtohandle

    hardtohandle Registered

    Joined:
    Jul 15, 2013
    Messages:
    364
    Likes Received:
    54
    Sure I have seen those servers, and I see the testing has got better since the 'join button gate' which is of course welcome. I have never seen more than 8 people in those testing servers though, perhaps they get 30 in when I am in bed for all I know. In any event I stand by my post and agree with you that even so, further wider testing can only be a good thing.
     
  4. stonec

    stonec Registered

    Joined:
    Jun 19, 2012
    Messages:
    3,399
    Likes Received:
    1,488
    I don't think there is an easy way to get full test grids. The internal test group is not for everyone, due to obvious reasons. The unstable public builds were not generating much feedback. Reading the unstable forum you can see it's pretty dead, none cares and those who care are already internal testers.
     
  5. Valter Cardoso

    Valter Cardoso Registered

    Joined:
    Nov 4, 2011
    Messages:
    601
    Likes Received:
    1
    Understand your point but what are those problems you refer? We have being using rf2 for some time now (since middle 2012) and apart that greyed button and b910 classes issue we dont have any major question with server.

    Maybe those issues you refer are not rf2 related but 3rd party mods/tracks?

    Anyway, if you need we can group 20 or 30 guys to appear at some scheduled test.
     
  6. jjcook

    jjcook Registered

    Joined:
    Apr 12, 2012
    Messages:
    345
    Likes Received:
    69
    legends league - F1 92(ver 1.7) at Silverstone - large grid 28-30. Many people experienced game crashes - and cars were blinking on and off on the server with no measurable connection or server issues. That mod works with large grids prior to the most recent build -this was not the series first large grid but , the first one that went badly -kinda smells like a build problem crept in -

    count me in HTH - just a matter of advertising it i believe -
     
  7. ErnieDaOage

    ErnieDaOage Registered

    Joined:
    Feb 26, 2013
    Messages:
    180
    Likes Received:
    0
    We did some public races in the skippy 12 drivers and in 20minutes of race the whole field disappears 2 to 4 times without a connection loss and its for everybody the same all cars just drop out and drop back in.
     

Share This Page