considering upgrade to SSD at first, then RAM
Both are very much worth it. The Nurburgring is the most demanding track for me in terms of CPU and memory utilisation (system and GPU).
As others have said - your GPU memory and system memory are both at the low end.
I recommend at least 16 GB system memory and 6 GB GPU memory for 1920x1080p resolution.
Off the top of my head, I'm pretty sure I use about 12 GB of system memory and almost 6 GB, but not quite of GPU memory in rF 2 at this track.
Verify resource utilisation (CPU, system memory, GPU and GPU memory) using a tool like RivaTuner's on screen display, which integrates with MSI Afterburner (it supports other products too I believe). I repeat - verify your resource utilisation rather than just guessing.
If you exceed your GPU memory - i.e. it gets fully used, then the game will end up having to shuffle data around, between system memory, gpu memory and reading from disk as needed - which is an obvious performance overhead - i.e. it will reduce your performance.
Running the game from SSD will speed up loading times nicely. In my case I have other games on my SSD too and I have a lot of content in rFactor 2, so my compromise is to relocate my tracks folder to the SSD - as that's a much larger chunk of data to be reading and I leave the vehicles folder on my old hard drive. That works for me and it's as easy as creating a junction point, or simlink (both names refer to the same thing). You can use the MKLINK command from an administrator command prompt to create them, using the syntax "MKLINK /J "c:\games\source game folder" "d:\games\destination folder". The source folder will be created to point to the target - so for a steam game, close steam, move the source folder to another location, create the junction point and then copy the data to the target. I leave a copy of the source data on the original disk, so I can revert the change quickly if I feel like it.