Cheap alternative to laser scanning

Discussion in 'Track Modding' started by FlyPT, Dec 3, 2013.

  1. FlyPT

    FlyPT Registered

    Joined:
    Aug 3, 2012
    Messages:
    27
    Likes Received:
    6
    START OF EDIT
    =================================================

    PROJECT VIDEO SCANNER IS NOW AVAILABLE!
    Go to post on page 3

    =================================================
    END OF EDIT


    Hi,

    I'm posting in the Project Cars forum a software I'm developing for some time.
    I created a thread in Assetto Corsa also and maybe I will do the same on other projects that are popping around.
    Some of you might already know what I'm doing there since I recognise here some users from PCars.
    The idea behind that program is to create point clouds from stereo vision movies.
    For that the only thing needed is two cheap webcams and a laptop.
    I have many ideas for future developments, but for now I just want to share some movies and pictures of my work:





    [​IMG]

    [​IMG]

    This one is the odometry of the above reconstruction
    [​IMG]

    [​IMG]

    [​IMG]

    Splash screen for the interface application:
    [​IMG]
     
    Last edited by a moderator: Mar 28, 2014
  2. FlyPT

    FlyPT Registered

    Joined:
    Aug 3, 2012
    Messages:
    27
    Likes Received:
    6
    With PS3 eye cameras:
    [​IMG]

    [​IMG]

    With Logitech's c920:
    [​IMG]

    [​IMG]

    [​IMG]

    [​IMG]

    [​IMG]

    The odometry from the last reconstruction in Google maps:
    [​IMG]

    [​IMG]

    Hope to have the application ready till the end of the year.
    Will post some news.
    Will try to answer questions, but lot's of work now and this is done in my spare time.
     
  3. Frankysco

    Frankysco Registered

    Joined:
    Aug 7, 2013
    Messages:
    134
    Likes Received:
    0
  4. Noel Hibbard

    Noel Hibbard Registered

    Joined:
    Oct 5, 2010
    Messages:
    2,744
    Likes Received:
    28
    Nice work! Thanks for sharing. Out of curiosity, does your project work with the Kinect? If so, does it work better than other webcams?

    Also, how does your app track elevation changes? From imagery alone or with a little help from GPS?
     
  5. A13

    A13 Registered

    Joined:
    Jul 31, 2012
    Messages:
    123
    Likes Received:
    29
    Indeed a very interesting project, looks very promising :)

    What accuracy is possible with this system ? As this works with visible light, do you get distortion with changing light? For example on a sunny day getting into the shadow or on a rainy day then there is less contrast.

    Would be a blast to have such kind of tool, keep up the work, I hope you'll get it finished :)
     
  6. FlyPT

    FlyPT Registered

    Joined:
    Aug 3, 2012
    Messages:
    27
    Likes Received:
    6
    Only two webcams.
    I have used two PS3 eye cameras that cost me €9.50 second hand.
    As the title on the thread says, it's a cheap alternative.
    Now I'm testing with better cameras (Logitech c920).

    Here's my testing rig with two PS3 Eye's and two Logitech's C920:
    [​IMG]
    [​IMG]
    [​IMG]

    The kinect works with infra-red light so it's not usable during daylight, specially for big distances.

    His it possible to see the images in the forum? They only appear has links? What am I doing wrong? (feeling a bit stupid with this)
     
  7. FlyPT

    FlyPT Registered

    Joined:
    Aug 3, 2012
    Messages:
    27
    Likes Received:
    6
    Good light his important.
    Ideal his to have a Gray day with diffuse light. That way we don't get hard shadows and we can use the textures on the reconstructed mesh, projecting our own shadows.
    In the posted pictures you will find many shadows. Where I live we have many sunny days :D
    Only reflections cause big errors in the reconstructions.

    It can be really precise if we want to. The below image was captured in 1280x720px. We can perfectly see the side walk profile:
    [​IMG]
     
  8. blakboks

    blakboks Registered

    Joined:
    Oct 20, 2010
    Messages:
    843
    Likes Received:
    29
    Will this work with pre-recorded video (as opposed to webcams), I.e. go-pro's?
     
  9. FlyPT

    FlyPT Registered

    Joined:
    Aug 3, 2012
    Messages:
    27
    Likes Received:
    6
    For now you need to calibrate the rig.
    Maybe one day I can implement some kind of auto calibration.
    The calibration corrects the lens errors and aligns the image of the two cameras. It gives you the correct size of the objects.
    Almost any camera can be used although high FOV lenses bring some calibration problems and the program is not made to use them.
    Also the cameras should be in synch. If they are not in sync we get severe reconstruction errors.
    So the Go Pro his not really a good solution. To use them we have to select a mode with lower FOV and make a movie at low speed because of the synch error (yes the Go Pro 2 3D his not in sync).
    My recommendation his two web cameras (preferably the same) capable of at least 30fps @ 1280x720px.

    The cameras I used with success are:
    Logitech C625 - 1920x1080@30fps
    Logitech C920 - 1920x1080@30fps
    PS3 Eye - 640x480@60fps

    The faster they are the better to get frames in sync.
    They should have a good separation (I used 30cm and 60cm) so separated cameras are necessary.
    I even tried a Sony 3D Handycam. It works for close objects, but anything further away is just plain.
    And the program although you can use any video you have from other sources, has a capture mode for web cams (DirectShow).
     
  10. Rony1984

    Rony1984 Registered

    Joined:
    May 10, 2012
    Messages:
    485
    Likes Received:
    22
    I'm not a modder, but this looks damn interesting. Thanks for posting!
     
  11. Saro

    Saro Registered

    Joined:
    Nov 19, 2012
    Messages:
    31
    Likes Received:
    2
    Very nice, that's exciting stuff :)
     
  12. Noel Hibbard

    Noel Hibbard Registered

    Joined:
    Oct 5, 2010
    Messages:
    2,744
    Likes Received:
    28
    FlyPT, I saw some drivers for the PS3 Eye a long time ago that would run 180fps or somewhere around there. I was looking for them recently and couldn't find them anymore. I think it drops the resolution to 320x240 though which would probably be a bigger draw back than the 60fps framerate.
     
  13. Mario Morais

    Mario Morais Member

    Joined:
    Oct 26, 2010
    Messages:
    1,464
    Likes Received:
    168
    I'm impressed! Is a very interesting project

    I use point cloud for track creation and only need point cloud as a reference.


    Good work!
     
  14. FlyPT

    FlyPT Registered

    Joined:
    Aug 3, 2012
    Messages:
    27
    Likes Received:
    6
    Yep, it's the CLDriver
     
  15. blakboks

    blakboks Registered

    Joined:
    Oct 20, 2010
    Messages:
    843
    Likes Received:
    29
    How is orientation and scale handled (i.e. starting off on a hill)? Is there any limiting factor for how long you can record (i.e. disk space, RAM, processor speed, transfer speed of the camera)?

    This seems to be exactly what I've been looking for the past couple of years. Can't wait to get my hands on it and give it a try as well!
     
  16. FlyPT

    FlyPT Registered

    Joined:
    Aug 3, 2012
    Messages:
    27
    Likes Received:
    6
    All frames are generated in relation to the first one.
    The first frame his considered at origin, pointing in the direction of the z axis. If you are in a hill, all the road will be generated in function of the starting point.
    But I'm implementing some corrections to the drift and roll errors.
    Each generated frame can introduce some error in odometry generation, and for that reason I will have 3 methods where you can say that x frame as a determined position and direction. Here we can use GPS position to correct the point cloud.
    A bit hard to explain, I'm preparing a manual that will come with the application explaining all the parameters and rig construction.
    Has for scale, she's determined the moment you calibrate the cameras. So if you have a 4 meters wide road that's what you have in the reconstruction.
    Here's a picture of Meshlab, measuring the width of the road (+-2.6 meters on each direction - It's correct, I measured in reality)
    [​IMG]

    Note: The red dot's are the car movement for the reconstructed frames.
     
  17. alpha-bravo

    alpha-bravo Registered

    Joined:
    Dec 22, 2012
    Messages:
    1,379
    Likes Received:
    31
    That's a really cool solution :cool:
     
  18. FlyPT

    FlyPT Registered

    Joined:
    Aug 3, 2012
    Messages:
    27
    Likes Received:
    6
    Decided to share one of the point clouds.
    If anyone has the curiosity to view the results, just download the ZIP file.
    Inside there are multiple point clouds. One for odometry points and the other ones are 10 frames reconstruction groups.
    Just drop them in Meshlab to view the point cloud.
    That's 19 million points without any treatment. No sub sampling or odometry correction.
    Reconstruction file (218MB)
    Any feedback welcome.
     
  19. Mario Morais

    Mario Morais Member

    Joined:
    Oct 26, 2010
    Messages:
    1,464
    Likes Received:
    168
    The most important for track creation is a good elevation accuracy and what i see in meshlab is very good.

    Is possible send some file format with Geographic coordinates (like Google Earth decimal degrees)?
     
  20. DJCruicky

    DJCruicky Registered

    Joined:
    Jan 25, 2012
    Messages:
    1,169
    Likes Received:
    238
    That looks very clever.
     

Share This Page