START OF EDIT ================================================= PROJECT VIDEO SCANNER IS NOW AVAILABLE! Go to post on page 3 ================================================= END OF EDIT Hi, I'm posting in the Project Cars forum a software I'm developing for some time. I created a thread in Assetto Corsa also and maybe I will do the same on other projects that are popping around. Some of you might already know what I'm doing there since I recognise here some users from PCars. The idea behind that program is to create point clouds from stereo vision movies. For that the only thing needed is two cheap webcams and a laptop. I have many ideas for future developments, but for now I just want to share some movies and pictures of my work: This one is the odometry of the above reconstruction Splash screen for the interface application:
With PS3 eye cameras: With Logitech's c920: The odometry from the last reconstruction in Google maps: Hope to have the application ready till the end of the year. Will post some news. Will try to answer questions, but lot's of work now and this is done in my spare time.
Nice work! Thanks for sharing. Out of curiosity, does your project work with the Kinect? If so, does it work better than other webcams? Also, how does your app track elevation changes? From imagery alone or with a little help from GPS?
Indeed a very interesting project, looks very promising What accuracy is possible with this system ? As this works with visible light, do you get distortion with changing light? For example on a sunny day getting into the shadow or on a rainy day then there is less contrast. Would be a blast to have such kind of tool, keep up the work, I hope you'll get it finished
Only two webcams. I have used two PS3 eye cameras that cost me €9.50 second hand. As the title on the thread says, it's a cheap alternative. Now I'm testing with better cameras (Logitech c920). Here's my testing rig with two PS3 Eye's and two Logitech's C920: The kinect works with infra-red light so it's not usable during daylight, specially for big distances. His it possible to see the images in the forum? They only appear has links? What am I doing wrong? (feeling a bit stupid with this)
Good light his important. Ideal his to have a Gray day with diffuse light. That way we don't get hard shadows and we can use the textures on the reconstructed mesh, projecting our own shadows. In the posted pictures you will find many shadows. Where I live we have many sunny days Only reflections cause big errors in the reconstructions. It can be really precise if we want to. The below image was captured in 1280x720px. We can perfectly see the side walk profile:
For now you need to calibrate the rig. Maybe one day I can implement some kind of auto calibration. The calibration corrects the lens errors and aligns the image of the two cameras. It gives you the correct size of the objects. Almost any camera can be used although high FOV lenses bring some calibration problems and the program is not made to use them. Also the cameras should be in synch. If they are not in sync we get severe reconstruction errors. So the Go Pro his not really a good solution. To use them we have to select a mode with lower FOV and make a movie at low speed because of the synch error (yes the Go Pro 2 3D his not in sync). My recommendation his two web cameras (preferably the same) capable of at least 30fps @ 1280x720px. The cameras I used with success are: Logitech C625 - 1920x1080@30fps Logitech C920 - 1920x1080@30fps PS3 Eye - 640x480@60fps The faster they are the better to get frames in sync. They should have a good separation (I used 30cm and 60cm) so separated cameras are necessary. I even tried a Sony 3D Handycam. It works for close objects, but anything further away is just plain. And the program although you can use any video you have from other sources, has a capture mode for web cams (DirectShow).
FlyPT, I saw some drivers for the PS3 Eye a long time ago that would run 180fps or somewhere around there. I was looking for them recently and couldn't find them anymore. I think it drops the resolution to 320x240 though which would probably be a bigger draw back than the 60fps framerate.
I'm impressed! Is a very interesting project I use point cloud for track creation and only need point cloud as a reference. Good work!
How is orientation and scale handled (i.e. starting off on a hill)? Is there any limiting factor for how long you can record (i.e. disk space, RAM, processor speed, transfer speed of the camera)? This seems to be exactly what I've been looking for the past couple of years. Can't wait to get my hands on it and give it a try as well!
All frames are generated in relation to the first one. The first frame his considered at origin, pointing in the direction of the z axis. If you are in a hill, all the road will be generated in function of the starting point. But I'm implementing some corrections to the drift and roll errors. Each generated frame can introduce some error in odometry generation, and for that reason I will have 3 methods where you can say that x frame as a determined position and direction. Here we can use GPS position to correct the point cloud. A bit hard to explain, I'm preparing a manual that will come with the application explaining all the parameters and rig construction. Has for scale, she's determined the moment you calibrate the cameras. So if you have a 4 meters wide road that's what you have in the reconstruction. Here's a picture of Meshlab, measuring the width of the road (+-2.6 meters on each direction - It's correct, I measured in reality) Note: The red dot's are the car movement for the reconstructed frames.
Decided to share one of the point clouds. If anyone has the curiosity to view the results, just download the ZIP file. Inside there are multiple point clouds. One for odometry points and the other ones are 10 frames reconstruction groups. Just drop them in Meshlab to view the point cloud. That's 19 million points without any treatment. No sub sampling or odometry correction. Reconstruction file (218MB) Any feedback welcome.
The most important for track creation is a good elevation accuracy and what i see in meshlab is very good. Is possible send some file format with Geographic coordinates (like Google Earth decimal degrees)?
