AI tools for 3D modeling

[mantasisg]

This is my first try with meshy.ai, from a single image done in few minutes.

Source image:

Very impressive. The model by itself isn't very usable. Much too much geometry, and the quality is obviously not great lots of offs in fine details, however general shaping seems to be there. I am sure it would get few times better with few more source images plugged in. This is is still satisfying as a reference 3D model, that would cut modeling time by 4-6 days or more.

If it was 2016 now, modders would go crazy haha

 

[ThomasJohansen]

I tried it too the other day.
Even if you feed it with multiple pictures from different angles it somehow guesses wrong on some parts it cant see in on picture but in another, like if you also added at rearview picture it still would make it wrong.

I have not used a paid AI model generator, so maybe they can do better for money.

 

[mantasisg]

Not impressed ? I am very impressed anyway. And in some way it is best if it can't make it perfect. This means artist work is still going to be necessary. But it will just be quicker.

I would say these models are perfectly good already for background objects, just poly count needs to be reduced a lot.

For highly realistic models this seems to be similar to LiDAR data, which you can not always use directly (sometimes you can), but it is perfect as guiding, reference data.

I know that there are many 3D AI tools, and I will be studying them to learn about their strengths, what are best best choices. For now I am gambling with meshy.ai, just made my payment, and I hope it to pay off shortly.

There is some learning. As there is some randomness you probably have to make multiple tries. But so far I have found better result by using a single image. for that particular end. So maybe it would be best to use separate AI generated models, one for front, other for rear.

Here is how front end looked in first try by using single image, it is quite accurate:

After it I gambled and paid money, used multiple views, main image same as before, it is less accurate particularly at the very front corners, I guess it is harder to get right for AI as it is for humans too, however human would usually get it better by having more reference view angles:

This is how rear end looked when I used multi-mage function:

Rear end that AI generated based on single image:

There is a lot of inaccuracy at rear, more than at front for some reason, but it would still be major help while starting the modeling of this car. For some reason wheels were most accurate in the very first one.

And this is original source picture of rear end:

It would be interesting to try pictures strictly of front/side/rear.

 

[Emery]

I would say these models are perfectly good already for background objects, just poly count needs to be reduced a lot.

For highly realistic models this seems to be similar to LiDAR data, which you can not always use directly (sometimes you can), but it is perfect as guiding, reference data.

Pretty much agree. I had tried the google AI that just uses words and, as I recall, it has a setting for number of polygons. Problem there is that if it hasn't been trained on the brand of car (or other object), then you get a random image.

Wonder why the AI inset the fuel filler?

 

[mantasisg]

Good news AI is there and it works great, it is a great tool, and I think we can expect it to get a lot better during next five years till naturally improvements will slow down.

I am not working on car at the moment, to be honest. I am working on a track. AI is proving to be very helpful so far, in some cases the results are amazing, in other cases they are less amazing, but in most cases pretty good. There is little bit of skill to use AI properly too, but it is not rocket science, however there are probably tricks to do to boost results a lot that I am not aware of.

Check out how good is this simple object:

Obviously having proper source image is big deal. Often single image seems to give better results, probably because I don't have proper side view images most of the time.

It is not always practical to use AI tools instead of just modeling an object properly yourself, but there are definitely times it is the way to go. I haven't tried it, but I guess it would be interesting to try model something, but then have AI to do UV and texturing, which can be done with very good results.

I am currently learning to achieve good results that would hold less polygones and would be in general cleaner. There are little bit of inconvenience where separate objects are needed that would usually benefit of using same material, therefore same textures. It can be done manually of course...

I wish in future those AI tools would improve a lot on topology, polygones economy, and ability to batch different objects into a single material would also be very good.

I am using so far Meshy.ai and Hunyuang. Sometimes one is better than another, but both are very good.

Back to cars for a second... I am trying Chaparral 2F, and still my best results comes with using single isometric pictures. And again much better result when using front.

separate for rear:

These would definitely be useful at chasing the true shapes and positions of features of the car body.

 

[mantasisg]

Yo.... check this out.

I wrote prompt that I didn't even expect to be good enough, used reference picture which I didn't expect to be good enough:

Nano Banana made these images:

Nano Banana also added few more sets of similar wheels, which was big surprise to me.

Then AI turned it into model, check it out, this is Fipestonne tire, rim does look ok, but everything would have to be remodeled for end result, this could be used as a guide object:

Meanwhile, life is hard.

 

[ThomasJohansen]

What a great way to make quick static track objects, with textures.
Then just add optimize modifier to lower the polycount.

 

[mantasisg]

I like your enthusiasm.

Spoiler: not important

Optimizing with modifiers usually doesn't work. That particular gas station sign object is 1200tris straight out from AI. I believe I could have used remesh AI tool to at least get it reduced by half. In Blender I could use "decimate" modifier to reduce geometry by 10x, it still looks ok, but it immediately looses nice details.

Very good technique is to remodel object around AI object, that by itself can be done in few different ways, then AI object texture can be baked on properly UV mapped mesh with neat topology.

Eitherway. I can see immediately how modeling almost anything could be done twice or thrice faster now. It is not going to be easier or automatic, unless you will be satisfied with slightly warped and messy AI creations. AI adds up completely new layer and lots of new extremely powerful tools to get results that also almost certainly will be better too, if person modeling will be aware to catch AI mistakes.

I can not wait to master AI in my workflow, and hopefully create lots of good stuff significantly faster than I would have done it before. Also looking forward how much more better AI 3D modeling can get.

I wonder how AC and BeamNG modding communities discuss this matter, I guess they should be super enthusiastic, and probably started getting deep into it a year or so ago, I am not following neither of these.

Which modding community currently is most alive ? Or is everything so devastatingly spoiled right now ?

 

[mantasisg]

Not gonna lie, making these in five minutes is fun. Lets be honest they are just AI/photogrammetry pointcloud meshes. I wish I would have time right now to make one of these cars with help of AI just to see how fast it can be done and how good the results can be. You can't really pull these generated objects and use them straight. But they at least should cut the time needed to create by half, at minimum. Maximum maybe four times faster to make a complete car.

Why it is useful ? More time to do proper polishing, implementation, physics, sound work and testing. Cutting a lot of time of creating a scratch model makes every legit creator more competitive against rippers who just ports stuff. Although there is no need to compete with them, it is just nice to know that now they has less leverage in terms of their "production", so maybe modding can get less corrupted.

What is it taking away ? The modeling of these general details, finding them, sorting out the shapes and positions is probably most enjoyable part when you manage to get it right (if you do), so now you'd be sent more straight to "work" part. Although you would also be set on right track quicker and more certain, certainty by itself adds lots of pace to anything people does, if you haven't noticed yet. Some people may also feel like they are sharing credit with AI, and that might be true, but so would they share it with laserscan point clouds if they had them.

What I don't agree with is "AI is going to replace us". Maybe there will be instances it will. But on most part it will just speed up initial phases of many actions, which will allow artist to bring end result way higher than it could have gotten before. Usually it will be mostly like just having laserscans, I haven't heard before that laserscans replaced track artists, they just boosted quality. I suppose I could see how AI might mean reduction of employees in larger studios, because same work can be done faster, but other studios might just keep the employees and opt to get higher results. The question is what users will reward.

Here is one special 1000hp BMW

What I find is that right now single images works better than multiview generations. But smart think to do seems to do front and rear separately. In this case front vew front seems fine, bu the rest of the car is quite inaccurate ever past A pillars. Model genrated from the rear seems to be more accurate up till front. Putting both together would speed up base framework greatly. Just start modeling around this and adjust at the end where AI likely has been off.

Having a good reference image can be useful to create various objects of the car. This AI wheel model would make a superb reference in modeling usable clean "low poly" steering wheel. Possibly could even be used for texture bake. Could bake texture in 3D software. Or could try modeling the fine model, then give it to AI to UV map it and bake texture from this same reference image, then clean it up where it needs.

You can do trackside cars super easily, can pretty much use this straight in the track. Might need some cleaning and optimizing.

Rare cars are so much more tempting now

The best part to me is that you still have to be capable artist to use these. Those that just rips stuff couldn't and wouldn't even have will to use any of these.

 

[philrob]

Lovely work and great choice of cars.

 

[redapg]

@mantasisg Just out of Interest. How many Polys has the above shown Steering Wheel for Example?
I guess it's AI builded, right?

My Doubts about AI-"Modelling" is, that the Polycount is much too high for Video Games and as far as i know, there is no Poly-Reducing Tool available, that Results in satisfying Quality 3D-Models, that can be used as "HD" Models for the max LOD in Games.
E.g. the Resizement Tool of 3DSMax is good to build Lod Meshes, but not to reduce a HD Model with an insane Poly-Count, to a "Game-Ready" Model.

 

[mantasisg]

@redapg yes they are not usable, I find that AI generated objects are rarely usable straight up. Some may be, but they too would be better cleaned up. There is still solid place for AI in the workflow, and I am sure it still gonna get quite a bit better till naturally AI engineers will reach point when they won't be able to do major improvements, as I believe it must get exponentially more difficult as it goes further.

But as reference objects they are very usable, and AI might still come in handy for UV mapping and texturing. I think such steering wheel should be about 3000-6000 polys. Maybe these days it could be 8000-9000 polys. I haven't followed latest standards, how many materials and polygones does latest cars in video games have, I assume it depends on graphical engines.

That steering wheel is 50 000 tris or polys. But it is simply straight up generated. Hunyuang has retopology function that works alright, it can clean up mesh make topology better and reduce polycount. I haven't attempted it on this steering wheel. But I believe it probably could reduce the polycount at least by half by keeping most of visible quality. Meshy AI is also supposed to introduce their low poly, retopology functions in near future that I assume might be pretty good and already has beta version that works ok.

So in near future I assume some less visible objects might become mostly usable, first of all lighter on performance. To me bigger problem is that you mostly get textures/material per object, ideally multiple objects would share materials, and overall amount of materials would be kept to minimum. Always nicer to implement models with rationally managed materials. It is better for performance to have less materials to have less draw calls, isn't it so ? At least I remember it was so in Assetto Corsa.

In Blender there is "Decimate" tool that is same thing as 3Dsmax resizement tool. It also almost is good only for LODS, or meshes that won't certainly ever be tinkered with again, as they totally mess up topology.

 

[gianluca]

Guys, out of curiosity, what program are you using for 3D modeling cars and tracks? I see concept artists using Plasticity in combination with Maya/Blender for hard surface modeling. Would it make the workflow quicker os slower for videogames cars?

 

[mantasisg]

I don't know what Plasticity is, never heard about it. I just use Blender. Big guys who are super rich use 3Dsmax. You also obviously need something to work with texturing. Tools branches out. For example I have used "Materialize" a lot of times to adjust textures, make normals, specular maps... If you work with Blender or anything that is not 3Dsmax you will also almost certainly need 3DsimED for implementing rF2 models.

So far I am having this:

I would start working from there. I made two AI generated models, one from front 45degrees angle, other from rear 45 degrees angle. Find best reference picture you can. I find right now that for this task at the moment Hunyuang is working well. Basically I just cut them in half and join together to have better ends from each. The further AI model goes deep into perspective the more off it gets. Also AI doesn't now correct scales, so have to adjust that, best way would probably be to get car dimensions and wheelbase and trackwidth data to make scaling is all correct. You can see that meshes in the middle aren't fitting well, but it is small issue. What I have there is superb base to start modeling, get the general shape, even down to some more subtle details. To make a good model inevitably I would have to rely on examining real car pictures and using them as additional reference because AI does make mistakes, although in general view at first glance the models usually look good.

Modeling of these exteriors would be very fast, then hopefully AI could help with element or a few for interior and mechanicals, and there would be so much more time to make good interiors.

AI also helps a lot with textures, I have found that Nano Banana accessible in Meshy is very good to generate textures such as for example tire sidewalls, which for me has always been a bit of an issue. I am sure it could make a lot of seamless pattern textures based on references and so on.

 

[CeeBee]

Kinda sideways but we use this daily in jewellery now for organic stuff, openai/gemini to correct customer suplied images then hunyuan to gerenate the 3d. Real disrupter, we used to have half a dozen Zbrush outsource moddelors charging us obscene money.. they haven't recieved work from us in last 6 months or so

 

[mantasisg]

@CeeBee Interesting, sounds like they lost some easy money. But I guess it depends on how complex the jewelry was. I believe AI will be disrupting some of the work where goal is only assets, which doesn't have to be very clean and can get away with a few AI quirks and no need for neat topology.


********************

Meanwhile I am finding that modding (at least publicly) is also loosing interest across all platforms. I have been trying to get some involvement in racedepartment (overtake). There is similarly no movement going on. I guess everything just got saturated. I have ran into few modders that were unhappy about AI, which is weird, because to me it boosts enthusiasm. They didn't elaborate though, I guess they use too much tinder and caught apathy radiation LOL

I posted this there before, maybe it is not interesting anymore...

I am currently experimenting with AI on generating some kind of usable reference for interior, but it seems like it can not do that, it can't comprehend complex variety of objects in it and that there is no clear shape, AI also seems to like reflections of environment. I guess I was looking for the word - obscure, AI does not like that, but sometimes it is able to predict well what is cut out from view.

On the other hand there isn't anything magical about modeling interiors, shapes usually aren't anything special, although there might be some subtle curves there, but it is not as necessary to get just right as for exterior. When modeling interior, or exterior too, placing details helps a lot to get shapes right as everything has to look right eventually from different angles. For example I sometimes shrinkwrap simple lines as panel gaps placeholders, that helps to judge shapes and positions....

Anyone skilled can probably make objects like these from scratch in an hour. So how much AI stuff is needed there I don't know without trying, but I think it would save hours of looking at pictures and trying to understand shapes and positions, eventually probably would help with texturing pretty well.

From full cockpit picture, AI gave steering wheel and seat. Steering wheel is very off, seat is good.

Steering wheel separately is good, just back steel rim of it is too thick and bolts are off placed.

This is from obscure picture, second seat is usable as reference, not completely correct, but enough information to model proper seat based on it. Seats are actually difficult to model, in general very curved planes that bend close to 90degs and more are difficult.

Alpine A210 dash, I expected full panel plus dash plus steering wheel. Didn't get full panel, Dash is good. Steering wheel is a bit off, steering column is complete fiction, air vent is oval, should be round. But the dash is pretty good, few details are fiction.

Lagonda DP115 steering wheel, AI got close, few mistakes, but mostly great reference.

Yeah I think AI could eventually be some sort of ghetto photogrammetry tool for interiors for object by object if you have good photos.

 

[CeeBee]

Even things you would not expect it to be able to get right it does well. Hunyuan leads this race.
scale is way off of course but for a free tool its pretty damn good, and this clip was back in V2.5 the new model is insanely better.

https://www.facebook.com/video.php?v=1391329161897901

 

[mantasisg]

Yes it is pretty good, I also tend to use Hunyuan more often now than meshy.AI. I use meshy.AI more for 2d images generation, and modeling fro mthose very images, and buildings and stuff with letters, it works so good and I am also looking forward for it improving.

Here is Maserati shifter by Hunyuan:

If it wasn't 50 000 tris, It could be placed right into the game. Fortunately it would be pretty simple now to just model over it with proper topology and bakign over this very texturing to new geometry. Hunyuan has perry good retopology though, but it is not always good enough.