AI 3D Generation Models, Applications and Tools: The Big List

When it comes to technology hype, AI is overshadowing the metaverse to capture the attention of businesses and consumers everywhere.

But AI could further enhance the metaverse, at least in one sense: asset creation. AI has the potential to scale the creation of 3D assets for use in virtual environments.

AI Business explores the various AI 3D generation capabilities.

What is AI 3D generation?

AI 3D generation uses artificial intelligence to generate 3D models or objects.

There are several techniques:

Each technique can potentially help creators customize and generate virtual assets and environments at speed, especially helping those without prior 3D modeling expertise.

Potential applications of AI 3D generation are for creating assets in video games and metaverse environments, as well as design concepts for products in business. It could even be applied to creating virtual representations of factory floors, generating potential mock-ups of industrial plants.

Text-to-3D AI models

DreamFusion

Creators: Google, UC Berkeley

First published: September 2022

DreamFusion can generate 3D models from text inputs. It is essentially an expanded version of Dream Fields, a generative 3D system Google unveiled back in 2021.

Related:12 Language Models You Need to Know

DreamFusion requires little to no prior training – meaning it can generate 3D representations of objects without 3D data. It uses 2D images of an object generated by the Imagen text-to-image diffusion model to understand different perspectives of the model it is trying to generate.

Read the DreamFusion paper.

Explore 3D models generated by DreamFusion.

Read more about DreamFusion on AI Business.

Point-E

Creator: OpenAI

First published: December 2022

Point-E works the same as OpenAI's image generation tool DALL-E – simply type in something like, 'a corgi wearing a Santa hat' and the system will generate a 3D model.

Point-E can generate 3D models in just one to two minutes on a single GPU. It also boasts image-to-3D capabilities, meaning images can be used in tandem for inputs alongside text.

Read the Point-E paper.

Access the Point-E code.

Read more about Point-E on AI Business.

GAUDI

Creator: Apple

First published: August 2022

Named after the famous Spanish architect, Gaudi was created by researchers at Apple to generate 3D scenes from text prompts. Users can redraw the scene from any angle using natural language – allowing for the creation of interior scans of buildings.

GAUDI also can generate new camera movements through 3D indoor scenes via text, such as a user input to 'go through the corridor.'

Related:AI Image-Generation Models and Tools: The Ultimate List

Read the GAUDI paper.

Explore GAUDI 3D generations.

Read more about GAUDI on AI Business.

MVDream

Creators: TikTok parent ByteDance, University of California, San Diego

First published: 31 August 2023

MVDream is one of the newest models on this list. Researchers from ByteDance and the University of California, San Diego developed a diffusion model that can generate high-quality 3D renderings from text inputs.

MVDream can generate multiple views of the model that are geometrically consistent multi-view images.

The model can be fine-tuned for personalized 3D generation using tools like DreamBooth3D.

Read the MVDream paper.

Magic3D

Creator: Nvidia

First published: March 2023

Magic 3D is a pre-trained text-to-image diffusion model. It can create quality 3D mesh models from text prompts.

Magic3D utilizes a two-stage process:

Magic3D can create high-quality 3D mesh models in 40 minutes.

Read the Magic3D paper.

Image-to-3D AI models

COLMAP

Creator: Johannes L. Schönberger, principal scientist at Microsoft Mixed Reality & AI Lab

First published: October 2021

COLMAP is an open source 3D reconstruction pipeline with a graphical and command-line interface. It can be used to generate detailed 3D environments from a handful of images.

COLMAP users can export a 3D mesh – although to refine it, programs like MeshLab would be required.

Access COLMAP via GitHub.

Read more about COLMAP on AI Business.

Related 3D AI models

PointLLM

Creators: The Chinese University of Hong Kong, Shanghai AI Laboratory, Zhejiang University

First published: August 2023

PointLLM is designed to aid 3D generation. It's an approach that helps large language models like PaLM and Llama understand 3D data.

Using PointLLM, AI researchers can greatly utilize point clouds – data points that represent a 3D shape or object – in their datasets.

Read the PointLLM paper.

Access the PointLLM codes and dataset.

AI 3D generation applications to try

Gliastar

AI-powered video creation tool that can animate a brand's characters through text inputs.

Simply provide Gliastar with a 2D or rough 3D representation of your character, integrate the model into the GliaStar AI-Animation system, which will then automatically generate a 3D animation of the desired mascot in minutes.

Try Gliastar for yourself - https://www.gliastar.com/

3DFY.ai

3DFY.ai can create high-quality 3D models from a text prompt or a single image.

3DFY.ai 3D generation pipeline can automatically generate 3D outputs at speed – with users able to test out the text-to-3D web service to try out prompts of their own.

Try 3DFY.ai for yourself - https://3dfy.ai/

Fotor

Fotor lets you generate 3D models from text descriptions. Users can create detailed 3D models for various applications.

Fotor also has an AI 3D character generator where users can create unique and detailed characters in minutes.

Check out Fotor - https://www.fotor.com/features/ai-3d-model-generator/

Spline AI

With Spline, users can generate 3D models and representations in real time using natural language inputs.

Spline can be used not only to make 3D models – but also animations and textures.

Explore Spline AI - https://spline.design/ai

Meshcapade

Meshcapade builds virtual avatars – using generative AI to create accurate digital humans for business applications, like e-commerce settings.

Meshcapade's SMPL system uses machine learning to capture the statistics of an individual's 3D body shape.

Try Meshcapade for yourself - https://meshcapade.com/

Masterpiece Studio

With Masterpiece Studio, you can generate 3D assets using AI.

Masterpiece Studio builds 3D creation software that users with non-technical backgrounds can use to create 3D content from concept to animation.

The Masterpiece team partnered with Meta to build Masterpiece X, which allows creators to remix existing 3D assets. It's also accessible on the Meta Quest store.

Explore Masterpiece Studio - https://masterpiecestudio.com/

Luma AI

With Luma, users can generate photorealistic 3D assets and environments in minutes.

Luma is available via mobile – so iOS users can use its 3D generation tools from their handsets.

Check out Luma AI - https://lumalabs.ai/