In nuce: The US company OpenAI, which researches artificial intelligence, has published Point-E. Point-E can be used to create 3D objects using text input. This is according to a report by journalist Andrew Tarantola, published on the Engadget news service. Point-E works in a similar way to another development from OpenAI: Dall-E, the text-image generator. The difference is that Point-E creates 3D models – Dall-E only creates still images. The background to this: Dall-E is an artificial intelligence that is used to generate images based on text input (these text inputs are also known as prompts in technical jargon). Digital Production last reported on Dall-E on 27 October 2022 , when the generator was integrated into the Microsoft Bing internet search engine.
In toto: Point-E is used to create 3D point clouds, also based on prompts. While DreamFusion from Google – as Andrew Tarantola writes – “usually takes several hours to create 3D objects”, Point-E minimises the process to just a few minutes. In addition, Tarantola goes on to say that Point-E should already work properly with just one GPU – instead of, as with DreamFusion, only producing results when several GPUs are used.
How does Point-E work? In the scientific paper Point-E: A System for Generating 3D Point Clouds from Complex Prompt , the team led by Alex Nichol, researcher at OpenAI, writes that Point-E utilises “a large corpus of (text, image) pairs”. However, Nichol and his team go on to explain that Point-E was trained using a “smaller dataset of (image, 3D) pairs”. The paper also discusses how 3D objects are created with the support of Point-E. The article states that in the case of a text input (e.g. “A cat eats a burrito”), Point-E first generates a synthetic 3D representation of a burrito-eating cat. The image generated in this way is then run through several diffusion models; these models serve as the basis for a 3D RGB point cloud – first for a coarse 1,024-point cloud model, then a finer 4,096-point model.
Click to continue: on 6 September 2022 , Digital Production showed how the text-to-image AI Stable Diffusion allows users to create images using text input.
Source: engadget.com ( report by Andrew Tarantola)
