SCADBench

SCADBench is an open platform created to test how well AI language models can create 3D objects. It acts as a place where people can compare and choose the best 3D models made by different AI models from the same text description.

Benefits

SCADBench helps users see which AI models are best at generating 3D objects. By using a crowdsourced system, it provides a fair way to rank these AI models based on the quality of their creations. This allows for a clear understanding of AI capabilities in 3D design.

Use Cases

This platform is useful for anyone interested in the progress of AI in 3D modeling. Developers can use it to test and improve their AI language models. Designers and hobbyists can explore the variety of 3D objects AI can create from simple text prompts. The "Arena" feature allows users to directly compare two AI-generated models side-by-side and vote for their favorite, contributing to the overall ranking.

Vibes

SCADBench uses a blind comparison system where users vote on which of two AI-generated models they prefer without knowing which AI made them. After voting, the AI identities are revealed. The rankings are determined using the ELO rating system, similar to how chess players are ranked. This ensures that models are judged on their output quality rather than their name.

Additional Information

SCADBench uses a detailed six-step process to create and evaluate 3D models. This includes sending prompts to AI models, extracting code, compiling it into 3D shapes, separating parts, assigning materials, and rendering the final model in Blender. The platform also features a "Gallery" to view all models and a "Leaderboard" showing model rankings. SCADBench is seeking support from users to help cover the costs of its API calls and rendering infrastructure.