Umbra: Private Inference on Attested Apple Silicon

Introduction

Umbra is a platform that connects idle high-end Apple Silicon computers with developers who need private and uncensored AI tools. It acts as a bridge between unused Mac hardware and the growing demand for AI models that major cloud providers often block or censor. By using Apple's built-in security features, Umbra ensures that no one, not even the machine owner or the service coordinator, can read the messages sent between users and the AI.

Benefits

Umbra offers two main advantages for different groups of people. For developers, it provides access to a wide catalog of open-weight models, community fine-tunes, and uncensored versions that are not available on standard cloud APIs. This allows them to use powerful AI tools without restrictions. For Mac owners, the platform turns their idle computer power into a money-making opportunity. Users can host public AI models on their own M1 or newer Macs and earn revenue every time someone uses their machine to process a request.

The most important benefit is privacy. Umbra uses a strict security system called attestation. This means the AI request is encrypted before it leaves the user's device. The machine owner cannot see the content because the private keys stay locked inside the chip's Secure Enclave. The system also checks that the correct model is running and that nothing is being saved to a disk or log file. This creates a trusted environment where data remains strictly private.

Use Cases

Umbra is designed for developers who need specific AI models that are not hosted by big tech companies. It works well for anyone using OpenAI or Anthropic compatible tools. Developers can simply point their code to the Umbra base URL and start using the models without changing their existing infrastructure. The platform supports streaming responses, which makes it easy to integrate into current workflows.

Mac owners can use the platform to monetize their hardware. They install the software, choose which public models they want to host, and then wait for requests to come in. This is ideal for people who already have powerful Apple computers sitting idle and want to put them to work. The system handles the routing automatically, sending requests to the Mac that has the right model and available capacity.

Pricing

Umbra offers a free tier for new users. When you sign up, you get up to 10 million free tokens without needing to add a credit card. For paid usage, the cost is based on the number of tokens processed. The price varies depending on the specific model you choose. For example, some models cost $0.10 per million input tokens and $0.30 per million output tokens, while others are cheaper. Users can check the live rates by looking at the models list in the system.

Mac owners who host models earn money per token. The amount they make depends on the demand for the models they host and the rates set by the platform. This creates a flexible earning opportunity for anyone with a compatible Mac.

Vibes

The platform is currently in an alpha phase, which means it is still being tested and improved. Early feedback suggests that the focus on privacy and security is a major selling point. Users appreciate the ability to access uncensored models and the transparency of the security architecture. The developer experience is praised for being simple and compatible with existing tools. However, since it is in early stages, some features like signed receipts for every response are still rolling out.

Additional Information

Umbra is built on top of Apple's hardware security stack, specifically using the Secure Enclave and Mobile Device Management features. This technical foundation ensures that the privacy promises are backed by the hardware itself. The platform uses a tool called llama.cpp to run the AI models directly in the computer's memory, which reduces the risk of attacks and lowers latency. This approach minimizes the attack surface and makes the system more secure than traditional server setups.