William Vambenepe, VP Product, Mithril
Randolph Chung, CTO, MantisGrid AI
Introduction: Democratizing World-Class Compute
Demand for high-performance GPUs has transformed compute into humanity's most critical operational resource. Founded by CEO Jared Quincy Davis with the mission to "orchestrate the world's compute capacity, rendering it universally accessible and useful," Mithril is building a new breed of public AI cloud. Mithril’s goal is simple yet ambitious: make accessing AI compute as seamless and reliable as turning on a light switch.
At a recent keynote, William Vambenepe, VP of Product at Mithril shared how Mithril delivers real-time GPU allocation optimization. By aggregating compute supply from a wide variety of infrastructure providers, Mithril optimizes GPU efficiency under the hood and delivers it to AI practitioners through flexible, cost-effective cloud mechanisms.
How Mithril Works: Algorithmic Allocation & Flexible Economics
Rather than relying on static, rigid cloud pricing, Mithril operates as a dynamic orchestration engine that pairs multi-supplier GPU inventory with flexible consumption models tailored to diverse AI workloads.
1. The Spot Market
Mithril’s spot market allows users to specify their compute parameters (instance types, GPU count, storage, location) and set a maximum price limit. Users never pay more than their limit; instead, they pay the real-time market-clearing price. Customers apply this mechanism to meet the needs of their workloads, with the two extremes being:
- High-Limit Strategy: Bidding above market rate for immediate, uninterrupted execution—ideal for mission-critical training runs.
- Low-Limit Strategy: Bidding near market floor allows fault-tolerant, background workloads (e.g., batch inference or parameter sweeps) to run at steep discounts whenever capacity opens up, yielding massive cost savings.
2. Flexible Reservations
For workloads that cannot tolerate preemption, Mithril offers Flexible Reservations. Users secure dedicated nodes, but retain the freedom to temporarily pause or permanently return instances in exchange for credits:
- Pause Credits: Temporarily pausing a node generates pause credits. The user retains ownership of the node, while Mithril seamlessly re-allocates the idle hardware into the spot market.
- Return Credits: Permanently returning a node releases it back to Mithril’s general supply pool for both spot and reservation markets, yielding higher credit rates.
The Operational Challenge: Heterogeneity and Stranded Compute
Operating an AI omnicloud at hypercloud reliability comes with unique operational hurdles, especially for a lean engineering organization:
- Infrastructure Diversity: Mithril manages heterogeneous environments spanning bare-metal servers and virtualized instances, across diverse data center providers.
- Full-Stack Complexity: Maintaining optimal performance requires monitoring the entire hardware and software stack—GPUs, host CPUs, storage systems, Infiniband fabrics, network uplinks, and Kubernetes orchestration layers.
- The "Stranded Capacity" Problem: In GPU infrastructure, a server that is offline or unlisted on the market due to a soft fault inherently suffers from 0% utilization. Unlocking and recovering this stranded compute is directly linked to market availability and unit economics.
The MantisGrid AI Solution: Full-Stack Diagnostics & Capacity Recovery
To maintain hypercloud-grade quality of service across complex infrastructure, Mithril partnered with MantisGrid AI. MantisGrid AI provides autonomous reliability and continuous optimization across the AI infrastructure stack—correlating signals across GPUs, servers, networks, virtualization, and Kubernetes to detect issues, identify root causes, recover stranded capacity, and continuously improve utilization, availability, performance, and operational efficiency.
- Automated Root Cause Analysis: Unlike legacy SRE, which relies on reactive alerts and siloed monitoring, MantisGrid AI autonomously correlates cross-stack signals from hardware to Kubernetes to instantly surface root causes. Rather than inundating on-call engineers with disconnected telemetry, MantisGrid AI learns the relationships between hardware metrics, kernel logs, and cluster events to surface intelligence in pre-analyzed incident reports featuring clear root causes and recommended and automatable actions. This significantly cuts down on the number of tickets that the operations team has to service, and reduces mean time to detection and repair.
- Recovering Stranded Compute: By rapidly diagnosing hardware glitches that prevent nodes from joining the market, MantisGrid AI helps Mithril safely return stranded capacity back to the active supply pool through automated remediation. When operational anomalies occur, they are automatically triaged and approved automated actions—such as returning a degraded host to an upstream supplier or recycling a stuck driver—are executed instantly. Meanwhile, underlying software flaws are routed directly to development teams to ensure vulnerabilities are fixed permanently at the source.
- Optimizing Sellable Capacity: MantisGrid AI continuously identifies idle, underutilized, paused, or stranded GPUs and nodes across Mithril’s heterogeneous supply pool. By detecting placement inefficiencies, node idle gaps, instance-transition delays, and preemption churn, MantisGrid AI provides optimization intelligence and automated workflows that help recover capacity, reduce wasted compute cycles, and maximize the GPU capacity available to Mithril’s marketplace.
The Road Ahead: Towards Autonomous Infrastructure
The partnership between Mithril and MantisGrid AI is advancing toward a new operating model for AI infrastructure—where reliability, performance, utilization, and cost are continuously optimized, and operational issues can be detected, diagnosed, and safely remediated with increasing levels of autonomy.
By combining Mithril’s dynamic compute orchestration with MantisGrid AI’s autonomous reliability and continuous optimization platform, we are building toward a resilient, self-optimizing AI cloud that continuously learns from infrastructure behavior, recovers stranded capacity, reduces operational friction, and maximizes billable GPU utilization.
The goal is simple: maximize GPU yield and availability while delivering reliable, high-performance AI compute to builders worldwide.