On Democratising AI and the GPU Shortage: Part 3
White Star Capital Digital Assets Fund - Newsletter #160
On Democratising AI, the GPU Shortage and the Potential of Decentralised Training and Inference Networks
White Star Capital Digital Asset Fund - newsletter #160
By Marthe Naudts
Last time, I looked at the state of the GPU industry and explained how Nvidia’s latest H100 and A100 chips are the near sole hardware behind the AI boom.
Faced with both skyrocketing demand and supply bottlenecks, even hyper scalers like Amazon Web Services (AWS) and Google Cloud Platform (GCP) cannot keep up with demand.
This leaves a multi-billion dollar opportunity for SaaS and marketplace businesses connecting disparate idle compute capacity like CoreWeave and Lambda, or more creative decentralised or blockchain-based solutions seen in the likes of Gensyn, Together.ai and Akash.
Most of these companies compete, at least initially, through lower prices than incumbent hyper scalers. This is not sustainable, and without novel value layers, these marketplaces will simply form a superfluous middle layer, all competing to secure a small piece of a fundamentally commoditised and highly divisible GPU pie.
In Part 2, I explained how companies are competing through clustering and coordinating disparate idle GPUs. In this final Part 3, I will explore how companies can differentiate through building a trust layer between the two unknown parties in their marketplaces.
Businesses are only successful if they manage to attract users and retain them, while growing their lifetime value. And, for peer-to-peer and distributed marketplaces, winners like Uber, Vinted, and Deliveroo often succeeded by adding an extra value layer to their interface which increases trust between the two sides of the exchange. Trust that designer items are verified, that drivers are safe, that delivery times are accurate, and that payment is immediate, disputable, and refundable.
In the case of outsourcing compute power, there are three unique trust assumptions that marketplace engineering, financial, and sales teams will need to ensure.
Verifiable compute
Outsourcing computational tasks from relatively weak devices to powerful computation services is extremely common. The entire internet is run on so-called cloud computing, which is easy to verify because either the proof is in the pudding so to speak, or because the work is reproducible by splitting the state-independent work. But with training large language models, reproducing the work would defeat the entire purpose of saving computational effort. This is because ML problems are state-dependent, meaning each layer in a deep learning model is using the output of the previous layer, so validation by replication would involve doing all the work again. Moreover, this then relies on trusting that the verifying party is also honest, and verifying this would instigate an infinite chain of replication.
Blockchains can be used to create staking and slashing incentive games to force all participants to be honest through rationality. Tromero, for example, is creating ‘proof-of-work’ systems resembling Bitcoin mining and consensus models. Others like Gensyn have explored new methods of proof (such as probabilistic proof-of-learning and graph-based pinpoint protocols), to then confirm on-chain. Either way, because companies like Vast.ai, Lambda, and Fluidstack have managed to exist without this verification piece, these teams will win through the ability of their sales teams to communicate the viability and significance of these complicated mathematical proof solutions to their customers.
Confidential compute
Most GPU marketplace start-ups target ML researchers at academic institutions and AI start-ups as initial customers. But, when presented with decentralised networks, almost all ML researchers I have spoken with expressed concerns about trusting unknown providers with their datasets and the containers of code to remotely run their models. When the model was for an educational purpose, they were typically trained on publicly available or insensitive information, such as online content. But most AI start-ups are competing on novel datasets, which often means unique access to either private enterprise data, or consumer data including sensitive PII like health or financial records. Moreover, outsourcing training also involves sending a container of code which outlines the rules and weights by which to train that dataset. This is, fundamentally, intellectual property. If the customer is a start-up, this IP is all they have.
Marketplaces can therefore carve out a competitive edge by building out software that ensures, either through encryption and proof methods for data at rest or transit, or remote hardware monitoring for data in use that data has not been tampered with. They can also be selective with their supply, only sourcing from reputable data centres verified through extensive KYC and AML checks on hardware providers.
The ultimate guarantee of security for data in use would be to only use fully homomorphic encryption (in which computers can run code without ever decrypting it, which comes with some latency sacrifices), or hardware-based trusted execution environments (TEEs). In the latter case, confidential data is only released once the TEE is determined as trustworthy. Luckily, Nvidia is implementing support for TEEs at the hardware layer in its latest chips, so we can expect this to become the default offering over time.
KYC and Liability
For all marketplaces, suppliers also need to make various trust assumptions about their consumers - mostly about their ability to pay but also, in cases of physical services or an exchange of goods, that they are a verified and safe person. In the case of decentralised compute networks that assure confidentiality described above, the supplier has no idea what code it is running, and therefore needs to trust that malicious entities are not using their systems for harmful purposes like fraud, terrorism, or organised crime. They also want to know that they are not running a DDOS or other security attack on their systems. Marketplaces could therefore either use a third-party KYC provider, or assume liability themselves and compete on providing legal protection and insurance against this.
Final thoughts
As we come to the end of the three-part series, the key takeaway is as follows: due to the unique features of the massive and growing GPU market, there is a lot of potential for unmet demand and underutilised supply to be matched by a decentralised and distributed marketplace. However, the winners will be those that are adding a value layer beyond securing high-end H100/A100 chip supply and undercutting hyperscaler pricing. Cryptocurrencies will only work in this context if they are entirely distinct from the direct payments, and exclusively used as a dividends-based rewards system.
There are two broad areas in which I think this competition will/should fall.
Firstly, cluster coordination: GPU marketplace start-ups should compete with superior clustering of weaker GPUs, both through an improved front-end UX/UI for ML developers, and resource orchestration for the data centres that may not have the right interconnect hardware and software solutions in place to reach this customer pool.
Secondly, security: due to the two parties being unknown to one another, companies can make meaningful differences to different customer and supplier pools by ensuring, insuring and assuring each party that they can trust the other. This applies to verifiable and confidential computing mechanisms, as well as simple KYC checks that may or may not use blockchains as proof systems.
This is an exciting engineering, cryptographic, and mathematical challenge, and we’re excited to see the role of blockchains in the various emerging solutions. If you are building in this space, or would like to discuss any of the above, please reach out to me at marthe@whitestarcapital.com.
🔦 White Star & Portfolio Spotlight
What the Index Coop Is Building In 2024
Read about what Index Coop is up to this year!
The Evolution of Private Credit in the Digital Age
Read Atlendis Labs’ exploration of private credit.
THL/USD price feed is now live!
Thala is amongst the first Aptos-native assets listed on the Pyth network.
🏦 Enterprises & Institutions
B2C2 Gains Luxembourg Virtual Asset License as EU's Crypto Rules Set to Kick In
The liquidity provider is expanding to Luxembourg in a bid to widen its EU presence six months after gaining a license to operate in France.
MetaMask and Robinhood Connect Integrate to Make it Easier to Access Web3
New integration allows customers to purchase crypto through Robinhood’s low cost order engine, facilitating web3 adoption through a secure, user-friendly, and self-custodial approach.
CleanSpark jumps on plans to buy four bitcoin mining facilities ahead of the halving
The company agreed to buy three “turnkey” sites — meaning they need only to plug their existing hardware into the facility — in Mississippi for $19.8 million in cash. That transaction will close within 21 days. The company expects the sites to support about 14% of its revenue shortly after closing.
⚖️ Government & Regulation
Thailand slashes crypto trade tax in push to become digital asset hotspot
The Ministry of Finance said Tuesday it will extend a tax exemption previously in place for cryptocurrency trading, according to local reports.
Hong Kong Chamber of Commerce Wants Chinese Yuan-Backed Stablecoin
The business advocacy organization suggested the issuance of yuan-backed stablecoins alongside the launch of a "Virtual Asset Connect Scheme."
US Treasury Secretary Calls For Crypto Legislation On ‘Non-Security’ Tokens
Existing laws leave gaps in crypto that enforcement agencies don't account for, Janet Yellen argues.
Polish town adopts stablecoin for local commerce
The Polish town of Mińsk Mazowiecki is launching a stablecoin that users can spend at local businesses. The so-called MinsCoin debuted alongside UrbanChange, a platform that launches USDC-backed stablecoins meant for local use in cities.
SEC Delays Another Ether ETF Application
The U.S. Securities and Exchange Commission (SEC) on Tuesday delayed a decision on a spot ether (ETH) exchange-traded fund (ETF) jointly proposed by Invesco and Galaxy Digital, a filing shows.
Judge approves BlockFi and 3AC settlement deal but details remain confidential
A U.S. Bankruptcy Court approved on Tuesday a settlement between crypto lending firm BlockFi and collapsed crypto hedge fund Three Arrows Capital, with the settlement details remaining sealed.
💰 Funding & Exits
Code Raises $6.5 Million to Revolutionize Microtransactions for Digital Creators
Code, a groundbreaking global payments platform enabling microtransactions for online creators, secures $6.5 million in seed funding to accelerate growth and democratize digital monetization.
Filecoin liquid leasing protocol Glif raises $4.5 million, offers reward points ahead of token launch
Multicoin Capital led the round, with Zee Prime Capital, Fintech Collective, Big Brain Holdings, Protocol Labs and other investors participating.
Singapore-based Web3 startup nets $3.5m funding
Startale Labs, a Singapore-based Web3 startup, has raised US$3.5 million in a series A follow-up round, according to Alternatives.pe.
GigaStar Secures $3M in Additional Funding to Bolster a New Asset Class in the Creator Economy
The funding will enable the firm to accelerate efforts to scale GigaStar Market (its current primary market platform), attain its broker-dealer registration, and develop its secondary market, which is anticipated to launch by the end of 2024.
Layer 2 startup LightLink secures $4.5 million in seed round led by T&B Media Global and MQDC
LightLink's technology will help support a "multiverse project" called Translucia that "blends the virtual and physical worlds" and is backed by $300 million in funding from T&B Media Global MQDC.
Omega Secures $6 Million in Funding from Lightspeed Faction, Borderless Capital, Bankless Ventures, XBTO Humla, Blockchain.com, Wave Digital, and others
Omega will use these funds to introduce DeFi ecosystems to Bitcoin holders.
Kodiak Finance Raises $2M Seed Round
The funding will further enable them to increase the pace at which they scale Kodiak and allow them to continue delivering cutting-edge trading products native to Berachain.
Merlin Chain Secures Funding to Empower “Bitcoin-native” Innovations
Merlin Chain offers a native scaling solution that integrates ZK-Rollup network, decentralized oracle network, and on-chain BTC fraud proof modules.
Nibiru Chain Secures $12 Million to Fuel Developer-Focused L1 Blockchain
Venture investments included contributions from Kraken Ventures, ArkStream, NGC Ventures, Master Ventures, Tribe Capital, and Banter Capital.
Crypto Payments App Oobit Raises $25M in Series A Funding Round Led by Tether
The round also included participation from Anatoly Yakovenko, the co-founder of Solana.
Bowled.io raises an undisclosed funding round from global investors
Singapore-based Bowled, the world’s fastest-growing sports tech company built at the intersection of sports, gaming, and blockchain technology, raised an undisclosed amount from marquee global investors, including Axie Infinity’s Sky Mavis Labs.
Web3 Accelerator Beacon Raises Over $150 Million in First Year
According to Foresight News, Polygon co-founder Sandeep Nailwal's Web3 accelerator, Beacon, has announced that it has raised over $150 million in funds during its first year of operation.
Hybrid crypto exchange Cube reaches $100 million valuation in new funding round
The new round comes just over three months after Cube raised $9 million in seed funding.
After closing $9 million round, Delegate Labs announces Clusters, a service to connect crypto wallets across chains
Clusters is a cross-chain, multi-wallet name service that allows users to manage digital assets in a unified place.
🚀 Project Launches & Updates
Bluesky is now open for anyone to join
After almost a year as an invite-only app, Bluesky is now open to the public. Funded by Twitter co-founder Jack Dorsey, Bluesky is one of the more promising micro-blogging platforms that could provide an alternative to Elon Musk’s X.
No Cap: Buzzy Eigenlayer Blows Past $3 Billion Total Value Locked
With limits removed until Friday, Feb. 9, the security-focused staking protocol is trying to balance being neutral and being decentralized.
Helix brings Japanese Yen on-chain
Helix, a decentralized exchange on Injective, will launch the Japanese Yen (JPY) forex pair on its platform, making it one of the first exchanges to introduce the Yen onto the blockchain.
🔥 Other Bits We're Excited About
Ethereum experiment ERC404 is an NFT-cryptocurrency hybrid — and it’s already generated $87m in trades
ERC404 is a novel token standard on Ethereum, combining the attributes of NFTs and fungible tokens.