A project is trying to cut the cost of making machine learning applications for Nvidia hardware, by developing on an Apple Silicon Mac and exporting it to CUDA.
Machine learning is costly to enter, in part due to the expensive hardware it uses to service queries at speed. While Nvidia's chips and components are favored due to their performance, Apple is trying to make it easier for developers to use that hardware.
Work is being on Apple MLX, the company's open-source machine learning framework, to add CUDA backend support, Ycombinator. CUDA is Nvidia's software layer for working with its graphics processing units (GPUs) on graphics cards, which is also used to handle processing tasks such as machine learning.
By adding CUDA support to MLX, the project will make it so that code can be developed using MLX. Crucially, that code can then be exported in a format that works with CUDA.
To be clear, the addition of support is specifically for exporting the code from an Apple-friendly MLX environment so it will work on Nvidia graphics cards and server hardware.
This does not mean you can attach an Nvidia card to a Mac Pro or an eGPU enclosure to use it locally on a Mac for ML applications. It also won't suddenly allow ML projects coded for CUDA to run on Apple Silicon either.
For developers, this will give them the opportunity to write and test code locally on an Apple Silicon Mac, before deploying it to Nvidia's CUDA-based hardware.
There are potential cost benefits to the project. Nvidia's hardware setups can get very expensive, multiple times that of a high-specification Mac at least.
Instead of having dedicated Nvidia hardware for development, organizations can save money by using Apple Silicon for small-scale efforts before scaling up for production.
There is also the benefit of MLX projects being run on a CUDA-based system with far higher performance than a Mac. With Nvidia hardware widely used for machine learning applications, this could be a big opportunity for developers to use more powerful hardware, too.
Still in development
Listed as a work-in-progress, the project says it is work "sponsored by Apple," indicating it has some backing from the company. We could not independently verify this claim.
While work on the backend started on March 21, it is still in development.
The top post claims that not much works at the moment, but the tutorial example apparently does. Testing has also been performed on a device running Ubuntu 22.04 with CUDA 11.6.
It is unlikely that the project will be fully usable anytime soon, due to the complexity involved. However, with Apple's potential backing and a potentially great benefit for other developers, there is an impetus to continue and get the project over the line.
Machine learning research
Having a project to make Apple's machine learning platform work with Nvidia hardware may seem like an odd pairing, considering the history of the two companies. Apple famously stopped cooperating with Nvidia on drivers for macOS Mojave, killing off the ability to use external GPUs with the Mac.
However, under the umbrella of machine learning research, the two companies have been more friendly.
In December, Apple released research in how it created Recurrent Drafter (ReDrafter), a speculative decoding method that sped up LLM token generation. Specifically, it explained how it ported ReDrafter to work with Nvidia GPUs.
With machine learning becoming more of a feature of modern day computing, there is a massive incentive to improve performance.
