For people interested in this: I've been working on an open-source libtpu implementation for https://tenstorrent.com/ accelerators as a side project, it can already run most of torchtpu, see https://github.com/pcmoritz/libtt (based on the tenstorrent software stack). It shows all the things involved in lowering torch to tile based accelerators.
show comments
dalemhurley
Google and Apple are going to move consumers and ordinary business users to edge inference. This is more disruptive to the upcoming IPOs than most investors understand.
show comments
npalli
Interesting, Amazon doing the same for AWS Trainium.
Cool! XLA is pretty good + potentially a better compiler infrastructure than inductor which will lead to improved performance on all hardware.
Also I guess this is the final nail in the coffin of Tensorflow.
ismailmaj
Amazing news, I remember the days when you had to fight XLA to make PyTorch run on TPUs, it seems we finally got first-party support!
Since HW specifics live mostly in the kernel layer (Pallas or CUDA), I imagine the Jax vs PyTorch question is more of a style preference now?
nico
Pardon my ignorance, but who else besides Google has TPUs? Are they commercially available somewhere? What’s their main application right now outside of Google?
show comments
23agt
Google is selling TPU shovels via Cloud services and also physically to select partners. So this makes sense.
There will be an abundance of shovels. Nebius already crashed by almost half from its hype peak.
show comments
Lucasoato
Are there TPUs outside Google Cloud?
show comments
amelius
Soon to be written by AI anyway, so they might as well release it now.
yipinwong
Dumb question from non-AI engineer.
Is this somehow Google's answer to today's best DeepSeek open-sourcing inferencing engine?
show comments
saucesaft
bad news for jax
show comments
nelaggy
so if the moat isn't the model and the moat isn't the inference layer, i guess the moat is the hardware now? what does this mean for frontier labs?
For people interested in this: I've been working on an open-source libtpu implementation for https://tenstorrent.com/ accelerators as a side project, it can already run most of torchtpu, see https://github.com/pcmoritz/libtt (based on the tenstorrent software stack). It shows all the things involved in lowering torch to tile based accelerators.
Google and Apple are going to move consumers and ordinary business users to edge inference. This is more disruptive to the upcoming IPOs than most investors understand.
Interesting, Amazon doing the same for AWS Trainium.
https://github.com/aws-neuron/torch-neuronx
Cool! XLA is pretty good + potentially a better compiler infrastructure than inductor which will lead to improved performance on all hardware.
Also I guess this is the final nail in the coffin of Tensorflow.
Amazing news, I remember the days when you had to fight XLA to make PyTorch run on TPUs, it seems we finally got first-party support!
Since HW specifics live mostly in the kernel layer (Pallas or CUDA), I imagine the Jax vs PyTorch question is more of a style preference now?
Pardon my ignorance, but who else besides Google has TPUs? Are they commercially available somewhere? What’s their main application right now outside of Google?
Google is selling TPU shovels via Cloud services and also physically to select partners. So this makes sense.
There will be an abundance of shovels. Nebius already crashed by almost half from its hype peak.
Are there TPUs outside Google Cloud?
Soon to be written by AI anyway, so they might as well release it now.
Dumb question from non-AI engineer.
Is this somehow Google's answer to today's best DeepSeek open-sourcing inferencing engine?
bad news for jax
so if the moat isn't the model and the moat isn't the inference layer, i guess the moat is the hardware now? what does this mean for frontier labs?
Is the Linux driver available?