Repository navigation
Cuda version #117
Description
Activity
Actually, a follow-up of my question, I think the issue can be resolved by installing from source.
Instead of
pip install pytorch-fast-transformers, cloning the repo and runningpython setup.py installgets rid of the NaN I had before.But I'd like to hear more thoughts.
Actually, a follow-up of my question, I think the issue can be resolved by installing from source.
Instead of
pip install pytorch-fast-transformers, cloning the repo and runningpython setup.py installgets rid of the NaN I had before.But I'd like to hear more thoughts.
I followed your instructions ( pytorch1.7.1 + cuda11.0 & install from source ) but still have the NaN error.
It seems the function in 'causal_product_cuda.cu' cannot be imported.
Is this package tested on more recent cuda and pytorch versions?
My code calls
fast_transformers.causal_product, which is actually the only function I call from this package.I setup this package with latest pytorch 1.13.0+cuda11.6, and get NaN errors at training. This, however, doesn't happen with the older pytorch 1.7.1+cuda11.0.