Hi,
I am working with a sparse matrix A of integers with a size of roughly M x N = 6 000 000 x 100 000. Previously I was using the SVD implementation in SciPy in Python, but I could simply not fit my whole matrix in memory. I then found out about IRLBA and how it efficiently handles sparse matrices, somthing that is yet to be implemented in Python, so I turned to R and your library.
When I run my calculatation I get the following error:
> S <- irlba(A, dim(A)[2])
Error in irlba(A, dim(A)[2]) :
max(nu, nv) must be strictly less than min(nrow(A), ncol(A))
By design, my matrix A is of full column rank (rank(A) = N) and all singular values exist. I am aware that IRLBA recommends using regular SVD when many singular values are desired, but I just want to utilize the benefits of sparsity in this case, which SVD clearly lacks. If I exclude one of the singular values (N-1) I am worried that the solution will be non-physical and that my solution is just pure nonsense for my analysis.
So, is it easy and safe to switch off this internal check in your code or do I have to take any other measure? Or maybe you can recommend a different solution altogether?
Thanks!
Hi,
I am working with a sparse matrix
Aof integers with a size of roughlyM x N = 6 000 000 x 100 000. Previously I was using the SVD implementation in SciPy in Python, but I could simply not fit my whole matrix in memory. I then found out about IRLBA and how it efficiently handles sparse matrices, somthing that is yet to be implemented in Python, so I turned to R and your library.When I run my calculatation I get the following error:
By design, my matrix
Ais of full column rank (rank(A) = N) and all singular values exist. I am aware that IRLBA recommends using regular SVD when many singular values are desired, but I just want to utilize the benefits of sparsity in this case, which SVD clearly lacks. If I exclude one of the singular values (N-1) I am worried that the solution will be non-physical and that my solution is just pure nonsense for my analysis.So, is it easy and safe to switch off this internal check in your code or do I have to take any other measure? Or maybe you can recommend a different solution altogether?
Thanks!