Conversation
The default was AutomaticBlockSize(max_cpu_bytes = 1 << 30), which means to use up to 1GB of memory (unless any block size would use >1GB). Setting `bin/sum_product.py -B 64` makes a difference for parsing long strings.
|
Is there any way to do this without passing the block_size option through so many functions? Like hide multiple options inside an "Options" object? |
This commit eliminates the following out-of-memory error when parsing a
string of length 93:
```
Traceback (most recent call last):
File "fggs/bin/sum_product.py", line 91, in <module>
zs = fggs.sum_products(fgg, method=args.method, tol=args.tol, kmax=args.kmax)
File "fggs/fggs/sum_product.py", line 455, in sum_products
comp_values = SumProduct.apply_to_patterned_tensors(fgg, comp_opts, inputs.keys(), comp_labels, *inputs.values())
File "fggs/fggs/sum_product.py", line 412, in apply_to_patterned_tensors
(nonphysicals, *physicals) = SumProduct.apply(
File "python/lib/python3.8/site-packages/torch/autograd/function.py", line 506, in apply
return super().apply(*args, **kwargs) # type: ignore[misc]
File "fggs/fggs/sum_product.py", line 362, in forward
newton(lambda x: F(fgg, x, inputs, semiring),
File "fggs/fggs/sum_product.py", line 97, in newton
dX = multi_solve(JF, F0 - x0)
File "fggs/fggs/multi.py", line 224, in multi_solve
b[z].copy_(a[z,z].solve(b[z], semiring))
File "fggs/fggs/indices.py", line 1305, in solve
x = semiring.solve(dense_a, dense_b)
File "fggs/fggs/semirings.py", line 142, in solve
return Semiring.solve(self, a, b)
File "fggs/fggs/semirings.py", line 83, in solve
a[:,k+1:] = self.add(a[:,k+1:], self.mul(a[:,k,None], a[k,k+1:]))
File "fggs/fggs/semirings.py", line 107, in mul
return x.mul(y).nan_to_num_(nan=0., posinf=inf)
RuntimeError: [enforce fail at alloc_cpu.cpp:75] err == 0. DefaultCPUAllocator: can't allocate memory: you tried to allocate 10207632960 bytes. Error code 12 (Cannot allocate memory)
```
Unfortunately we can't use `addcmul_` because we need `nan_to_num_`
To do: allow setting `block_size` instead of `AUTOMATIC_BLOCK_SIZE`
alongside dtype and device The diff is much smaller now
|
Which way (extra argument vs. hidden inside Semiring) do you like better? Could the argument you made about different einsums in the same computation requiring different block sizes apply to hiding the block size inside Semiring? |
The hidden-inside-Semiring way is growing on me; it helps that it makes the diff much smaller. I thought about different einsums in the same computation and feel that they would have different Semirings (so could still have different block sizes). |
| a[:,k] = self.mul(a[:,k], self.star(a[k,k])) | ||
| for rows, cols in equation.block_sizes_to_indexes((a.shape[0], a.shape[1]-k-1), block_sizes): | ||
| cols = slice(cols.start+k+1, cols.stop+k+1, cols.step) | ||
| a[rows,cols] = self.add(a[rows,cols], self.mul(a[rows,k,None], a[k,cols])) |
There was a problem hiding this comment.
My ulimit -v is 64GB but I had to lower max_cpu_bytes to 4GB in order for this self.mul(a[rows,k,None], a[k,cols]) to not run out of memory. I guess it was because there were other things in memory already. It would be nice for get_available_bytes to take current memory use into account (bdusell/semiring-einsum#37).
2 participants
The default was AutomaticBlockSize(max_cpu_bytes = 1 << 30), which means to use up to 1GB of memory (unless any block size would use >1GB). Setting
bin/sum_product.py -B 64makes a difference for parsing long strings.