Hi, I try to take a deep understanding of slot mapping and thanks for your implemention as a good refernce.
In the code, the slot mapping is inited as
slot_mappings = [torch.arange(seq_len, dtype=torch.long) + block * self.block_size for block in allocated]
my question is that if the seq_len larger than block size, maybe overflow would happen?
A short demo is as following
`
import torch
seq_len = 16 # Length of each sequence block
block_size = 8 # Size of each block
allocated = [0, 1, 2] # Indices of allocated blocks
self = type('', (), {})() # Create an empty object
self.block_size = block_size
slot_mappings = [torch.arange(seq_len, dtype=torch.long) + block * self.block_size for block in allocated]
for i, mapping in enumerate(slot_mappings):
print(f"Block {allocated[i]}: {mapping}")
`
The rsesult is
Block 0: tensor([ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15])
Block 1: tensor([ 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23])
Block 2: tensor([16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31])
Hi, I try to take a deep understanding of slot mapping and thanks for your implemention as a good refernce.
In the code, the slot mapping is inited as
slot_mappings = [torch.arange(seq_len, dtype=torch.long) + block * self.block_size for block in allocated]my question is that if the seq_len larger than block size, maybe overflow would happen?
A short demo is as following
`
import torch
seq_len = 16 # Length of each sequence block
block_size = 8 # Size of each block
allocated = [0, 1, 2] # Indices of allocated blocks
self = type('', (), {})() # Create an empty object
self.block_size = block_size
slot_mappings = [torch.arange(seq_len, dtype=torch.long) + block * self.block_size for block in allocated]
for i, mapping in enumerate(slot_mappings):
print(f"Block {allocated[i]}: {mapping}")
`
The rsesult is
Block 0: tensor([ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15])
Block 1: tensor([ 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23])
Block 2: tensor([16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31])