Skip to content

improve Dataset.create_torch_dataset and StandandLearnedReconstructor, enable more pickling - #30

Merged
jleuschn merged 4 commits into
masterfrom
dev
Feb 4, 2021
Merged

improve Dataset.create_torch_dataset and StandandLearnedReconstructor, enable more pickling#30
jleuschn merged 4 commits into
masterfrom
dev

Conversation

@jleuschn

@jleuschn jleuschn commented Feb 4, 2021

Copy link
Copy Markdown
Owner
  • fix CachedDataset for the case of all parts being cached. It supports random access then, but previously it was broken due to still calling self.dataset.get_sample[s] with the out parameter being all False.
  • improve Dataset.create_torch_dataset
    • add transform parameter
    • move the previously local torch dataset class definitions to dival.util.torch_utility
    • use torch.utils.data.IterableDataset (instead of using torch.utils.data.Dataset and ignoring the index) for generator-based datasets
    • fix docs
  • make 'ellipses' standard dataset pickleable and add fixed_noise_seeds parameter (still True by default, in contrast to noise_seeds (used for the ground truth image generation), which is False by default. Both (different) defaults are kept like they were).
  • fix and improve evaluation.run implementation by avoiding to save reconstructions when run with save_reconstructions=False, reducing RAM usage
  • add example scripts
  • add option to allow multiple workers with generator-based datasets in FBPUNetReconstructor, remove warning about shuffling not working for generator-based datasets (for shuffle=True, it will raise an error due to Dataset.create_torch_dataset now returning a torch.utils.data.IterableDataset, and for shuffle='auto' the docs for StandardLearnedReconstructor describe that shuffling is not used in this case)
  • improve StandardLearnedReconstructor
    • add possibility for a transform (data augmentation) via implementing init_transform in a subclass
  • add shuffle option, defaulting to 'auto' which depends on whether the dataset supports random access. Previously shuffle=True was always used, but for generator-based datasets the index was ignored, effectively disabling shuffling.
  • add dival.util.odl_utility.patch_ray_trafo_for_pickling

@jleuschn
jleuschn merged commit f996de3 into master Feb 4, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant