Skip to content

Lightning integration #991

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
sarthakpati merged 117 commits into from
May 1, 2025
Merged

Lightning integration #991

sarthakpati merged 117 commits into from
May 1, 2025

Conversation

@szmazurek
Copy link
Collaborator

@szmazurek szmazurek commented Mar 7, 2025

Fixes #

Proposed Changes

  • moving gandlf logic to Lightning framework

Checklist

  • CONTRIBUTING guide has been followed.
  • PR is based on the current GaNDLF master .
  • Non-breaking change (does not break existing functionality): provide as many details as possible for any breaking change.
  • Function/class source code documentation added/updated (ensure typing is used to provide type hints, including and not limited to using Optional if a variable has a pre-defined value).
  • Code has been blacked for style consistency and linting.
  • If applicable, version information has been updated in GANDLF/version.py.
  • If adding a git submodule, add to list of exceptions for black styling in pyproject.toml file.
  • Usage documentation has been updated, if appropriate.
  • Tests added or modified to cover the changes; if coverage is reduced, please give explanation.
  • If customized dependency installation is required (i.e., a separate pip install step is needed for PR to be functional), please ensure it is reflected in all the files that control the CI, namely: python-test.yml, and all docker files [1,2,3].
  • The logging library is being used and no print statements are left.

szmazurek and others added 30 commits December 6, 2024 09:35
@szmazurek
Lightning port stage 1
8ff6cbb 
- basic primitves for lightning port
@szmazurek
Remove redundant todos, change assert to warn
bff4381
@szmazurek
Merge pull request #43 from szmazurek/port_lightning_corrected
90c98cd 
Port lightning corrected
@szmazurek
Merge branch 'mlcommons:master' into master
732d0db
@szmazurek
GANDLF/models/lightning_module.py
4a500ca
@szmazurek
Implementation of forward pass in lightning module
c4fe1c3
@szmazurek
Implementing info outputs
4c79fd2
@szmazurek
Model checkpoint saving WIP
2dacf5e
@szmazurek
Correct loss value retrieval for saving
fb2b2d5
@szmazurek
Early stoppoing & model saving
e7c7ad6
@szmazurek @sarthakpati
Update GANDLF/models/lightning_module.py
a5dc2fc 
Co-authored-by: Sarthak Pati <patis@iu.edu>
@szmazurek @sarthakpati
Update GANDLF/models/lightning_module.py
025b4ce 
Co-authored-by: Sarthak Pati <patis@iu.edu>
@szmazurek @sarthakpati
Update GANDLF/models/lightning_module.py
679c8f1 
Co-authored-by: Sarthak Pati <patis@iu.edu>
@szmazurek @sarthakpati
Update GANDLF/models/lightning_module.py
2f1109c 
Co-authored-by: Sarthak Pati <patis@iu.edu>
@szmazurek @sarthakpati
Update GANDLF/models/lightning_module.py
2c163fa 
Co-authored-by: Sarthak Pati <patis@iu.edu>
@szmazurek @sarthakpati
Update GANDLF/models/lightning_module.py
5d4dd55 
Co-authored-by: Sarthak Pati <patis@iu.edu>
@szmazurek @sarthakpati
Update GANDLF/models/lightning_module.py
27c0b1a 
Co-authored-by: Sarthak Pati <patis@iu.edu>
@szmazurek
Training pass enhancements
1f24054 
- added calculation of overall metrics
- added proper handling of data formats for both progress bar logging
  and gandlf custom logging
- added tests for regression and classification
- little refactoring of tests
@sarthakpati
Update .spelling/.spelling/expect.txt
f8e761d
@szmazurek
Merge branch 'mlcommons:master' into master
a58daec
@szmazurek
Merge pull request #44 from szmazurek/forward_pass_lightning
53add5e 
Forward pass lightning - training pipeline
@szmazurek
Logic for validation pass WIP
cf2ae25
@szmazurek
Implementing validation logic
df6a3e9
@szmazurek
Validation pass for classification/regression
3e51224
@szmazurek
Validation tests debugging WIP
c38bcd4
@szmazurek
Finalizing validation implementation
9efc455
@szmazurek
Added testing step logic
eadb654
@szmazurek
Progbar fixes WIP
b72476d
@szmazurek
2D cases validation/regression working
d2c8ea4
@szmazurek
Added 3D rad tests for the existing pipelines
801ffae
@szmazurek szmazurek requested a review from a team as a code owner March 7, 2025 10:37
@github-actions
Copy link
Contributor

github-actions bot commented Mar 7, 2025 •
edited
Loading

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅

@codecov

This comment was marked as outdated.

Sign in to view
@sarthakpati sarthakpati marked this pull request as draft March 7, 2025 15:25
szmazurek and others added 5 commits March 7, 2025 17:12
@szmazurek
workflow debugging
e9b47cc
@sarthakpati
Merge branch 'master' into lightning_integration
33dad60
@szmazurek
Removing OOM error on regression 3D test
a1ec623 
No idea where it came from, had to just downsize the patch size. It is
caused during evaluation of efficient net, and not always
@szmazurek
Correct codacy errors
6e58d4e
@szmazurek
correcting codacy scan errors
f22a431
@szmazurek
Copy link
Collaborator Author

szmazurek commented Apr 7, 2025

@benmalef can you help us with the code covergae? I think it is mostly related to the files that are no logner used and some lines in new code. Can you take a look?

@benmalef
Copy link
Contributor

benmalef commented Apr 8, 2025

@benmalef can you help us with the code covergae? I think it is mostly related to the files that are no logner used and some lines in new code. Can you take a look?

Yes, of course. I will take a look. 😁

sarthakpati
sarthakpati reviewed May 1, 2025
View reviewed changes
@sarthakpati sarthakpati marked this pull request as ready for review May 1, 2025 14:09
@sarthakpati sarthakpati merged commit 759d28e into master May 1, 2025
32 of 33 checks passed
@sarthakpati sarthakpati deleted the lightning_integration branch May 1, 2025 14:10
@github-actions github-actions bot locked and limited conversation to collaborators May 1, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

@sarthakpati sarthakpati sarthakpati approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@szmazurek @benmalef @sarthakpati @scap3yvt