Hi!

I am facing a problem in maintaining the reproducibility in the ML project. I believe the core snippet of my issue is

clf = Clf(random_state=cfg.seed)
    # instantiate the K-fold cross-validation object
    cv = StratifiedKFold(
        n_splits=cfg.train.n_folds,
        shuffle=False,#cfg.train.is_shuffle,
        random_state=cfg.seed
    )
    # instantiate the grid search object
    grid_search = TSCGridSearchCV(
        estimator=clf,
        param_grid=dict(cfg.train.grid_params),
        cv=cv,
        n_jobs=cfg.train.n_jobs,
        scoring=cfg.train.scoring,
        verbose=4-(logger.level(cfg.log.cli.level).no//10),
    )

Where Clf is CNNClassifier. When cfg.train.n_jobs=1, I am managing to reproduce the same results across the runs, although it is ridiculously time-consuming. For instance, the learning curve of the best fitted model for three independent runs when n_jobs=1 is:

However, when n_jobs>=2 , the training phase performs differently across runs, even though the parameter set is kept the same (previous plot with two more runs where n_jobos=4):

Making runs reproducible is one of the project requirements, I need that. How to ensure reproducibility?