N3LO fits with IHOU by giacomomagni · Pull Request #1698 · NNPDF/nnpdf

giacomomagni · 2023-03-21T12:10:39Z

This PR is to introduce IHOU + MHOU theory covmat and allow for N3LO theories.

This reverts commit 1ce47f5.

giacomomagni · 2023-11-08T12:26:21Z

@giacomomagni please remove the changes you did to filters.yaml as they are now in #1835

Thanks, done!

RoyStegeman · 2023-11-08T13:40:39Z

With #1835 approved, is this also ready to be merged (barring the updating of the theoryids)?

Co-authored-by: Roy Stegeman <roystegeman@live.nl>

giacomomagni · 2023-11-08T13:56:01Z

With #1835 approved, is this also ready to be merged (barring the updating of the theoryids)?

Maybe before merging it would be good is someone else try to do one N3LO fit, so we are sure the machinery works also for others...

RoyStegeman · 2023-11-08T14:06:24Z

Are the theories those corresponding as this PR is right now? And are they on the server?

giacomomagni · 2023-11-08T15:56:26Z

Are the theories those corresponding as this PR is right now? And are they on the server?

They should

andreab1997 · 2023-11-10T09:55:43Z

@giacomomagni can you post here the runcard you want someone to try before merging this?

giacomomagni · 2023-11-10T10:12:06Z

you can try:

231106-gm-n3lo-global-th-3pt:

#
# Configuration file for n3fit
#
###################################################################
description: NNPDF4.0 NNLO thcovmat fit with theory 439 (nFONLL), global, 3pt N3LO missing theory covmat 
############################################################
# frac: training fraction
# ewk: apply ewk k-factors
# sys: systematics treatment (see systypes)
dataset_inputs:
- {dataset: NMCPD_dw_ite, frac: 0.75}
- {dataset: NMC, frac: 0.75}
- {dataset: SLACP_dwsh, frac: 0.75}
- {dataset: SLACD_dw_ite, frac: 0.75}
- {dataset: BCDMSP_dwsh, frac: 0.75}
- {dataset: BCDMSD_dw_ite, frac: 0.75}
- {dataset: CHORUSNUPb_dw_ite, frac: 0.75}
- {dataset: CHORUSNBPb_dw_ite, frac: 0.75}
- {dataset: NTVNUDMNFe_dw_ite, cfac: [MAS], frac: 0.75}
- {dataset: NTVNBDMNFe_dw_ite, cfac: [MAS], frac: 0.75}
- {dataset: HERACOMBNCEM, frac: 0.75}
- {dataset: HERACOMBNCEP460, frac: 0.75}
- {dataset: HERACOMBNCEP575, frac: 0.75}
- {dataset: HERACOMBNCEP820, frac: 0.75}
- {dataset: HERACOMBNCEP920, frac: 0.75}
- {dataset: HERACOMBCCEM, frac: 0.75}
- {dataset: HERACOMBCCEP, frac: 0.75}
- {dataset: HERACOMB_SIGMARED_C, frac: 0.75}
- {dataset: HERACOMB_SIGMARED_B, frac: 0.75}
- {dataset: DYE886R_dw_ite, frac: 0.75}
- {dataset: DYE886P, frac: 0.75, cfac: []}
#- {dataset: DYE605_dw_ite, frac: 0.75, cfac: []}
#- {dataset: DYE906R_dw_ite, frac: 0.75, cfac: [ACC]}
- {dataset: CDFZRAP_NEW, frac: 0.75, cfac: []}
- {dataset: D0ZRAP_40, frac: 0.75, cfac: []}
- {dataset: D0WMASY, frac: 0.75, cfac: []}
- {dataset: ATLASWRAP36PB, frac: 0.75, cfac: []}
- {dataset: ATLASZRAP36PB, frac: 0.75, cfac: []}
- {dataset: ATLASZHIGHMASS49FB, frac: 0.75, cfac: []}
- {dataset: ATLASLOMASSDY11EXT, frac: 0.75, cfac: []}
- {dataset: ATLASWRAP11CC, frac: 0.75, cfac: []}
- {dataset: ATLASZRAP11CC, frac: 0.75, cfac: []}
- {dataset: ATLASWZRAP11CF, frac: 0.75, cfac: []}
- {dataset: ATLASDY2D8TEV, frac: 0.75, cfac: []}
- {dataset: ATLAS_DY_2D_8TEV_LOWMASS, frac: 0.75, cfac: []}
- {dataset: ATLAS_Z_TOT_13TEV, frac: 0.75, cfac: [NRM,]}
- {dataset: ATLAS_W_TOT_13TEV, frac: 0.75, cfac: [NRM,]}
- {dataset: ATLAS_WP_JET_8TEV_PT, frac: 0.75, cfac: []}
- {dataset: ATLAS_WM_JET_8TEV_PT, frac: 0.75, cfac: []}
#- {dataset: ATLAS_WCHARM_WP_DIFF_7TEV, frac: 0.75, cfac: []}
#- {dataset: ATLAS_WCHARM_WM_DIFF_7TEV, frac: 0.75, cfac: []}
- {dataset: ATLASZPT8TEVMDIST, frac: 0.75, cfac: []}
- {dataset: ATLASZPT8TEVYDIST, frac: 0.75, cfac: []}
- {dataset: ATLASTTBARTOT7TEV, frac: 0.75, cfac: []}
- {dataset: ATLASTTBARTOT8TEV, frac: 0.75, cfac: []}
- {dataset: ATLAS_TTBARTOT_13TEV_FULLLUMI, frac: 0.75, cfac: []}
- {dataset: ATLAS_TTB_DIFF_8TEV_LJ_TRAPNORM, frac: 0.75, cfac: []}
- {dataset: ATLAS_TTB_DIFF_8TEV_LJ_TTRAPNORM, frac: 0.75, cfac: []}
- {dataset: ATLAS_TOPDIFF_DILEPT_8TEV_TTRAPNORM, frac: 0.75, cfac: []}
- {dataset: ATLAS_1JET_8TEV_R06_DEC, frac: 0.75, cfac: []}
- {dataset: ATLAS_2JET_7TEV_R06, frac: 0.75, cfac: []}
- {dataset: ATLASPHT15_SF, frac: 0.75, cfac: [EWK]}
- {dataset: ATLAS_SINGLETOP_TCH_R_7TEV, frac: 0.75, cfac: []}
- {dataset: ATLAS_SINGLETOP_TCH_R_13TEV, frac: 0.75, cfac: []}
- {dataset: ATLAS_SINGLETOP_TCH_DIFF_7TEV_T_RAP_NORM, frac: 0.75, cfac: []}
- {dataset: ATLAS_SINGLETOP_TCH_DIFF_7TEV_TBAR_RAP_NORM, frac: 0.75, cfac: []}
- {dataset: ATLAS_SINGLETOP_TCH_DIFF_8TEV_T_RAP_NORM, frac: 0.75, cfac: []}
- {dataset: ATLAS_SINGLETOP_TCH_DIFF_8TEV_TBAR_RAP_NORM, frac: 0.75, cfac: []}
- {dataset: CMSWEASY840PB, frac: 0.75, cfac: []}
- {dataset: CMSWMASY47FB, frac: 0.75, cfac: []}
- {dataset: CMSDY2D11, frac: 0.75, cfac: []}
- {dataset: CMSWMU8TEV, frac: 0.75, cfac: []}
- {dataset: CMSZDIFF12, frac: 0.75, cfac: [NRM]}
- {dataset: CMS_2JET_7TEV, frac: 0.75, cfac: []}
- {dataset: CMS_1JET_8TEV, frac: 0.75, cfac: []}
- {dataset: CMSTTBARTOT7TEV, frac: 0.75, cfac: []}
- {dataset: CMSTTBARTOT8TEV, frac: 0.75, cfac: []}
- {dataset: CMSTTBARTOT13TEV, frac: 0.75, cfac: []}
- {dataset: CMSTOPDIFF8TEVTTRAPNORM, frac: 0.75, cfac: []}
- {dataset: CMSTTBARTOT5TEV, frac: 0.75, cfac: []}
- {dataset: CMS_TTBAR_2D_DIFF_MTT_TRAP_NORM, frac: 0.75, cfac: []}
- {dataset: CMS_TTB_DIFF_13TEV_2016_2L_TRAP, frac: 0.75, cfac: []}
- {dataset: CMS_TTB_DIFF_13TEV_2016_LJ_TRAP, frac: 0.75, cfac: []}
- {dataset: CMS_SINGLETOP_TCH_TOT_7TEV, frac: 0.75, cfac: []}
- {dataset: CMS_SINGLETOP_TCH_R_8TEV, frac: 0.75, cfac: []}
- {dataset: CMS_SINGLETOP_TCH_R_13TEV, frac: 0.75, cfac: []}
#- {dataset: CMSWCHARMTOT, frac: 0.75}
#- {dataset: CMSWCHARMRAT, frac: 0.75}
#- {dataset: CMS_WCHARM_DIFF_UNNORM_13TEV, frac: 0.75}
- {dataset: LHCBZ940PB, frac: 0.75, cfac: []}
- {dataset: LHCBZEE2FB_40, frac: 0.75, cfac: []}
- {dataset: LHCBWMU7TEV, frac: 0.75, cfac: [NRM,]}
- {dataset: LHCBZMU7TEV, frac: 0.75, cfac: [NRM,]}
- {dataset: LHCBWMU8TEV, frac: 0.75, cfac: [NRM,]}
- {dataset: LHCBZMU8TEV, frac: 0.75, cfac: [NRM,]}
- {dataset: LHCB_Z_13TEV_DIMUON, frac: 0.75, cfac: []}
- {dataset: LHCB_Z_13TEV_DIELECTRON, frac: 0.75, cfac: []}

############################################################
datacuts:  
  t0pdfset: 210715-n3fit-1000-001     # PDF set to generate t0 covmat
  q2min: 13.96                       # Q2 minimum
  w2min: 12.5
  maxTau: 0.020
  use_cuts: internal
############################################################
theory:
  theoryid: 439
theorycovmatconfig:
  point_prescription: "n3lo 3pt hadronic"
  theoryids:
    from_: scale_variation_theories
  pdf: 210715-n3fit-1000-001
  use_thcovmat_in_fitting: true
  use_thcovmat_in_sampling: true
sampling:
  separate_multiplicative: false
     # database id
############################################################
trvlseed: 376191634
nnseed: 2080989803
mcseed: 75955222
save: false
genrep: true      # true = generate MC replicas, false = use real data

parameters: # This defines the parameter dictionary that is passed to the Model Trainer
  nodes_per_layer: [25, 20, 8]
  activation_per_layer: [tanh, tanh, linear]
  initializer: glorot_normal
  optimizer:
    clipnorm: 6.073e-6
    learning_rate: 2.621e-3
    optimizer_name: Nadam
  epochs: 17000
  positivity:
    initial: 184.8
    multiplier:
  integrability:
    initial: 10
    multiplier:
  stopping_patience: 0.1
  layer_type: dense
  dropout: 0.0
  threshold_chi2: 3.5

fitting:
  fitbasis: EVOL  # EVOL (7), EVOLQED (8), etc.
  basis:
  - {fl: sng, trainable: false, smallx: [1.091, 1.119], largex: [1.471, 3.021]}
  - {fl: g, trainable: false, smallx: [0.7795, 1.095], largex: [2.742, 5.547]}
  - {fl: v, trainable: false, smallx: [0.472, 0.7576], largex: [1.571, 3.559]}
  - {fl: v3, trainable: false, smallx: [0.07483, 0.4501], largex: [1.714, 3.467]}
  - {fl: v8, trainable: false, smallx: [0.5731, 0.779], largex: [1.555, 3.465]}
  - {fl: t3, trainable: false, smallx: [-0.5498, 1.0], largex: [1.778, 3.5]}
  - {fl: t8, trainable: false, smallx: [0.5469, 0.857], largex: [1.555, 3.391]}
  - {fl: t15, trainable: false, smallx: [1.081, 1.142], largex: [1.491, 3.092]}

############################################################
positivity:
  posdatasets:
  - {dataset: POSF2U, maxlambda: 1e6}        # Positivity Lagrange Multiplier
  - {dataset: POSF2DW, maxlambda: 1e6}
  - {dataset: POSF2S, maxlambda: 1e6}
  - {dataset: POSFLL, maxlambda: 1e6}
  - {dataset: POSDYU, maxlambda: 1e10}
  - {dataset: POSDYD, maxlambda: 1e10}
  - {dataset: POSDYS, maxlambda: 1e10}
  - {dataset: POSF2C, maxlambda: 1e6}
  - {dataset: POSXUQ, maxlambda: 1e6}        # Positivity of MSbar PDFs
  - {dataset: POSXUB, maxlambda: 1e6}
  - {dataset: POSXDQ, maxlambda: 1e6}
  - {dataset: POSXDB, maxlambda: 1e6}
  - {dataset: POSXSQ, maxlambda: 1e6}
  - {dataset: POSXSB, maxlambda: 1e6}
  - {dataset: POSXGL, maxlambda: 1e6}

############################################################
integrability:
  integdatasets:
  - {dataset: INTEGXT8, maxlambda: 1e2}
  - {dataset: INTEGXT3, maxlambda: 1e2}

############################################################
debug: false
maxcores: 8

andreab1997 · 2023-11-14T09:40:54Z

What is missing here?

RoyStegeman · 2023-11-15T15:44:19Z

Maybe from 784 onward the "Comments" key in the theory.db should be updated?

andreab1997

Just change the comments in the theory.db and I am happy to merge this

andreab1997 · 2023-11-15T16:02:10Z

Ok good we can merge this

giacomomagni added 20 commits December 13, 2022 10:57

Merge branch 'master' into n3lo_dis_fit

9e270f9

update evolve_n3fit eko 0.11 compatibility

1ce47f5

Revert "update evolve_n3fit eko 0.11 compatibility"

50e0da1

This reverts commit 1ce47f5.

implement x > xmin cut

8cb3d83

fix stored xgrid

98fe8be

add n3lo_ad_var to sv list

8f63b4f

add prescription for N3LO ad variations

7374c06

Merge branch 'master' into n3lo_dis_fit

d82c473

Merge branch 'compound_cf' into n3lo_dis_fit

a3ad7fe

Merge branch 'master' into n3lo_dis_fit

9220706

fix new n3lo ad variations

ff51ef3

more fixes for new N3LO ad variations

0b70010

Merge branch 'compound_cf' into n3lo_dis_fit

ae0273c

more updates on N3LO ad variations

0c83376

Merge branch 'compound_cf' into n3lo_dis_fit

4932808

Merge branch 'master' into n3lo_dis_fit

c9ee6d5

Merge branch 'evolfit_w_eko_0.12' into n3lo_dis_fit

2d84c34

Merge branch 'evolfit_w_eko_0.12' into n3lo_dis_fit

81ff83a

Merge branch 'master' into n3lo_dis_fit

3a7325a

revert wrong fix

3a45831

giacomomagni self-assigned this Mar 21, 2023

giacomomagni marked this pull request as draft March 21, 2023 12:10

giacomomagni added 8 commits May 5, 2023 00:34

Merge branch 'master' into n3lo_dis_fit

355bce2

update n3lo full covmat presctiption

9223d1d

allow for ad or mhou covamat only

f91b05b

allow for ihou covmat

e86b455

Merge branch 'master' into n3lo_dis_fit

858f2a3

Merge branch 'master' into n3lo_dis_fit

ed2b022

Merge branch 'master' into n3lo_dis_fit

ccdbefa

make point prescriptions more uniform

8cef18a

Merge remote-tracking branch 'origin/cuts_for_mhou' into n3lo_dis_fit

a81e410

giacomomagni changed the base branch from master to cuts_for_mhou November 8, 2023 12:25

RoyStegeman reviewed Nov 8, 2023

View reviewed changes

Comment thread validphys2/src/validphys/cuts/defaults.yaml Outdated

Update validphys2/src/validphys/cuts/defaults.yaml

7555344

Co-authored-by: Roy Stegeman <roystegeman@live.nl>

giacomomagni added 2 commits November 9, 2023 14:15

remove xmin cut as suggested by @andreab1997

cdf3af0

remove xmin cut as suggested by @andreab1997

64d7919

Base automatically changed from cuts_for_mhou to master November 9, 2023 16:49

Merge branch 'master' into n3lo_dis_fit

d75cc0b

giacomomagni and others added 5 commits November 14, 2023 16:45

Merge branch 'master' into n3lo_dis_fit

006e407

Merge branch 'master' into n3lo_dis_fit

194264a

move n3lo theories 439->722 in theoryDB

007198a

put theories 792 and 793 in correct place in theoryDB

6610501

update n3lo theoryids e.g. 439->722 in

1d0be97

RoyStegeman requested a review from andreab1997 November 15, 2023 15:41

update description of theories 784 to 797

c901882

andreab1997 approved these changes Nov 15, 2023

View reviewed changes

update description of theories 784 to 797

9c9c88d

RoyStegeman merged commit 7592b58 into master Nov 15, 2023

RoyStegeman deleted the n3lo_dis_fit branch November 15, 2023 16:49

RoyStegeman mentioned this pull request Nov 15, 2023

run black isort on recently edited files #1848

Merged

Navigation Menu

Uh oh!

N3LO fits with IHOU - #1698

N3LO fits with IHOU#1698
RoyStegeman merged 72 commits into
masterfrom
n3lo_dis_fit

giacomomagni commented Mar 21, 2023 •

edited

Loading

Uh oh!

giacomomagni commented Nov 8, 2023

Uh oh!

Uh oh!

RoyStegeman commented Nov 8, 2023

Uh oh!

giacomomagni commented Nov 8, 2023

Uh oh!

RoyStegeman commented Nov 8, 2023

Uh oh!

giacomomagni commented Nov 8, 2023

Uh oh!

andreab1997 commented Nov 10, 2023

Uh oh!

giacomomagni commented Nov 10, 2023 •

edited

Loading

Uh oh!

andreab1997 commented Nov 14, 2023

Uh oh!

RoyStegeman commented Nov 15, 2023 •

edited

Loading

Uh oh!

andreab1997 left a comment

Uh oh!

andreab1997 commented Nov 15, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

giacomomagni commented Mar 21, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

giacomomagni commented Nov 8, 2023

Uh oh!

Uh oh!

RoyStegeman commented Nov 8, 2023

Uh oh!

giacomomagni commented Nov 8, 2023

Uh oh!

RoyStegeman commented Nov 8, 2023

Uh oh!

giacomomagni commented Nov 8, 2023

Uh oh!

andreab1997 commented Nov 10, 2023

Uh oh!

giacomomagni commented Nov 10, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

andreab1997 commented Nov 14, 2023

Uh oh!

RoyStegeman commented Nov 15, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

andreab1997 left a comment

Choose a reason for hiding this comment

Uh oh!

andreab1997 commented Nov 15, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

giacomomagni commented Mar 21, 2023 •

edited

Loading

giacomomagni commented Nov 10, 2023 •

edited

Loading

RoyStegeman commented Nov 15, 2023 •

edited

Loading