Skip to content

gmsh2nek: Precision mismatch in write_xyz produces corrupt re2 v003 files for Hex20 meshes #894

@landerhadron

Description

@landerhadron

Summary

The gmsh2nek tool produces corrupt .re2 files when converting Gmsh Hex20 meshes. The root cause is a single/double precision mismatch between the coordinate storage arrays and the re2 v003 binary format.

Affected Code

  • File: 3rd_party/nek5000/tools/gmsh2nek/gmsh2nek.f90
  • Related: 3rd_party/nek5000/tools/gmsh2nek/mod_SIZE.f90

Bug Description

1. Precision mismatch in coordinate arrays

In mod_SIZE.f90 (line 25), the coordinate arrays are declared as single precision:

real,save,allocatable,dimension(:,:,:,:) ::  xm1, ym1, zm1

But the re2 header is written as #v003 format (line 1641 of gmsh2nek.f90), which NekRS expects to contain double-precision (real*8) coordinate data.

2. Corrupt byte layout in write_xyz

In write_xyz (lines 1648-1708):

real     xx(8), yy(8), zz(8)       ! single precision (4 bytes each)
real*8   buf2(30)                   ! double precision (8 bytes each)
...
call copy(buf2(1), xx, 8)          ! copies 8 × 4 = 32 bytes into buf2
call byte_write(buf2(1), 16, ierr) ! writes 16 × 4 = 64 bytes from buf2

The copy subroutine (from nek5000 core) operates on real*4. It copies 32 bytes of valid x-coordinates into the start of buf2. Then byte_write writes 64 bytes where only the first 32 are valid. The remaining 32 bytes are uninitialized.

When NekRS reads the re2 v003 file, it interprets each 8-byte block as a real*8. This means:

  • Corners 0–3: each real*8 value is formed from two adjacent real*4 values concatenated bit-by-bit, leading to error.
  • Corners 4–7: read from the uninitialized region

3. Observed Impact

For a 201,600-element Hex20 mesh:

  • 1,335,332 degenerate corner pairs detected in the output re2
  • NekRS fails during connectivity calculation with: element_check failed in parRSB
  • The error is reported as: ERROR: Connectivity calculation failed! Try tightening mesh::connectivityTol
  • Tightening connectivityTol does not help because the coordinates themselves are corrupt

Proposed Fix

Change mod_SIZE.f90 line 32 to use double precision:

real*8,save,allocatable,dimension(:,:,:,:) ::  xm1, ym1, zm1

And change write_xyz to use real*8 for the local coordinate arrays:

real*8   xx(8), yy(8), zz(8)

This ensures the copy into buf2 and byte_write operate on consistent 8-byte values matching the #v003 format.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions