The wrong library name is configured in fmt.pc file when with CMAKE_BUILD_TYPE=Debug.

As a result

$pkg-config --libs fmt

Returns -lfmt however the library name is actually "fmtd". The solution is to configure fmt.pc via variable. For example
...
prefix=@CMAKE_INSTALL_PREFIX@
exec_prefix=@CMAKE_INSTALL_PREFIX@
libdir=@CMAKE_INSTALL_FULL_LIBDIR@
includedir=@CMAKE_INSTALL_FULL_INCLUDEDIR@
library=@FMT_LIBRARY_NAME@

Name: fmt
Description: A modern formatting library
Version: @FMT_VERSION@
Libs: -L${libdir} -l${library}
Cflags: -I${includedir}

Alternately change:

CMakeLists.txt:183

set_target_properties(fmt PROPERTIES
VERSION ${FMT_VERSION} SOVERSION ${CPACK_PACKAGE_VERSION_MAJOR}
DEBUG_POSTFIX d)

It seems unnecessary to have different names for library builds?