Tags: skhynix/hmsdk
Tags
Release hmsdk v4.0
This is a new release for hmsdk v4.0, which supports enhanced bandwidth
and capacity expansion as well as bwprof, which is a bandwidth profiling
tool.
The changes from hmsdk v3.0 to v4.0 are as follows.
1. Bandwidth expansion enhancement
- support multi-socket weighted interleave
- with socket-aware weighted interleave
- with this feature, users don't have to use "-N/--cpunodebind nodes"
option to numactl to avoid UPI overhead between sockets
- instead, "numactl -w all" is all you need now
- support weighted interleave APIs in libnuma
- the following APIs for weighted interleave is added to libnuma
- numa_alloc_weighted_interleaved_subset()
- numa_alloc_weighted_interleaved()
- numa_weighted_interleave_memory()
- hmctl supports -w/--weighted-interleave option
- to support object level weighted interleave (e.g. malloc level)
- libnuma APIs are page level weighted interleave (e.g. mmap level)
2. Capacity expansion enhancement
- support multi-threaded monitoring and migration
- to enhance monitoring accuracy and migration speed
- by multiple kdamonds covering split ranges of whole address
3. bwprof tool for system memory profiling
- system-wide bandwidth profiling tool for both DRAM and CXL memory
- it supports top, record, report, dump, and info commands for monitor,
record and analyze the result offline
4. notable submodule and tools update
- linux kernel: upgrade from v6.12 to v6.17
- with socket-aware weighted interleave feature on top of v6.17
- damo: upgrade from 2.4.7 to 3.0.0
- numactl: upgrade to 548e9ea for new libnuma APIs
- move damo and numactl submodules into tools directory
- mlc: upgrade from v3.11 to v3.12
- remove tools/gen_config.py symbolic link to tools/gen_migpol.py
Signed-off-by: Honggyu Kim <honggyu.kim@sk.com>
Signed-off-by: Rakie Kim <rakie.kim@sk.com>
Signed-off-by: Yunjeong Mun <yunjeong.mun@sk.com>
Release hmsdk v3.0
This is a new release for hmsdk v3.0, which is fully aligned its
features with various other open source projects including Linux kernel,
numactl, damo, etc.
The changes from hmsdk v2.0 to v3.0 are as follows.
- Apply interface changes of weighted interleaving from upstream
- drop task level weight setting
- drop set_mempolicy_node_weight and mrange_node_weight syscalls
- drop local changes of numactl, then use upstream numactl with
-w/--weighted-interleave option.
- thanks to Gregory Price (formerly at MemVerge)
- Apply interface changes of DAMON based tiered memory management
- DAMOS_{PROMOTE,DEMOTE} are renamed to DAMOS_MIGRATE_{HOT,COLD}
- set the migration target with 'target_nid' in sysfs instead of use
memory tiering info builtin kernel.
- rename gen_config.py to gen_migpol.py for clear meaning
- change --{dram,cxl}_nodes options of gen_config.py to
--{demote,promote} options in gen_migpol.py along with its usage.
- drop local changes of damo, then use upstream damo as all of hmsdk's
features are upstreamed.
- use 'target_nid' in migrate_{hot,cold} actions
- use json format output to yaml format
- Replace cemalloc to hmalloc
- reimplented hmalloc from scratch for cemalloc replacement.
- cemalloc supports many features but only explicit mode is
implemented in hmalloc after renaming of 'cxl_' prefixed APIs to
'h' prefixed APIs. For example, cxl_malloc is renamed to hmalloc.
- drop unused features to reduce complexity.
- such as implicit mode and Java/Python APIs
- drop jemalloc submodule as hmalloc can use it from official packages
without recompilation.
- introduce hmctl tool to avoid environment variable exposure.
Since hmsdk v3.0 is fully aligned with various open source projects, its
usage and interface won't be changed any time soon and it will no longer
require back porting its local features to the upgraded versions.
Signed-off-by: Honggyu Kim <honggyu.kim@sk.com>
Signed-off-by: Yunjeong Mun <yunjeong.mun@sk.com>
Signed-off-by: Hyeongtak Ji <hyeongtak.ji@sk.com>
Signed-off-by: Rakie Kim <rakie.kim@sk.com>
Release hmsdk v2.0 This is a new release hmsdk v2.0 for 2-tier memory management. Along with the existing bandwidth-aware interleaving in hmsdk-v1.1, this includes another way to manage 2-tier memory by monitoring memory access patterns and migrating pages depending on their access temperature. The 2-tier memory management is supported via DAMON(Data Access MONitor) framework and its DAMON-based Operation Schemes, called DAMOS. HMSDK 2.0 supports two new DAMOS actions; demotion from fast tiers and promotion from slow tiers. This prevents hot pages from being stuck on slow tiers, which makes performance degradation and cold pages can be proactively demoted to slow tiers so that the system can increase the chance to allocate more hot pages to fast tiers. The damo user space tool helps enabling DAMON and applying those new DAMOS actions based on the given json config file generated by tools/gen_config.py. This can be especially useful when the 2-tier memory system has high memory pressure on its first tier DRAM NUMA nodes by efficiently utilizing second tier CXL NUMA nodes. Signed-off-by: Honggyu Kim <honggyu.kim@sk.com> Signed-off-by: Hyeongtak Ji <hyeongtak.ji@sk.com> Signed-off-by: Rakie Kim <rakie.kim@sk.com>
Add hmsdk v1.1 HMSDK(Heterogeneous Memory Software Development Kit) is designed to support heterogeneous memory, especially CXL memory, which is a new promising memory system based on a CXL(Compute Express Link) open industry standard. The hmsdk consists of multiple git submodules as follows. hmsdk ├── cemalloc │ └── externals │ └── jemalloc ├── linux └── numactl This patch includes the entire project of hmsdk v1.1. Signed-off-by: Rakie Kim <rakie.kim@sk.com> Signed-off-by: Hyeongtak Ji <hyeongtak.ji@sk.com> Signed-off-by: Yunjeong Mun <yunjeong.mun@sk.com> Signed-off-by: Honggyu Kim <honggyu.kim@sk.com>