IEEE COMPUTER ARCHITECTURE LETTERS, VOL. 22, NO.
1, JANUARY-JUNE 2023 25
Intelligent SSD Firmware for Zero-Overhead Journaling
Hanyeoreum Bae , Donghyun Gouk , Seungjun Lee, Jiseon Kim, Sungjoon Koh , Jie Zhang ,
and Myoungsoo Jung
Abstract—We propose Check0-SSD, an intelligent SSD firmware to offer
the best system-level fault-tolerance without performance degradation and
lifetime shortening. Specifically, the SSD firmware autonomously removes
transaction checkpointing, which eliminates redundant writes to the flash
backend. To this end, Check0-SSD dynamically classifies journal descrip-
tor/commit requests at runtime and switches the address spaces between
journal and data regions by examining the host’s filesystem layout and
journal region information in a self-governing manner. Our evaluations
demonstrate that Check0-SSD can protect both data and metadata with Fig. 1. Performance comparison of journal modes.
89% enhanced storage lifetime while exhibiting similar or even better
performance compared to the norecovery SSD. on-the-fly transactions, the host-side filesystem first flushes all cached
Index Terms—Computational SSD, remapping, transaction data, and then, commits the data, which introduces transaction check-
management. pointing. It is a well-known issue that the checkpointing eventually
I. INTRODUCTION doubles the number of write requests, thereby degrading performance
seriously [4]. To address this penalty, WAL-SSD [5] proposed a journal
Recently, high-performance SSDs have been widely used for a remapping scheme. This study introduces a host-side module that keeps
variety of data-intensive applications and analytics. To meet the level recording journaling information through a redirection table and sends
of enterprise-scale requirements, the industry employs a high-speed address redirection information to the firmware of the underlying stor-
interface such as NVMe [1] and a new storage technology that exhibits age, via customized NVMe commands (tmap, remap). The firmware
ultra-low latency (ULL). However, the conventional storage stack is then remaps the address translation information based on the delivered
agnostic to such a high-speed interface and storage technology, and address mapping list. However, this approach requires heavy storage
unfortunately, it is difficult to take the full benefits brought by the stack modification of both host-side kernel and flash firmware modules,
enterprise-scale ULL SSDs. To be specific, since the enterprise systems and costs extra communication overheads as many commits occur.
deal with many mission crucial activities, their systems are mostly We propose Check0-SSD, an intelligent SSD firmware to guarantee
designed never to experience service interrupts, thereby requiring ex- the best system-level fault tolerance (by journaling both data and
cellent fault-tolerant storage. However, transaction management for metadata) without major performance degradation and lifetime short-
enterprise-scale SSDs resides on the critical path of the storage accesses. ening, compared to a norecovery SSD, in a self-governing manner. It
In particular, we observe that a fault tolerance mechanism (journals both autonomously removes checkpointing activities on journaling storage
data and metadata) significantly degrades the performance and shortens while requiring neither host-side modification nor hardware interface
the lifetime of an enterprise-scale SSD [2]. changes. Specifically, Check0-SSD dynamically classifies journal de-
We conduct a set of experiments to quantitatively analyze how the scriptor/commit requests at runtime by detecting the filesystem layout
transaction management can affect the performance and lifetime (write and journal region information. When a transaction commit is detected,
amplification factor, WAF [3]) of the enterprise-scale SSDs. Fig. 1 the SSD controller delays processing the transaction commit and simply
compares bandwidth, latency, and WAF for the journaling SSD and a switches the address space of the journal region to that of the data
norecovery SSD (without any journaling techniques). One can observe region. It then records the commit for the completion of the transaction
from Fig. 1(a) and (b) that the journaling SSD exhibits 3.21× worse such that the host can achieve the same level of fault tolerance as
bandwidth and 3.29× longer latency compared to the norecovery SSD, filesystem-assisted journaling. Check0-SSD not only eliminates all data
which can seriously degrade the overall performance of enterprise migrations for checkpointing completely but also converts all random
computing. While the journaling SSD guarantees the highest level write requests to sequential accesses, which can naturally minimize the
of fault tolerance, it exhibits 3.41× higher WAF, which shortens the number of block I/O structures and NVMe request preparations. Our
storage lifetime by 71%, compared with that of the norecovery SSD. evaluation results demonstrate that Check0-SSD exhibits 47% lower
The root cause of the aforementioned performance degradation write amplification factors (under Linux 4.19) while improving the
and lifetime shortening is checkpointing. Specifically, to complete all bandwidth and latency compared to a journaling SSD by 2.01× and
43%, respectively. The results also show that Check0-SSD offers similar
Manuscript received 23 December 2022; accepted 6 February 2023. Date or even better performance compared to a norecovery SSD even though
of publication 9 February 2023; date of current version 16 March 2023. This it fully journals both data and metadata.
work was mainly supported by Hynix research center (S3RC) and SK-Hynix
under Grant G01200477. This work was also in part supported by NRF under
Grant 2021R1A2C4001773, in part by IITP under Grants 2021-0-00524 & II. BACKGROUND
2022-0-00117, in part by the KAIST start-up package under Grant G01190015,
and in part by the KAIST IDEC. Recommended for acceptance by H. Tseng.
A. System-Level Fault Tolerance Mechanism
(Corresponding authors: Jie Zhang; Myoungsoo Jung.) Journaling and Transaction. A journaling filesystem writes all data
Hanyeoreum Bae, Donghyun Gouk, Seungjun Lee, Jiseon Kim, Sungjoon to a temporal persistent space of the underlying storage, called journal
Koh, and Myoungsoo Jung are with the School of Electrical Engineering, region. Once data is completely written to the storage, the system
Korea Advanced Institute of Science and Technology, Daejeon 34141, Ko-
rea (e-mail: hyr.bae@camelab.org; kukdh1@camelab.org; sjlee@camelab.org;
rewrites each data to the original place, referred to as data region.
wltjs27@camelab.org; skoh@camelab.org; mj@camelab.org). Fig. 2(a) shows the data layout of a journal region. The journal region
Jie Zhang is with the The School of Computer Science, Peking University, consists of multiple data blocks. The system places a special marking
Beijing 100871, China (e-mail: jie@camelab.org). block at the beginning and the end of the data collection to delimit
Digital Object Identifier 10.1109/LCA.2023.3243695 each of data chunks for writing the journal region. The delimiters are
1556-6056 © 2023 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See https://www.ieee.org/publications/rights/index.html for more information.
Authorized licensed use limited to: R V College of Engineering. Downloaded on April 29,2025 at 08:18:43 UTC from IEEE Xplore. Restrictions apply.
26 IEEE COMPUTER ARCHITECTURE LETTERS, VOL. 22, NO. 1, JANUARY-JUNE 2023
Fig. 4. The internal architecture of Check0-SSD.
(Checkpoint), device waiting (Waiting), and transaction data run
Fig. 2. The internal of conventional journaling. and commit (Run+Commit). Since the enterprise-scale SSDs are well
optimized to deliver high performance, Waiting consumes only 3%
of the execution time. However, Checkpoint spends 62% of the total
time, which sits on the critical path of system-level I/O services.
III. ZERO-OVERHEAD JOURNALING
The goal of our Check0-SSD is to achieve the performance same as
Fig. 3. Journal impacts on various SSD types. that of the norecovery SSD meanwhile offering the best system-level
fault tolerance. To this end, Check0-SSD detects the essential infor-
called descriptor and commit blocks. The system considers a sequence mation autonomously and performs checkpointing within the storage
of write operations from the descriptor block to the commit block as a device. Therefore, it allows the host to remove all the checkpointing
transaction, which is an atomic I/O task to support a strong level of fault activities of jbd2 from the storage stack. To be specific, Check0-SSD
tolerance. All these transactions are assigned with unique identifiers, first detects the address range of the journal region, then collects the
called trans-id, each increasing in chronological order. The system address information from incoming write requests. When the host
also employs an independent block at the head of the journal region, initializes checkpointing, Check0-SSD remaps the data from the journal
referred to as superblock, to record a trans-id. When the system writes a region to the data region by only updating the address translation
transaction to the journal region, it inserts a logical block address (LBA) information.
list to the descriptor block for crash recovery. Each entry of the LBA
list points to the data region location where the journaled data should A. High-Level Architecture
be written back. Once the descriptor and data blocks are completely Fig. 4 shows the architecture of our Check0-SSD. The front-end SSD
written to the target SSD, the filesystem writes a commit block to mark controller is an embedded processor, while the backend consists of mul-
that the ongoing transaction is valid. tiple flash memory controllers (FMC) and flash packages. Additionally,
Checkpointing. Fig. 2(b) shows the checkpointing process. Once Check0-SSD implements three new components in the SSD controller:
the transaction is completely journaled (i.e., committed), it invokes i) journal detector, ii) block classifier, and iii) checkpoint flipper. The
a daemon process, referred to as journaling block device (jbd2), to block classifier is implemented along with NVMe command control
conduct the rewriting process, which is also known as checkpointing. logic [1], whereas the journal detector and checkpoint flipper are
To this end, jbd2 employs a committing transaction list, of which each integrated in a flash translation layer (FTL) that maps the logical page
entry contains a journal head (JH). Each JH includes a buffer head (BH) number (LPN) of the writes to the flash physical page number (PPN).
pointing to the target data (written to the journal region before) and Specifically, the journal detector identifies filesystem layout and journal
the original LBA, where jbd2 requires writing the data in the data region information. Based on the information, the block classifier can
region. During checkpointing, jbd2 visits all BHs in the committing dynamically detect transaction-related operations and extract essential
transaction list and writes their data to the underlying storage through information to handle on-the-fly transactions. In particular, it extracts
the NVMe driver. After writing the data, jdb2 marks the completion of the logical address of the journal region where the data are currently
this checkpointing process by recording the trans-id of the checkpointed located (called J-LPN) and the logical address of the data region where
transaction to the superblock. such data should be reclaimed (called D-LPN). The extracted address
Crash Recovery. Since the superblock records the checkpointed list is recorded to the SSD internal memory, called checkpoint address
trans-id, the system scans all transactions in the journal region and table (CAT). Lastly, the checkpoint flipper eliminates the checkpointing
detects the stable transaction whose trans-id is equal to the value activities. Specifically, it collects the mapping information by referring
recorded in the superblock. If there exists a transaction whose trans-id to the CAT. It then remaps data residing in the journal region to the
is greater than the one recorded in the superblock, it means that this target data region by updating address translation table (ATT) in FTL.
transaction fails in checkpointing. The system then checks the commit
block for data integrity. If the commit block is valid, the operating
B. Transaction Management
system starts writing the journaled data to the target data region by
referring to the LBA list stored in the descriptor block. This process, Autonomous Detection. All journaling data coming from the host
also known as replay, recovers system crashes from service interrupts include journal header information (12B, different from JH of jbd2),
or failures. which can be used for detection and classification. However, journal
detection cannot be simply achieved by parsing the journal header
information. This is because incoming requests also represent other
B. Challenge Analysis
types of data blocks, making it unable to extract the header information
To quantitatively analyze the performance degradation of a journal- without an appropriate block identification. To address this, during
ing system (compared to a norecovery system), we perform 4KB-sized the filesystem initialization time, Check0-SSD parses the filesystem
sequential and random writes atop enterprise-scale NVMe [6] and ULL metadata and the journal’s superblock as shown in Fig. 5. Check0-SSD
SSDs [2]. The evaluation results are shown in Fig. 3. The journaling first checks whether the partition type is master boot record (MBR) or
system, on average, degrades the performance of NVMe and ULL GUID partition table (GPT) by checking if there is a GPT signature in
SSDs by 5.86× and 3.19×, respectively, compared to the norecovery LBA 1. The journal detector then checks the partition table by reading
system. The main reason behind this performance slowdown is the the MBR or GPT. Since this partition table includes the filesystem’s
checkpointing activities coming after the transaction commits. To be starting LBA, the journal detector can figure out where the filesystem
precise, we decompose the execution time of sequential and random locates. Check0-SSD then waits for the filesystem initialization by
writes in the journaling system into the latency of checkpointing parsing the incoming NVMe write command heading to the superblock
Authorized licensed use limited to: R V College of Engineering. Downloaded on April 29,2025 at 08:18:43 UTC from IEEE Xplore. Restrictions apply.
IEEE COMPUTER ARCHITECTURE LETTERS, VOL. 22, NO. 1, JANUARY-JUNE 2023 27
TABLE I
WORKLOAD CHARACTERISTICS
Fig. 5. Autonomous journal detection.
ATT entries corresponding to the D-LPNs (associated with J-LPNs)
for checkpointing ( 3 ). Once the updates in ATT are completed, the
checkpoint flipper informs FTL of the ATT updates such that FTL can
make the table persistent (if needed). This flipping operation makes
the transaction checkpointed, and thus, the host can simply omit the
checkpointing activities of jbd2. Note that, our flipping operation
does not introduce any data migration or flash writes as moving the
data from the journal region to the data region is only performed by the
conversion of ATT based on CAT information. Even in cases where a
variation of FTL uses a demand paging for ATT, the corresponding ATT
entries are all loaded when the transaction runs (writing the data). Thus,
the checkpoint flipper does not touch the underlying flash to migrate
transactions from the journal region to the data region.
Note that Check0-SSD can be applied to any filesystems which have
Fig. 6. Block classification and flipped checkpointing.
logging systems. This is because a logging system generally contains
three types of blocks: i) start, ii) end, and iii) data blocks, which are
region of the filesystem (i.e., a block located at the starting LBA). If the the same as the aforementioned descriptor, commit, and data blocks,
magic signature (s_magic) of the superblock corresponds to the target respectively. As the start or end blocks contain the mapping information
filesystem, Check0-SSD concludes that the filesystem is initialized. The between the journal and data regions, our journal detection scheme
detector then examines which inode number points to the journal region can leverage such information to compose CAT, thereby being able to
by reading the superblock. Check0-SSD then jumps to the filesystem’s support filesystems with various layout/transaction protocols.
block group by referring to the detected inode number and parses
the corresponding inode table. The journal detector reads the journal
inode, which stores the root node of the journal’s extent tree. Extent
IV. EVALUATION
tree stores addresses of journal blocks, including the address of the We use a full-system event-driven simulation framework for the
journal’s superblock (ee_start). Therefore, the journal detector can evaluation of diverse host systems and SSD models [7]. We measure the
access the journal’s superblock and get the start address of the journal latency overheads of Check0-SSD by simulating the key components
region (s_start), and it records the address with the region length (e.g., block classifier) in a cycle-accurate ARM simulator [8]. We
(s_maxlen), which will be used for our block classifier at runtime. then apply the latency values in our full-system simulator. The storage
This journal detection is processed only once at the initialization time backend that we simulate contains 60 ULL flashes over 12 channels [7],
or an install time of the filesystem, which has no impact on runtime I/O which has been validated with a real prototype of ULL devices [2].
processes. We configure six different platforms atop the backend flash device.
Runtime Block Classification. The block classifier filters the requests Specifically, i) journal enables journaling for both metadata and
targeting to the journal region, and identifies the block types (e.g., data to achieve the highest level of fault tolerance; ii) async employs
descriptor or commit). Fig. 6(a) shows the detailed process. The block the same journaling scheme as journal, but it writes the commit
classifier first parses the target NVMe command (16 double words) and block along with the descriptor and data blocks to shorten the latency
checks the start LBA therein (❶). If the LBA is not in a range of the of journaling; iii) ordered enables journaling for metadata only; iv)
journal region (between s_start and s_start + s_maxlen), norecovery disables journaling to accelerate the I/O write services;
the classifier can simply forward them to the underlying FTL thereby v) WAL-SSD [5] employs a new set of remapping APIs for the host
processing the requests as normal I/O services. If the LBA is in the to explicitly manage the address remapping in the flash firmware; vi)
journal region, the block classifier then accesses the block that the Check0-SSD is our checkpoint-free firmware, which employs our
NVMe data pointer (DPTR [1]) points to (❷), and examines its journal journal detector, block classifier, and checkpoint flipper modules. We
header’s type (i.e., h_blocktype, ❸). In the case of a descriptor execute microbenchmark and 6 real workloads [9] on Linux 4.19 with
block, the block classifier extracts the LBA list (including D-LPN) ext4 as the filesystem. Table I summarizes the important characteristics
and records the LBA information to CAT with its trans-id, which of the real workloads that we tested.
is kept in the journal header’s h_sequence (cf. ❹ in Fig. 6(b)).
For the following data blocks of the transaction, which are going to A. Performance Analysis of Microbenchmark
be written in the journal region, the block classifier extracts trans-id
and LBA (J-LPN). Since the block classifier maintains D-LPNs and No Synchronization. Fig. 7(a) shows the bandwidth and latency for
trans-id of active blocks over CAT, it updates the D-LPNs of the writing sequential and random writes without synchronization (i.e., fsync()).
transactions with the achieved J-LPNs until detecting the following While async can make the target system fault tolerant, its design
commit block (❺). Lastly, it finishes the composition of CAT (❻). achieves no performance gains compared to journal (under 1%).
This is because the current storage stack is pretty well optimized to
overlap the jbd2’s CPU burst with the underlying I/O activities. Even
C. Checkpointing Elimination
though norecovery and ordered cannot offer filesystem-level
Once a transaction is committed, our checkpoint flipper refers to consistency on the data management, they exhibit the best performance
its trans-id and updates FTL’s ATT for all the previously written data among all the options we tested. The reason why ordered shows
blocks from the journal region to the data region. Specifically, the similar performance to that of norecovery is because it only protects
checkpoint flipper looks up the D-LPNs from CAT by referring to metadata of the target filesystem. Thus, it does not suffer from the
the trans-id and its corresponding J-LPNs ( 1 ). It then retrieves the performance degradation imposed by checkpointing. Both WAL-SSD
corresponding PPNs from ATT by referring to the target J-LPNs as and Check0-SSD provide the highest level of fault tolerance, the same
the indices (2 ). The checkpoint flipper fills the retrieved PPNs in the as journal. Unfortunately, the bandwidth and latency of WAL-SSD
Authorized licensed use limited to: R V College of Engineering. Downloaded on April 29,2025 at 08:18:43 UTC from IEEE Xplore. Restrictions apply.
28 IEEE COMPUTER ARCHITECTURE LETTERS, VOL. 22, NO. 1, JANUARY-JUNE 2023
Fig. 8. Real workload analysis.
Fig. 7. Microbenchmark analysis.
average. This shortens the lifetime of the target SSD by 50%. In contrast,
Check0-SSD and WAL-SSD show WAF close to the ideal even with
are 44% and 85% worse than those of norecovery in the tested
frequent metadata updates, as all the requests are written in a sequential
workloads, respectively. This is because WAL-SSD generates 65%
manner without checkpointing.
more NVMe commands to manage the transactions, which exacerbates
the communication traffic between the host and the SSD controller. V. RELATED WORK
In contrast, Check0-SSD outperforms WAL-SSD by 55% as it re-
moves the communication overheads for transaction management. The One of the approaches to remap the block addresses at checkpoint-
performance of Check0-SSD is slightly worse than norecovery ing is to expose the firmware’s address translation table through a
(13%). This is because Check0-SSD always writes data back to the vendor-specific interface [10]. Thus, the host can update the location
storage while norecovery and ordered leverage page cache to of journal blocks to the target data region. However, these methods can
buffer the write I/Os rather than writing them back to the storage impose more overheads on the kernel in managing journal and data
(thereby inconsistent). Fig. 7(c) analyzes the amount of data written to address spaces. In addition, address remapping via the extra interface
the storage. We use WAF [3] as the metric to measure how much the user can increment loads of NVMe command management as it needs to
writes data, which can exacerbate the SSD lifetime reduction. In this deliver the data region address of each page cache information and
paper, WAF is defined as the ratio of total data volume written in flash block I/O structure (bio) to the underlying ULL SSD. In contrast,
over data written by users. norecovery and ordered mostly do not Check0-SSD autonomously detects descriptor and commit blocks. It
write the data to the SSD as the host-side page cache accommodates identifies the journal region and filesystem layout within the target SSD,
data without flushing. Note that Check0-SSD successfully removes thereby remapping the address for checkpointing directly. While log-
all the checkpointing activities. Therefore, it reduces WAF by 47% structured filesystem avoids the duplicated writes caused by journaling,
compared to journal. deploying it in SSDs introduces write amplification due to log-on-log
Synchronization. Fig. 7(b) analyzes the bandwidth and latency of problem [11]. In contrast, our Check0-SSD leverages the intelligent
different platforms with synchronization. The evaluated platforms use flash firmware to improve the journaling mechanism. It eliminates the
the same system configuration that we used in the previous analysis duplicated writes but reaps all the benefits from the journaling.
except that fsync() is called at the end of each iteration of workload
execution. Since fsync() strictly waits for the write completion, the VI. CONCLUSION
bandwidths for all the storage options degrade by 17%, on average. For
We propose an intelligent SSD firmware to offer the best system-level
both sequential and random patterns, Check0-SSD achieves 41% and
fault tolerance without performance degradation and lifetime short-
30% higher bandwidth than ordered and norecovery, on average,
ening compared to norecovery SSDs. Our evaluations show that the
respectively. This is because both ordered and norecovery suffer
proposed SSD can fully protect data and metadata with 89% enhanced
from the latency overheads imposed by the fsync() process, which
storage lifetime while outperforming the norecovery SSD.
flushes all cached data from the page cache to the storage. Fig. 7(d)
shows the detailed analysis of writeback cache’s side effects; ordered
ACKNOWLEDGMENTS
and norecovery hold the data on the host-side DRAM, and flush
them to storage only at the fsync() call. Thus, dumping the cached data The author would like to thank the anonymous reviewers for their
takes more time while Check0-SSD has less amount of data to flush. constructive feedback.
REFERENCES
B. Real Workload Analysis
[1] NVM Express, Inc., “NVM express specification.” [Online]. Available:
We execute the real workloads along with the fsync() calls peri- https://bit.ly/3ILw9rS
odically (every 32 I/O requests). Fig. 8(a) shows the execution time [2] Intel Corporation, “Intel optane DC SSD series.” [Online]. Available: http:
of the tested real workloads at the user-level. While journal and //bit.ly/41fgurU
async achieve similar performance, they both consume 47% longer [3] X.-Y. Hu et al., “Write amplification analysis in flash-based solid state
execution time than norecovery for all the workloads that we tested. drives,” in Proc. Israeli Exp. Syst. Conf., 2009, Art. no. 10.
ordered consumes slightly (1%) longer execution time than nore- [4] V. Chidambaram et al., “Optimistic crash consistency,” in Proc. 24th ACM
Symp. Operating Syst. Princ., 2013, pp. 228–243.
covery as it spends time in journaling metadata. Our Check0-SSD
[5] K. Han et al., “WAL-SSD: Address remapping-based write-ahead-logging
costs 38%, 11%, and 22% shorter execution time than journal, solid-state disks,” IEEE Trans. Comput., vol. 69, no. 2, pp. 260–273,
norecovery, and WAL-SSD, on average, respectively. In particular, Feb. 2020.
when executing the write-intensive workloads, which invoke intensive [6] Intel Corporation, “Intel SSD 750 series.” [Online]. Available: https://bit.
journaling operations (i.e., ikki, mail and online), Check0-SSD costs ly/3KuUh3k
47% and 31% shorter execution time than journal and WAL-SSD, [7] D. Gouk et al., “Amber: Enabling precise full-system simulation with
on average, respectively. Note that Check0-SSD shows even better detailed modeling of all SSD resources,” in Proc. IEEE/ACM 51st Annu.
performance than norecovery while guaranteeing the highest level Int. Symp. Microarchitecture, 2018, pp. 469–481.
of fault tolerance. This is because the transaction is always sequentially [8] N. Binkert et al., “The gem5 simulator,” ACM SIGARCH Comput. Archi-
tecture News, vol. 39, pp. 1–7, 2011.
written to the journal region. Therefore, all writes are performed in
[9] M. Kwon et al., “Tracetracker: Hardware/software co-evaluation for large-
sequential in Check0-SSD. Additionally, since the host-side NVMe scale I/O workload reconstruction,” in Proc. IEEE Int. Symp. Workload
driver aggregates multiple subsequent requests into a single and large Characterization, 2017, pp. 87–96.
request, Check0-SSD can minimize the number of block I/O struc- [10] M. Bjørling et al., “Open-channel solid state drives,” Vault, Mar. 2015.
tures and NVMe request preparations. Fig. 8(a) analyzes WAF under the [Online]. Available: https://bit.ly/3EtUP5k
execution of real workloads. Similar to the previous analysis, journal [11] J. Yang et al., “Don’t stack your log on my log,” in Proc. 2nd Workshop
and async increase WAF compared to norecovery by 1.99×, on Interact. NVM/Flash Operating Syst. Workloads, 2014, pp. 1–10.
Authorized licensed use limited to: R V College of Engineering. Downloaded on April 29,2025 at 08:18:43 UTC from IEEE Xplore. Restrictions apply.