Measurement 165 (2020) 108165

Contents lists available at ScienceDirect

Measurement
journal homepage: www.elsevier.com/locate/measurement

Performance evaluation of laser trackers using the network method

Ling Wang a,b, Bala Muralikrishnan b,⇑, Octavio Icasio Hernandez c, Craig Shakarji b,
Daniel Sawyer b
a
School of Mechanical and Electrical Engineering, China Jiliang University, Hangzhou, Zhejiang Province 310018, People’s Republic of China
b
Dimensional Metrology Group, Sensor Science Division, National Institute of Standards and Technology, Gaithersburg, MD 20899, United States
c
Centro Nacional de Metrología: Apdo. Postal 1-100 Centro, 76000 Querétaro, Mexico

a r t i c l e i n f o a b s t r a c t

Article history: Laser trackers (LTs) are widely used for large-scale dimensional metrology. Periodic LT performance eval-
Received 24 February 2020 uation is a key aspect in ensuring measurement reliability. The existing documentary standards for LT
Received in revised form 19 June 2020 performance evaluation require scale bars and special artifacts that are often unavailable for many LT
Accepted 29 June 2020
users. A network-based method is considered in this paper as a simple and effective approach to evaluate
Available online 6 July 2020
LT performance without the need for expensive artifacts or reference instruments. In this method, a set of
fixed targets are measured from different locations of the LT. The reference values for the lengths
Keywords:
between pairs of targets are determined by incorporating the geometrical error model of the LT (i.e.,
Laser tracker
Performance evaluation
the LT data from each location are corrected for systematic geometric misalignments, quantities that
Network method are also obtained from the same data), thus the LT under test is itself used to calibrate the lengths in
Geometrical error model parameters the network. Subsequently, these length values are used as references to evaluate the LT performance.
We validate this method of establishing reference lengths by comparing the values for select lengths
in the network against more traditional line-of-sight interferometry. We show that the uncertainties in
the reference lengths obtained in this manner are sufficiently small in comparison to the maximum per-
missible error (MPE) specifications so that they can be used for performance evaluation. A case study
using three LTs is presented in this paper. The results obtained from that case study is also verified by
a simulation based on the ASME B89.4.19 standard, showing the feasibility of the proposed network-
based LT performance evaluation method.
Published by Elsevier Ltd.

1. Introduction mance evaluation of LTs, such as the ASME B89.4.19 [1], VDI/VDE
2617-10 [2], and the ISO 10360-10 [3]. The tests described in these
1.1. Background and objectives standards are comprehensive, require specialized equipment, and
therefore not suitable as a field check procedure. An interim test
Due to their large working volume, high accuracy, and excellent was described by Lee at al. in a National Institute of Standards
portability, laser trackers (LTs) are widely accepted and used in and Technology (NIST) internal report NISTIR 8016 [4], for perfor-
engineering applications and scientific research. Undoubtedly, LT mance evaluation of laser trackers in the field but this test also
measurement performance is a key concern for users, especially required the use of specialized equipment, i.e., a scale bar.
because factors such as environmental conditions, vibration during In this context, we describe a method based on a stationary net-
use, installation, transport, can degrade performance over their work of targets, referred to as the network method, for perfor-
lifetime. Performing a periodic evaluation is therefore a critical mance evaluation of laser trackers. The network method is
aspect for ensuring that an instrument is operating within its per- widely documented in the literature, and its applicability in
formance specifications yielding reliable data, and that the mea- large-scale metrology is described by Calkins [5]. The basic idea
surement results are metrologically traceable. Performance of the network method is to obtain ‘‘average” point coordinates
evaluation tests, clearly, must be sensitive to the different error from measurements on a set of stationary targets from multiple
sources in the LT, thus modeling and understanding LT errors is a locations of an instrument. While this averaging process may be
key aspect in developing useful performance test procedures. as straightforward as determining the rotation and translation
There are several published documentary standards for perfor- parameters of each of the LTs with respect to the LT in the first
position, a more complex bundle adjustment with weighted resid-
⇑ Corresponding author. uals is shown to provide better estimates of the target coordinates
E-mail address: bala.muralikrishnan@nist.gov (B. Muralikrishnan). [5]. However, the averaging in the bundle adjustment process does

https://doi.org/10.1016/j.measurement.2020.108165
0263-2241/Published by Elsevier Ltd.
2 L. Wang et al. / Measurement 165 (2020) 108165

not guarantee that the systematic errors in an LT are removed. In coordinate measuring machine (CMM), which may not be available
order to obtain more accurate estimates of the target coordinates, for most LT users. Conte et al. [22] focused on the performance of
we fit the laser tracker geometric error model as part of the opti- different LT calibration strategies based on network measure-
mization process, thus largely removing the systematic errors of ments, and then determined whether nominal distances are
the LT in the coordinate determination, as reported by Hughes needed. Zhou et al. [23] proposed a bundle adjustment method
et al. [6]. We validate this approach through several line-of-sight in order to evaluate LT coordinate precision in a large-size coordi-
(LOS) interferometer measurements and quantify the uncertainty nate measuring system. Morse et al. [24] noted that published
in the inter-target distances. We show how these inter-target dis- standards and guidelines for evaluating LT performance focus on
tances may be used as the reference lengths for evaluating the per- the measurement of a series of static point-to-point distances
formance of LTs, thus the method does not require the use of and the errors that are reported from those measurements. As a
specialized equipment such as a scale bar. result, they developed a series of tests characterizing a laser track-
Calkins [5], in his dissertation, noted two potential applications er’s performance by collecting data from a moving target to quan-
of the network method. The primary application and the focus of tify dynamic performance of LTs. Sugahara et al. [25] reported the
his dissertation was to use the network method to determine the performance test of an LT and showed the following items: the
uncertainty of the LT. Calkins also notes performance evaluation drift of the output, the long-term variation of the output, and the
as a potential application of the network method, as does Hughes reproducibility of point measurement. Based on a D-H error model,
et al. [6]. We want to clearly note that the novel work described Acero et al. [26] used an indexed metrology platform in order to
in this paper is not the network method itself, rather, it is the enable the evaluation of different working volumes of their LT
demonstration of the network method for laser trackers with care- using the same physical calibrated gauge. This gauge remained still
ful consideration of the factors involved in this process, and the during the verification, and the LT rotated with the platform. The
advantages and limitations of the method. LT performance evalua- nominal distances of this gauge were measured with a CMM.
tion using the network method has not been carefully documented Since the mid-1990s, there had been increasing calls to develop
in the literature to the best of our knowledge. standardized test procedures for performance evaluation so that
Next, we briefly review different performance evaluation test users and manufacturers do not have to develop individual or
procedures described in the literature leading up to published per- idiosyncratic test procedures to assess LT performance. The ASME
formance evaluation documentary standards and the motivation B89.4.19-2006 [1] was the first performance standard released
for the work described in this paper. for this purpose. The VDI/VDE 2617-10 [2] and the ISO 10360-10
[3] followed some years later. Muralikrishnan et al. [27] provide
1.2. Literature review a comparison of the different documentary standards highlighting
the advantages and limitations of each, while Nasr et al. [28]
The laser tracker was invented at the National Institute of Stan- describe the ASME B89.4.19 laser tracker verification facility at
dards and Technology (NIST) in the mid-1980s. In their seminal the NPL in the UK. As mentioned earlier in Section 1.1, the test pro-
paper on LTs, Lau et al. [7] describe the technology and some error cedures described in the documentary standards are comprehen-
sources, thus providing insight into how users might test for those sive and require special artifacts and are typically performed in
errors. There have been several studies since, that describe differ- controlled environmental conditions (note that controlled environ-
ent LT error sources. Zhuang and Roth [8] quantify mirror center ment is not a requirement; testing anywhere within the rated
offset error. Ouyang et al. [9], Gassner and Ruland [10], Martin operating environmental conditions is acceptable, and a compliant
and Chetwynd [11], Giniotis et al. [12], and Muralikrishnan et al. instrument should pass anywhere within those conditions).
[13] describe modeling and detecting different error sources asso- The motivation for this work is to be able to develop a test pro-
ciated with the angle encoders. Zhang et al. [14] propose a method cedure that can be implemented in the field and that does not
of measuring the tilt error between the transit axis and the stand- require specialized equipment. A network-based method is such
ing axis of the laser tracker. These different error sources have been an approach that can be adopted for performance evaluation. The
combined into a model that captures the combined effect on the method requires no calibrated artifact, such as a scale bar, and
measured point coordinates. Such models have been presented can be performed in a room of reasonable size. The process
by Loser and Kyle [15] for LTs with a beam steering mirror and involves measuring a set of stationary targets from several LT loca-
by Muralikrishnan et al. [16] for LTs without a beam steering mir- tions. The data from these measurements are used to calibrate the
ror. Conte et al. [17,18] developed a kinematic model for a laser inter-target distances that are subsequently used to evaluate the
tracker which was compared with a geometric error model of performance of the LT.
LTs. After presenting a method for modeling LT systems, Lin and
Lu [19] analyzed the effects of mirror-mechanism dimension errors
and corner-cube alignment errors. Nasr et al. [20] determined the 1.3. Terminology and organization
azimuth angle encoder errors of their LT based on three methods,
which include: an artifact-based method of NIST, a multi-target Before we proceed, we note the following terminology used in
network technique of National Physical Laboratory (NPL), and a this paper. Coordinates obtained using just the bundle adjustment
method based on precision angular indexing table technique. are referred to as the composite coordinates, while coordinates
Hughes et al. [6] propose a model based on the NIST model that obtained when performing the bundle adjustment while simulta-
describe the effects of different LT error sources. We present the neously fitting to the LT error model are referred to as the cor-
above only to illustrate the fact that there is a significant body of rected composite coordinates. The inter-target distances
work done to understand and quantify the effect of different LT calculated using the composite coordinates are referred to as com-
errors on the measured point coordinates. The study of error posite lengths while, inter-target distances calculated using cor-
sources is not the primary focus of this paper; rather, the focus rected composite coordinates are referred to as corrected
here is the evaluation of the performance of a laser tracker. composite lengths. In this paper, we use the corrected composite
While there has been considerable literature in identifying LT coordinates to obtain the reference values for performance evalua-
error sources, that is not the case with performance evaluation of tion of the LTs, aiming to develop an LT performance evaluation
LTs. Osawa et al. [21] developed a compact and accurate LT in method based on the network without the need for scale bars or
2001. The performance of this LT was checked by a high precision special artifacts.
 L. Wang et al. / Measurement 165 (2020) 108165 3

We used three LTs for our study, one from each of three manu- described in Eq. (1). This process is repeated until the weights
facturers (identified as A, B, C). The software from manufacturer A converge.
allows us to turn off the error model parameters, thus significantly
increasing the errors in the measured point coordinates (which, as The overall procedure is described next, while experimental
will be seen later, allowed the systematic error reduction of the details are described in Sections 2.2 and 3.
network method to be accentuated). In this paper, A1 refers to
the measurements made in this uncompensated mode, while A2 Part I – Data acquisition:
refers to measurements made after turning on the error model (1) We perform a network measurement with n targets and m
parameters for the LT from manufacturer A. LT positions. In each LT position, both front-face and back-
The remainder of this paper is organized as follows. The pro- face measurements are separately performed (i.e., they are
posed method of evaluating LT performance is described in Sec- not averaged.) Therefore, we obtain 2  m sets of n-triples (x,
tion 2. We validate the uncertainties of the corrected composite y, z) using the LT under test.
lengths derived by the network method in Section 3. We describe Part II – Calibration: Determining composite lengths and
a case study for the proposed LT performance evaluation method their uncertainties
based on the network measurement in Section 4, and finally pre- This section is for informational purposes only. The composite
sent discussions and conclusions in Sections 5 and 6, respectively. lengths are not used for performance evaluation.
(2) We arbitrarily choose the frame of the first LT location as the
2. Procedure reference/world coordinate system. An optimization is per-
formed to determine the composite coordinates of the n targets
2.1. Overview in the frame of the first LT location. The rotation/translation
parameters of each LT location are also determined in the opti-
The bundle adjustment process described by Calkins [5] is a mization. The optimization minimizes the weighted residuals in
nested optimization method where the unknown parameters are the frame of each LT, where residuals are the differences
the composite coordinates of the targets and the rotation/transla- between the measured spherical coordinates (in the frame of
tion parameters of the LTs. We first modified this process to a sin- a given LT) and the composite coordinates transformed to the
gle optimization that determines the composite coordinates of the same frame. The calculation of the residuals and their weights
targets and the rotation/translation parameters of the LTs. We ver- is described by Calkins [5].
ified that our non-nested approach provides identical values for (3) We then calculate the (uncorrected) composite lengths in
the unknown parameters as the nested approach of Calkins [5]. the reference coordinate system. We do not use these as the ref-
We subsequently modified the optimization to also fit the LT error erence for performance evaluation of the LT (as this will be done
model parameters as part of the optimization. Thus, we determine with the corrected composite lengths). We only calculate these
the corrected composite coordinates (C), the rotation/translation to show that the residuals are larger than those when we
parameters of the LTs (M), and the LT error model parameters (P) include the LT error model in the fit (see Part III). This educa-
by a single optimization, where the objective function is given by: tional information is derived without any burden of requiring
X2m Xn additional measurements with the LT.
min f ðP; M; CÞ ¼ min j¼1 i¼1 ½ðwl; i;j  el; i;j Þ2 þ ðwh; i;j  eh; i;j Þ2 (4) We calculate the standard deviations of the residuals
þ ðwu;  eu; i;j Þ2  ð1Þ (el; i;j ; eh; i;j ; and eu; i;j Þ along the ranging and the horizontal and
i;j
vertical angle directions of the LT. We consider these standard
In Eq. (1): deviations as the LT uncertainty, i.e., the standard uncertainty
of a point coordinate obtained from the LT that was used for
n is the number of targets and m is the number of LT locations. the measurements. Here, we are assuming that there are no
P is the set of error model parameters of the LT. There are 15 remaining systematic effects due to the LT, the environment,
model parameters for trackers without a beam steering mirror etc. Clearly, this assumption may not be valid because the LT
and 20 parameters for trackers with a beam steering mirror, may have uncompensated geometric misalignments. There
see references [15,16] for more details. Thus, P is a vector whose may also be systematic errors from the environment and other
size is either 1  15 or 1  20. sources. The model errors are accounted for in Part III, but other
M is the set of three translation and three rotation (for a total of systematic error sources may remain; we discuss this as a lim-
6) parameters for each of the LT positions with respect to the itation of this method in Section 5.
reference position. Because we consider the first LT position (5) A Monte Carlo (MC) simulation is then performed to deter-
to be the reference, M is a vector of size 1  6 (m  1). mine the uncertainty in the composite length as follows. The
C is a matrix of composite coordinates of the n targets in the ref- input to the MC simulation are the LT uncertainties obtained
erence position and therefore of size n  3 matrix. as the standard deviation of the residuals (el; i;j ; eh; i;j ; and
el; i;j ; eh; i;j ; and eu; i;j are the residuals along the ranging direction, eu; i;j Þ. These uncertainties are then propagated to each of the
the horizontal angle and vertical angle directions for the jth tar- n measured points from each of the m LT stations. An optimiza-
get measured from the ith LT position. tion is performed per Eq. (1) (where the weights are also equal
wl; i;j ; wh; i;j ; and wu; i;j are the weights assigned to the residuals. to the inverse of the standard deviation of the residuals) to
See Ref. [5] for details. The initial weights (wl; i;j ; wh; i;j ; and determine the composite coordinates and thus, the composite
wu; i;j in Eq. (1)) used for the optimization may be generated lengths. Note that LT error model parameters are not deter-
from the MPEs of the LT. In our case, we determine the initial mined as part of the optimization. This process is repeated
weights as the inverse of the initial estimates of the correspond- 300 times to determine a set of 300 composite lengths. The
ing standard uncertainties, which we choose to be 0.0076 mm standard uncertainty of each length is the standard deviation
for range and 100 for angles. At the end of the optimization pro- of the 300 samples. We calculate these values only to show that
cess, the standard deviations of the range, the horizontal angle, the uncertainty in the corrected composite lengths (see Part III)
and the vertical angle are calculated. These are then used to are smaller because the error model accounts for a significant
determine the weights for the next iteration of the optimization portion of the residuals in the measured data.
4 L. Wang et al. / Measurement 165 (2020) 108165

Part III – Calibration: Determining corrected composite the U < MPE/4 condition as the uncertainty is not small enough
lengths and their uncertainties to determine conformance. For the lengths satisfying U < MPE/4,
(6) An optimization is performed to determine the corrected we compare their MPEs and length errors. The LT has passed the
composite coordinates of the n targets in the frame of the first test procedure if all length errors are smaller than the corre-
LT location. The laser tracker error model parameters and the sponding MPEs for the lengths satisfying the U < MPE/4
rotation/translation parameters of each LT location are also condition.
determined in the optimization. The objective function was
shown earlier in Eq. (1).
2.2. Experimental setup
(7) We then calculate the corrected composite lengths in the
reference coordinate system, which we use as the reference
The proposed performance evaluation based on the network
for performance evaluation of the LT.
method was realized through an experiment with n = 19 targets
(8) We calculate the standard deviations of the residuals
and m = 5 LT positions. The position of the targets and the LT
(el; i;j ; eh; i;j ; and eu; i;j Þ along the ranging and the horizontal and
was based on Hughes et al., [6], shown in Fig. 1. In the experiment,
vertical directions of the LT. We consider these standard devia-
all 19 targets were measured successively from the front-face and
tions as the LT uncertainty, i.e., the standard uncertainty of a
back-face modes by the LT in each of the five positions. As a result,
point coordinate obtained from the LT that was used for the
there are 10 sets of 19-triplets (x, y, z) from our experiment. Posi-
measurements. Here, we are assuming that there are no
tion 2 is at the same location as Position 1, except that the LT is
remaining systematic effects due to the environment and other
physically rotated about the vertical axis by 180° by re-orienting
sources, but that assumption may not be entirely valid. We dis-
the stand. This ensures that the targets are measured from two dif-
cuss this in Section 5. Note that the LT uncertainties so calcu-
ferent sets of azimuth angles. More information about the posi-
lated can be used as the LT’s contribution to the overall
tions can be found in Hughes et al. [6].
uncertainty of a point coordinate in any subsequent measure-
We have three LTs in our laboratory, one from each of three dif-
ment performed by the LT. Thus, the advantage of the network
ferent manufacturers, as mentioned earlier. They are identified as
method is that we not only recover the error model parameters
A, B, and C. Their structures and error models are given in Refs.
of the LT but also an estimate of the uncertainty that is due to
[15,16]. One of the LTs has the laser source in the base, and a beam
the LT.
steering mirror is used to deflect the beam to the target. Another LT
(9) An MC simulation is then performed to determine the
has the source also in the base but uses fiber to launch the beam.
uncertainty in the composite length, as discussed in Step 4 ear-
The third LT has the source directly mounted on the head. The
lier. The inputs to the MC simulation are the LT uncertainties
entire 19-target and 5-position experiment was separately per-
obtained as the standard deviation of the residuals (el; i;j ; eh; i;j ;
formed using all three LTs. The software for manufacturer A
and eu; i;j Þ. These uncertainties are then propagated to each of
allowed us to turn off the model parameters. Thus, we performed
the n measured points from each of the m LT stations. An opti- the experiments using LT A with, and without the model parame-
mization is performed per Eq. (1) (where the weights are also ters turned on. The data are identified as LT A1 and LT A2,
equal to the inverse of the standard deviation of the residuals) respectively.
to determine the corrected composite coordinates and, thus, For each LT, we calculated a total of 171 lengths from 19 targets.
the corrected composite lengths. This process is repeated 300 Thus, we have 171 composite lengths and 171 corrected composite
times to determine a set of 300 corrected composite lengths. lengths. Because there are 5 LT positions and 2 modes (front-face
The standard uncertainty of each length is the standard devia- or back-face) of measurement from each LT position, we have a
tion of the 300 samples. This method assumes that the effects total of 171  5  2 = 1710 lengths that can be used for perfor-
of systematic errors in the LT measurements are sufficiently mance evaluation.
removed in the corrected composite lengths and is not signifi-
cant (see Section 5).
(10) We validate the uncertainty claims by measuring selected 3. Calibration results
pairs of targets using LOS interferometry method (Wang et al.,
[29]) and demonstrate that the measured errors are consistent After measuring the n targets from the m LT positions, we per-
with our estimates of the expanded uncertainty. form the optimization described in Section 2 in Parts II and III. First,
Part IV – Performance evaluation: we only determine the composite coordinates (i.e., without fitting
(11) We consider each position of the LT as a test position and to the LT error model) and the uncertainty in the composite lengths
calculate the length between pairs of targets using data as described in Part II. These uncertainties are shown in Table 1. In
obtained from that position. Note that while LT model parame- a separate optimization process, we determine the corrected com-
ters were obtained in Part III, they are only used to obtain more posite coordinates and the uncertainties in the corrected compos-
accurate estimates of the composite coordinates. The data ite lengths as described in Part III. These are also shown in Table 1.
obtained from the LT is not corrected using these model param- Table 1 shows that the LT uncertainty (k = 1) (i.e., the standard
eters for purposes of performance evaluation. deviation of the residuals) decrease when we correct the measured
(12) We then derive the length errors between the measured data using the error models, and the uncertainties of the horizontal
lengths in each LT position and the corrected composite lengths. and vertical angles are less than or equal to one arc second. These
(13) We calculate the MPE values for all the lengths measured values are used to determine the uncertainties of the composite
from each LT position based on specifications provided by LT and corrected composite lengths as described in Section 2.1 Parts
manufacturers. II and III. The standard uncertainties of the corrected composite
(14) We use a 4:1 simple acceptance decision rule (ASME lengths are smaller than the ones of the composite length shown
B89.7.3.1 [30]]) to determine whether an LT has passed the test. in Table 1, which implies the error model improves the repeatabil-
First, we compare the MPE values to four times the expanded ity of the tracker for the lengths measured.
uncertainties (4U) for each length measured from each LT posi- We assume that the mean errors in the composite and corrected
tion (see Part III, step 9 for method to evaluate uncertainty of composite lengths are small. That is, the differences between the
the reference length). We discard the lengths that do not satisfy unknown true or reference value of each length and the corre-
 L. Wang et al. / Measurement 165 (2020) 108165 5

2000 2000
9
19 10
1500 6 11 1500 3
12
Tracker
1000 position
14 12
Y coordinate (mm)

5 1000

Z coordinate (mm)
Tracker
500 position 5

2 Tracker Tracker 500

0 3 18 5 position 8 position 2 Tracker Tracker
3 1/2 19 position 4 position 1/2 15
1 18
0 17 11
-500 16

Tracker
8
17 7 -500 Tracker
-1000 4 position
4 6 position 3 9
13 4
7
-1500 -1000
14 15 10
16 1 5 13
-1500
-6000 -5000 -4000 -3000 -2000 -1000 0 1000 -6000 -5000 -4000 -3000 -2000 -1000 0 1000
X coordinate (mm) X coordinate (mm)
(a) (b)

(c)
Fig. 1. Network layout of targets and LT positions (diamonds with crosses represent target positions): (a) plan view (b) side view (c) view of artifacts.

Table 1
Standard uncertainties of LT point coordinates, composite lengths, and corrected composite lengths.

Laser Without LT error model With LT error model

tracker
LT uncertainty LT uncertainty LT uncertainty Typical LT uncertainty LT uncertainty LT uncertainty Typical
along range (i.e., along horizontal along vertical standard along range along horizontal along vertical standard
the standard angle (i.e., the angle (i.e., the uncertainty (i.e., the angle (i.e., the angle (i.e., the uncertainty
deviation of standard deviation standard (k = 1) of standard standard deviation standard (k = 1) of
range residual) of horizontal angle deviation of composite deviation of of horizontal angle deviation of corrected
(lm) residual) (00 ) vertical angle length (lm) range residual) residual) (00 ) vertical angle composite
residual) (00 ) (lm) residual) (00 ) length (lm)
A1 77 7.6 3.8 33 3.2 0.8 1.0 3
A2 4 1.0 2.6 4 3.1 0.9 1.0 3
B 2 1.8 1.2 3 1.0 0.7 0.8 2
C 33 0.7 2.7 10 3.8 0.5 0.6 3

sponding corrected composite and composite lengths are small so responding target pairs of these 15 lengths are 1–4, 1–5, 1–7, 1–8,
that the standard deviations of the lengths are reasonable esti- 1–10, 5–7, 5–8, 5–9, 7–14, 7–17, 10–11, 10–12, 16–17, 16–18, and
mates of the uncertainties in the length. In order to validate this 16–19, whose lengths are relatively easier to measure by the LOS
assumption, 15 out of the 171 lengths in the 19-target network method. These LOS lengths are considered as the reference for this
are also measured by the LOS method (Wang et al. [29]). The cor- validation process. The length errors between the corrected com-
6 L. Wang et al. / Measurement 165 (2020) 108165

1.2 1.2
LT A
1 LT A 1
1 LT A 1
2 LT A
2
0.8 LT B 0.8 LT B
LT C
LT C
R

R
0.6 0.6

0.4 0.4

0.2 0.2

0 0
0 5 10 15 0 5 10 15
Length number Length number
(a) (b)
Fig. 2. Ratios (normalized errors) of 15 inter-target distances in the 19-target network: (a) for composite length errors (b) for corrected composite length errors.

posite lengths and the LOS lengths are obtained. In order to As addressed in the last section, each individual position of the
demonstrate the validity of our method, we compute the ratio R LT is considered as a test position for the purposes of performance
between length errors and expanded uncertainties (k = 2). This evaluation. The measured data from the 10 LT positions result in
ratio is given by: 171  10 test lengths.
  The next step is the determination of the MPE specification for
 
  each of the 171 lengths from each of the 10 LT positions. The MPE
 eLC 
R ¼ qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ð2Þ specifications are calculated by a formula based on the manufac-
 2 2 
 U C þ U LOS  turer’s specification (or otherwise specified) cited to the ISO
10360–10 standard [3], using the parameters shown in Eq. (3)
where eLC is the length error between the composite length (or cor- and Fig. 3.
rected composite length) and the LOS measured length; UC is the
MPE ¼ ½e2R1 sin a1 þ e2R2 sin a2 þ e2R0 ðsin a1 þ sin a2 Þ
2 2 2
expanded uncertainty (k = 2) of the composite length (or corrected
composite length); ULOS is the expanded uncertainty (k = 2) of the
þ e2T1 cos2 a1 þ e2T2 cos2 a2 
1
2 ð3Þ
length measured by the LOS method. For the purposes of this study,
we consider the LOS uncertainty ULOS to be 2  16 = 32 lm. This where eR1 and eR2 are the MPEs for range measurement to targets 1
value is obtained from manufacturer’s specifications for the LT that and 2, respectively, eR0 is the MPE of the R0 error in the range (i.e.,
was used to establish the LOS measurement. zero error), and eT1 and eT2 are the MPEs of the transverse errors to
As shown in Fig. 2, the ratio R (i.e., the normalized error) for the targets 1 and 2, respectively. See ISO 10360–10 [3] for the detailed
15 lengths are all smaller than one except for the case of length No. description.
1, tracker C and even for that one case, the normalized error is As an example, consider a hypothetical specification of
approximately equal to one. This result shows that the errors of eR1 = eR2 = 0.01 mm, eR0 = 0, eT1 = 0.01 + 0.005  R1/1000 mm,
the composite and corrected composite lengths are smaller than and eT2 = 0.01 + 0.005  R2/1000 mm, where R1 and R2 are the
our estimate of the expanded uncertainties, which at the (k = 2) distances (in millimeters) between the LT and the ends of the
level have about a 95% level of confidence. The claimed uncertain- length (Points P1 and P2), shown in Fig. 3; If the coordinates of
ties are consistent with our assumption that the mean error in the Points P1 and P2 are known in Fig. 3, then the MPE of the length
composite and corrected composite lengths are small relative to d can be calculated based on Eq. (3). For example, suppose the
the uncertainty estimated using the standard deviations. It is range, horizontal and vertical (zenith) angle to Point P1 are
important to mention that if the absolute value of R is greater than 500 mm, 30°, and 70°, respectively, and the range, horizontal and
0.5 and close to one, then uncertainties are evaluated in a realistic vertical angle to Point P2 are 2000 mm, 60°, and 90°, respectively,
and balanced way; this is the case for the evaluated lengths in then from the geometry, a1 = 44.1° and alpha a2 = 79.7°. From
Fig. 2. If R is close to zero, then either the error in the numerator the manufacturer’s specification, eR1 = 0.01 mm, eT1 = 0.013 mm,
is small or one or both uncertainties in the denominator are too eR2 = 0.01 mm, and eT2 = 0.020 mm. Thus, the MPE for the length
conservative. For users of the network method for performance so positioned is 0.015 mm.
evaluation of LTs, we note that reference LOS measurements (or
a calibrated scale bar) may be necessary only from the point of
establishing metrological traceability; the results shown here indi- P1 P2
cate that the LT is capable of self-calibrating the network with low Point 1 Point 2

uncertainties.

4. Performance evaluation results α1 P1 P1 = d

α2 OP1 = R 1
OP2 = R 2
The previous section described how we established the refer-
ence lengths (the corrected composite lengths) between pairs of Laser
targets, and the uncertainties in those lengths. In this section, we Tracker O

will show how we can use those reference lengths to assess the
performance of LTs. Fig. 3. Geometry for MPE calculation (ISO 10360-10 [3]).
 L. Wang et al. / Measurement 165 (2020) 108165 7

After the MPEs are determined (one for each length and tracker tify those lengths for which we can determine conformance. This is
location), the next step is to adopt a decision rule to determine shown for LTs A1, A2, B, and C in Figs. 4(a), 5(a), 6(a), and 7(a), respec-
whether the LT meets the specification. As mentioned earlier in Sec- tively. There are 1648 MPE values that are larger than or equal to the
tion 2.1, we adopt the 4:1 Simple Acceptance decision rule, which corresponding 4Us for LT A1. For LTs A2, B and C, there are 1654,
states that the LT has passed the test if the error does not exceed 1710, and 1062 lengths, respectively, that meet the 4:1 criteria.
the MPE, and the expanded uncertainty (U) in the reference length These lengths are therefore used for performance evaluation.
is no larger than one-fourth the MPE (MPE/4). Thus, we first com- After we identify the lengths that meet the 4:1 criteria, we plot
pare the MPE against four times the expanded uncertainty and iden- the MPEs vs the length errors for each LT. This is shown for LTs A1,

0.08 0.35 1.5

Length tests Length tests Two-face tests
0.3
0.06
0.25
1
MPE (mm)

MPE (mm)

MPE (mm)
0.2
0.04
0.15
0.5
0.1
0.02
0.05

0 0 0
0 0.02 0.04 0.06 0.08 0 0.1 0.2 0.3 0 0.5 1 1.5
4U (mm) |Error| (mm) |Error| (mm)
(a) (b) (c)
Fig. 4. Comparison of MPE, 4U, and measured errors for LT A1 (a): MPE vs 4U, (b) MPEs vs length errors, (c) MPEs vs two-face errors.

0.08 0.15 0.25

Length tests Length tests Two-face tests
0.2
0.06
0.1
MPE (mm)

MPE (mm)
MPE (mm)

0.15
0.04
0.1
0.05
0.02
0.05

0 0 0
0 0.02 0.04 0.06 0.08 0 0.05 0.1 0.15 0 0.1 0.2
4U (mm) |Error| (mm) |Error| (mm)
(a) (b) (c)
Fig. 5. Comparison of MPE, 4U, and measured errors for LT A2 (a): MPE vs 4U, (b) MPEs vs length errors, (c) MPEs vs two-face errors.

0.1 0.15 0.2

Length tests Length tests Two-face tests
0.08
0.15
MPE (mm)

MPE (mm)

0.1
MPE (mm)

0.06
0.1
0.04
0.05
0.05
0.02

0 0 0
0 0.05 0.1 0 0.05 0.1 0.15 0 0.05 0.1 0.15 0.2
4U (mm) |Error| (mm) |Error| (mm)
(a) (b) (c)
Fig. 6. Comparison of MPE, 4U, and measured errors for LT B (a): MPE vs 4U, (b) MPEs vs length errors, (c) MPEs vs two-face errors.
8 L. Wang et al. / Measurement 165 (2020) 108165

0.08 0.6
0.15
Length tests Length tests Two-face tests
0.5
0.06
0.4
MPE (mm)

MPE (mm)

MPE (mm)
0.1
0.04 0.3

0.05 0.2
0.02
0.1

0 0 0
0 0.02 0.04 0.06 0.08 0 0.05 0.1 0.15 0 0.2 0.4 0.6
4U (mm) |Error| (mm) |Error| (mm)

(a) (b) (c)

Fig. 7. Comparison of MPE, 4U, and measured errors for LT C (a): MPE vs 4U, (b) MPEs vs length errors, (c) MPEs vs two-face errors.

Table 2
Comparison results: MPE, four times expanded uncertainties, and errors.

Laser Length measurement tests based on the network method Two-face measurement tests
tracker
Number of Number of conforming Number of length Percentage of Number of Number of two-face Percentage of
name
measured length measurements measurements where failed length measured two- measurements where failed two-face
lengths with MPE  4U |Error| > MPE measurements face errors |Error| > MPE measurements
A1 1710 1648 1042 63.2% 95 95 100%
A2 1710 1654 87 5.3% 95 80 84.2%
B 1710 1710 234 13.7% 95 9 9.5%
C 1710 1062 316 29.8% 95 95 100%

A2, B, and C in Figs. 4(b), 5(b), 6(b), and 7(b), respectively. We find B89.4.19 performance tests. We could not, however, run the test
that the length errors are larger than the corresponding MPE for procedures on the LTs because the LTs were already moved/boxed
1042 out of the 1648 lengths, i.e., 63.2%, for LT A1 as shown in and setting them up again would have resulted in new error model
Table 2. For LTs A2, B, and C, these percentages are 5.3%, 13.7%, parameters. We therefore performed the following simulation. We
and 29.8%, respectively, as shown in Table 2. numerically constructed the coordinates of the end points for each
Finally, because we measured front-face and back-face mea- reference length and each two-face test described in the ASME
surements separately from each LT position, we also calculate B89.4.19 standard. We then imposed the model parameters (deter-
the errors of the 19  5 = 95 two-face measurements, as well as mined from the network method) of all the LTs into the simulation
their corresponding claimed MPEs that are based on the ISO to determine the length errors and two-face errors in the presence
10360–10 standard. Note the MPE specification for two-face error of the error model. The result (MPE vs |Error|) plots for the four
is simply twice the transverse specification, i.e., MPE = 2eT. Because cases are shown in Figs. 8–11, summarized in Table 3. Although
the two-face test involves essentially zero-length measurements LTs A2, B, and C pass the simulated length measurement tests of
made on a single, stationary retroreflector, the uncertainty in a the ASME B89.4.19 standard, all the LTs failed the simulated two-
two-face test is exceedingly small and therefore all 95 two-face face measurement tests. This indicates that the LTs do indeed
tests are included in determining whether an LT passed the test. require compensation. According to the experiment result and
We plot the MPE vs the two-face errors for each LT. This is shown the analysis above, we conclude that the proposed method based
for LTs A1, A2, B, and C in Figs. 4(c), 5(c), 6(c), and 7(c), respectively. on the network method can be used for LT performance evaluation.
We find that the two-face errors are larger than the corresponding
MPE for all 95 two-face tests of LT A1, i.e., 100%, as shown in Table 2.
For LTs A2, B, and C, these percentages are 84.2%, 9.5%, and 100%, 5. Discussion and future work
respectively, as shown in Table 2.
Based on the results above, none of the LTs pass the perfor- There are several points worth discussing regarding the realiza-
mance test based on the network method. Among them, LT A1 is tion of this method.
the worst with 63.2% of its length measurements failing the net-
work test. That is however not surprising because we had inten- (1) Uncertainty of the corrected composite lengths
tionally turned off the error model parameters for that LT when
performing for the test. The surprising observation though was In order to determine the uncertainty in the corrected compos-
that the other LTs failed the network test. All the three LTs in our ite lengths, we have propagated the LT point coordinate uncer-
experiment have been used for several years. They were not how- tainty (i.e., the one-standard deviation residuals from the
ever compensated immediately prior to this study. The error mod- network) to the corrected composite coordinates, and therefore,
els parameters are likely incorrect and not able to compensate the inter-target distances (see Section 2.1 for details). These are
their current geometric errors. This may be the reason why the shown in Table 1. While it is not practical, it is theoretically possi-
LTs failed the test in this study. We will explore this issue further. ble to obtain unrealistically low uncertainties in the corrected
After noticing that all the LTs failed the network test, we were composite lengths by greatly increasing the number of LT posi-
curious as to whether these LTs would have passed the ASME tions. This is because increasing the number of LT positions
 L. Wang et al. / Measurement 165 (2020) 108165 9

0.09 1.8

0.08 Length tests 1.6 Two-face tests

0.07 1.4

0.06 1.2
MPE (mm)

MPE (mm)
0.05 1

0.04 0.8

0.03 0.6

0.02 0.4

0.01 0.2

0 0
0 0.02 0.04 0.06 0.08 0 0.5 1 1.5 2
|Error| (mm) |Error| (mm)

(a) (b)
Fig. 8. Comparison of MPEs and simulated errors of (a) length tests, and (b) two-face tests for LT A1 based on ASME B89.4.19 standard.

0.2
0.08 Length tests Two-face tests

0.15
0.06
MPE (mm)

MPE (mm)

0.1
0.04

0.02 0.05

0 0
0 0.02 0.04 0.06 0.08 0 0.05 0.1 0.15 0.2
|Error| (mm) |Error| (mm)
(a) (b)
Fig. 9. Comparison of MPEs and simulated errors of (a) length tests, and (b) two-face tests for LT A2 based on ASME B89.4.19 standard.

0.2
Length tests Two-face tests
0.08

0.15
0.06
MPE (mm)

MPE (mm)

0.1
0.04

0.02 0.05

0 0
0 0.02 0.04 0.06 0.08 0 0.05 0.1 0.15 0.2
|Error| (mm) |Error| (mm)

(a) (b)
Fig. 10. Comparison of MPEs and simulated errors of (a) length tests, and (b) two-face tests for LT B based on ASME B89.4.19 standard.
10 L. Wang et al. / Measurement 165 (2020) 108165

0.08 0.6
Length tests Two-face tests
0.5
0.06
0.4
MPE (mm)

MPE (mm)
0.04 0.3

0.2
0.02
0.1

0 0
0 0.02 0.04 0.06 0.08 0 0.1 0.2 0.3 0.4 0.5 0.6
|Error| (mm) |Error| (mm)
(a) (b)
Fig. 11. Comparison of MPEs and simulated errors of (a) length tests, and (b) two-face tests for LT C based on ASME B89.4.19 standard.

Table 3
Simulation results of LT performance tests based on ASME B89.4.19.

Test name Length tests Two-face tests

Total number of errors 35 36
LT name A1 A2 B C A1 A2 B C
Number of failed tests because |Error| > MPE 10 0 0 0 36 24 19 36

*See ASME B89.4.19 [1] for a description of test positions.

provides an averaging effect on the composite coordinate. In real- interim test [4], the ISO 10360-10 [3] test procedures, and the pro-
ity, there are likely always systematic errors that are not removed posed network tests. This is to experimentally validate the network
by averaging. In the presence of large number of LT positions, we procedure against the more established and comprehensive
believe that a realistic estimate for the uncertainty of the corrected national or international documentary standards.
composite lengths cannot be smaller than that realizable using an
LOS interferometer. In our case, we used an estimate of 32 mm
(k = 2) for LOS measurements derived from manufacturer specifica- 6. Conclusions
tions which may be somewhat conservative. An estimate based on
a ranging test will likely yield more reasonable values for this We describe the practical realization of the network method for
uncertainty. performance evaluation of LTs in this paper. In order to reduce the
uncertainty in the reference lengths (which we refer to as the cor-
(2) Network robustness rected composite lengths), we fit the LT error model as a part of the
network bundle adjustment process. We validate the uncertainties
The robustness of the network, i.e., number and location of tar- of corrected composite lengths by comparing selected lengths
gets and LTs, is a key factor in determining the uncertainty in the against direct LOS interferometry. We then discuss issues pertain-
reference lengths. We have also performed these experiments with ing to the use of these reference lengths for performance evalua-
15 targets and 4 LTs, but the uncertainty of the reference lengths tion of LTs through a case study involving three LTs. The main
were larger than for the case of the 19 targets and 5 LTs network. conclusions include:
There is scope for substantial simulation work in this area to deter-
mine the optimal network configuration for performance evalua- (1) Fitting the LT error model as part of the network bundle
tion purposes. adjustment process in order to lower the uncertainties of
the corrected composite lengths is a novel contribution of
(3) New test positions this paper. We determined that uncertainties in the lengths
are smaller than the MPE specifications by at least a factor of
In the procedure we have used, the individual LT positions were four, thus these corrected composite lengths can be used as
considered as the test positions, while they also contributed to the the references for evaluating LT performance.
determination of the reference lengths. It may be argued that this (2) In the experimental data presented, all LTs failed the pro-
is not a fair test, i.e., the test positions are not independent of the posed performance evaluation test. As mentioned earlier,
calibration process. It is possible to modify this process by adding this is likely because we had not compensated the LTs prior
another LT position simply for purposes of performance evaluation. to testing. This result is also verified by a simulation based
We leave this task for the future. on the ASME B89.4.19 Standard. That is, for the performance
test of these LTs, the proposed method is consistent with the
(4) Other future work standard.
(3) The proposed LT performance evaluation procedure based
As future work, we plan on compensating each of our LTs, then on the network method is a feasible test that a user may
perform the ASME B89.4.19 test procedures, the NISTIR 8016 quickly perform in the field to determine LT health. The
 L. Wang et al. / Measurement 165 (2020) 108165 11

network of target and tracker locations in our experiment [2] Verein Deutscher Ingenieure, VDI/VDE 2617 part 10: Accuracy of Coordinate
Measuring Machines: Characteristics and their Checking: Acceptance and
did not cover all the required length measurements con-
Reverification Tests of Laser Trackers, Düsseldorf: Verein Deutscher
tained in the ISO 10360-10 or ASME B89.4.19 Standard. Ingenieure, 2011.
But that can be easily rectified by carefully choosing tracker [3] International Organization for Standardization, ISO 10360-10: Geometrical
and target locations to cover the ASME B89.4.19 or ISO Product Specifications (GPS)—Acceptance and Reverification Tests for
Coordinate Measuring Systems (CMS)—Part 10: Laser Trackers for Measuring
10360-10 test positions. Point-to-Point Distances, 2016.
(4) We must emphasize that the network method removes the [4] V.D. Lee, C. Blackburn, B. Muralikrishnan, D. Sawyer, M. Meuret, A.
need for a calibrated and traceable scale bar, however, the Hudlemeyer, A proposed interim check for field testing a laser tracker’s 3D
length measurement capability using a calibrated scale bar as a reference
question of how to obtain traceable results is something that artifact, NIST Internal Report, NISTIR 8016, 2014.
is not covered in this paper. One route to establishing metro- [5] J. Calkins, Quantifying Coordinate Uncertainty Fields in Coupled Spatial
logical traceability is by calibrating the frequency of the Measurement Systems, PhD Dissertation of Virginia Tech, 2002.
[6] B. Hughes, A. Forbes, A. Lewis, W. Sun, D. Veal, K. Nasr, Laser tracker error
interferometer used in the LOS measurements. We refer determination using a network measurement, Meas. Sci. Technol. 22 (4) (2011)
the topic of metrological traceability of performance tests 045103.
using the network method to a future publication. [7] K. Lau, R.J. Hocken, W.C. Haight, Automatic laser tracking interferometer
system for robot metrology, Precis. Eng. 8 (1) (1986) 3–8.
[8] H. Zhuang, Z.S. Roth, Modeling gimbal axis misalignments and mirror center
It is worth noting that a drawback of the proposed LT perfor- offset in a single-beam laser tracking measurement system, Int. J. Rob. Res. 14
mance evaluation method in this paper is that it requires a larger (3) (1995) 211–224.
[9] J.F. Ouyang, W.L. Liu, D.X. Sun, Y.G. Yan, Laser tracker calibration using
working volume than the ASME B89.4.19 Standard. More than a
coordinate measuring machine, Proceedings of the ASPE 20th Annual Meeting.
dozen SMR nests are also necessary during the implementation Norfolk, Virginia, 2005.
of this method. A comprehensive experimental comparison study [10] G. Gassner, R. Ruland, Laser tracker calibration – testing the angle
of the proposed method in this paper with the related documen- measurement system, SLAC-PUB-13476, 2008.
[11] D. Martin, D.G. Chetwynd, High precision angle calibration of robotic total
tary standards is a task for the future. stations and laser trackers, Proceedings of the SPIE vol. 7133 (2008).
[12] V. Giniotis, D. Bručas, L. Šiaudinytė, Arrangement for vertical angle calibration
of geodetic instruments, Mechanics (ISSN 1392–1207) 5 (79) (2009) 59–62.
Disclaimer [13] B. Muralikrishnan, C. Blackburn, D. Sawyer, S. Phillips, R. Bridges, Measuring
scale errors in a laser tracker’s horizontal angle encoder through simple length
Commercial equipment and materials may be identified in measurement and two-face system tests, J. Res. NIST 115 (5) (2010) 291–301.
[14] Z. Zhang, W. Zhou, H. Zhu, X. Lin, A novel method for measuring transit tilt
order to adequately specify certain procedures. In no case does
error in laser trackers, Proceedings of the SPIE 9446 (2015), https://doi.org/
such identification imply recommendation or endorsement by 10.1117/12.2182383.
the National Institute of Standards and Technology, nor does it [15] R. Loser, S. Kyle, Alignment and field check procedures for the Leica Laser
Tracker LTD 500, Boeing Large Scale Optical Metrology Seminar, 1999.
imply that the materials or equipment identified are necessarily
[16] B. Muralikrishnan, D. Sawyer, C. Blackburn, S. Phillips, B. Borchardt, W.T. Estler,
the best available for the purpose. ASME B89.4.19 Performance evaluation tests and geometric misalignments in
laser trackers, J. Res. Natl. Inst. Stand. Technol. 114 (1) (2009) 21–35.
[17] J. Conte, J. Santolaria, A. Majarena, A. Brau, J. Aguilar, Laser tracker error
CRediT authorship contribution statement modeling and kinematic calibration strategy, Key Eng. Mater. 615 (2014) 63–
69.
Ling Wang: Investigation, Software, Validation, Formal analysis, [18] J. Conte, A. Majarena, R. Acero, J. Santolaria, J. Aguilar, Performance evaluation
of laser tracker kinematic models and parameter identification, Int. J. Adv.
Writing - original draft, Writing - review & editing. Bala Muralikr- Manuf. Technol. 77 (2015) 1353–1364.
ishnan: Conceptualization, Software, Validation, Formal analysis, [19] P.D. Lin, C.-H. Lu, Modeling and sensitivity analysis of laser tracking systems by
Writing - review & editing. Octavio Icasio Hernandez: Conceptu- skew-ray tracing method, J. Manuf. Sci. Eng. 127 (3) (2005) 654–662.
[20] K.M. Nasr, B. Hughes, A. Forbes, A. Lewis, Determination of laser tracker angle
alization, Software, Validation, Formal analysis, Writing - review
encoder errors, EPJ Web of Conferences 77, Article No. 00002, 2014.
& editing. Craig Shakarji: Writing - review & editing. Daniel Saw- [21] S. Osawa, T. Tashiyuki, J. Hong, H. Noguchi, T. Kurosawa, Evaluation of the
yer: Resources, Writing - review & editing. performance of a novel laser tracker used for coordinate measurements,
Proceedings of the SPIE, 4401, 2001, https://doi.org/10.1117/12.445613.
[22] J. Conte, A.C. Majarena, S. Aguado, R. Acero, J. Santolaria, Calibration strategies
Declaration of Competing Interest of laser trackers based on network measurements, Int. J. Adv. Manuf. Technol.
83 (5–8) (2016) 1161–1170.
The authors declare that they have no known competing finan- [23] W.H. Zhou, L. Ding, Y.W. Wang, Z.L. Zhang, Application of bundle adjustment to
accuracy evaluation of laser tracker, Opt. Precis. Eng. 20 (4) (2012) 851–857 (in
cial interests or personal relationships that could have appeared Chinese).
to influence the work reported in this paper. [24] E. Morse, V. Welty, Dynamic testing of laser trackers, CIRP Ann. Manuf.
Technol. 64 (1) (2015) 475–478.
[25] R. Sugahara, M. Masuzawa, Y. Ohsawa, Performance test of laser trackers of
Acknowledgements Faro, in: Proceedings of the 7th Annual Meeting of Particle Accelerator Society
of Japan, August 4-6, 2010, Himeji, Japan, 2010, pp. 253–256.
The authors are grateful to Dr. Meghan Shilling and [26] R. Acero, J. Santolaria, M. Pueo, A Brau, Verification of a laser tracker with an
indexed metrology platform, Int. J. Adv. Manuf. Technol. 84 (1–4) (2016) 595–
Ms. Geraldine S. Cheok for carefully reviewing this paper. The 606.
author Ling Wang would like to thank the support from National [27] B. Muralikrishnan, D. Sawyer, C. Blackburn, S. Phillips, C. Shakarji, E. Morse, R.
Key Technologies Research & Development Program of China Bridges, Choosing test positions for laser tracker evaluation and future
Standards development, J. CMSC 6 (1) (2011) 12–19.
(2017YFC0804609), the Public Welfare Technology Application
[28] K.M. Nasr, A. Forbes, B. Hughes, A. Lewis, ASME B89.4.19 standard for laser
Research Project of Zhejiang Province Science and Technology tracker verification – Experiences and optimisations, Int. J. Metrol. Qual. Eng. 3
Department (LGG18F030010) and Key Research and Development (2012) 89–95.
Program of Zhejiang Province (2019C03114). [29] L. Wang, B. Muralikrishnan, V. Lee, P. Rachakonda, D. Sawyer, J. Gleason,
Methods to calibrate a three-sphere scale bar for laser scanner performance
evaluation per the ASTM E3125–17, Measurement 152 (2020) 1–11.
References [30] American Society of Mechanical Engineers, ASME B89.7.3.1-2001 (R2019)
Guideline For Decision Rules: Considering Measurement Uncertainty In
[1] American Society of Mechanical Engineers, ASME B89.4.19-2006: Performance Determining Conformance To Specifications, www.asme.org.
Evaluation of Laser-Based Spherical, Coordinate Measurement Systems, 2006.