A Survey of Dynamic Analysis and Test Generation

for JavaScript

ESBEN ANDREASEN, Aarhus University

LIANG GONG, University of California, Berkeley
ANDERS MØLLER, Aarhus University
MICHAEL PRADEL and MARIJA SELAKOVIC, TU Darmstadt
KOUSHIK SEN, University of California, Berkeley
CRISTIAN-ALEXANDRU STAICU, TU Darmstadt

JavaScript has become one of the most prevalent programming languages. Unfortunately, some of the unique
properties that contribute to this popularity also make JavaScript programs prone to errors and difficult for
program analyses to reason about. These properties include the highly dynamic nature of the language, a set
of unusual language features, a lack of encapsulation mechanisms, and the “no crash” philosophy. This article
surveys dynamic program analysis and test generation techniques for JavaScript targeted at improving the
correctness, reliability, performance, security, and privacy of JavaScript-based software.
CCS Concepts: • General and reference → Surveys and overviews; • Software and its engineering →
Software notations and tools; • Security and privacy → Web application security;
Additional Key Words and Phrases: Program analysis, dynamic languages, test generation
ACM Reference format:
Esben Andreasen, Liang Gong, Anders Møller, Michael Pradel, Marija Selakovic, Koushik Sen, and Cristian-
Alexandru Staicu. 2017. A Survey of Dynamic Analysis and Test Generation for JavaScript. ACM Comput.
Surv. 50, 5, Article 66 (September 2017), 36 pages.
https://doi.org/10.1145/3106739
66
1 INTRODUCTION
JavaScript has become one of the most prevalent programming languages. Originally, it was de-
signed as a simple scripting language embedded in browsers and intended to implement small
scripts that enhance client-side web applications. Since the mid-1990s, the language has evolved
beyond all expectations into one of the most prevalent programming languages. Today, JavaScript
is heavily used not only by almost all client-side web applications but also for mobile applications,

This work was supported by the European Research Council (ERC) under the European Union’s Horizon 2020 research and
innovation program (grant agreement No 647544), by the German Federal Ministry of Education and Research and by the
Hessian Ministry of Science and the Arts within “CRISP,” by the German Research Foundation within the Emmy Noether
project “ConcSys,” and by NSF grants CCF-1409872 and CCF-1423645.
Authors’ addresses: E. Andreasen and A. Moller, Aarhus University, IT-parken, Aabogade 34, DK-8200 Aarhus N, Denmark;
emails: esben@esbena.dk, amoeller@cs.au.dk; L. Gong and K. Sen, UC Berkeley, 735 Soda Hall #1776, Berkeley, CA, USA;
emails: {gongliang13, ksen}@berkeley.edu; M. Pradel, M. Selakovic, and C.-A. Staicu, TU Darmstadt, Mornewegstrasse 32,
64293 Darmstadt, Germany; emails: michael@binaervarianz.de, {m.selakovic89, cris.staicu}@gmail.com.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee
provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and
the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored.
Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires
prior specific permission and/or a fee. Request permissions from permissions@acm.org.
© 2017 ACM 0360-0300/2017/09-ART66 $15.00
https://doi.org/10.1145/3106739

ACM Computing Surveys, Vol. 50, No. 5, Article 66. Publication date: September 2017.
66:2 E. Andreasen et al.

desktop applications, and server applications.1 Many JavaScript programs are complex software
projects, including, for example, highly interactive web sites and online games, integrated develop-
ment environments, email clients, and word processors.2 The JavaScript language has been stan-
dardized in several versions of the ECMAScript language specification. At the time of this writing,
ECMAScript 7 (ECMA 2016) is the latest stable version. We refer to the language as “JavaScript”
in the remainder of this article.
Two reasons for the popularity of JavaScript are its dynamic and permissive nature. First, the
language is highly dynamic, allowing developers to write code without any type annotations, to
extend any objects, including built-in APIs, at any point in time, and to generate and load code at
runtime. Second, the language is very permissive with respect to potentially erroneous behavior.
Instead of failing an execution when other languages would, JavaScript often follows a “no crash”
philosophy and continues to execute. Even when JavaScript code produces a runtime error, for
example, when calling an undefined function, it only aborts the current event handler, allowing the
application to proceed. Many developers perceive these two properties—dynamic and permissive—
as beneficial for quickly implementing and for easily extending applications.
Unfortunately, the dynamic and permissive nature of JavaScript also causes challenges for pro-
ducing high-quality code. For example, not crashing on likely misbehavior can easily hide an error,
allowing errors to remain unnoticed even in production. As another example, dynamic code load-
ing increases the risk that attackers inject and execute malicious code. As a result, JavaScript is
known not only for its ease of use but also for being prone to correctness, efficiency, and security
problems.
To avoid or at least mitigate such problems, developers need program analyses that detect
and help understand errors. However, many traditional program analyses are not effective for
JavaScript. First, techniques developed for other widely used languages, such as Java, C, and C++,
often cannot be easily adapted because of various JavaScript-specific language features, such as
prototype-based inheritance, dynamic typing, and events. Furthermore, the tight interaction of
JavaScript code with its environment, for example, with the Domain Object Model (DOM) in a
browser, makes it difficult to adapt existing analyses for other dynamically typed languages, such
as Python. Second, static analysis, which is effective for statically typed languages, faces serious
limitations in JavaScript, because it cannot precisely reason about the many dynamic language
features. Instead, there has been a need for novel program analyses that address the challenges
created by JavaScript.
These limitations have triggered research on program analysis for JavaScript. In particular, dy-
namic analysis has proven to be an effective way to find and understand problematic code. By
reasoning about a running program, dynamic analysis avoids the challenge of statically approx-
imating behavior, a challenge that is particularly difficult for JavaScript. Since dynamic analysis
requires inputs that exercise the program, approaches for test generation have been developed.
These approaches automatically create inputs, such as sequences of UI events or input data given
to a function.
The success of dynamic analysis and test generation for JavaScript has led to an abundance of re-
search results, especially since the mid-2000s. While this development is good news for JavaScript
developers, the sheer amount of existing work makes it difficult for interested non-experts to un-
derstand the state of the art and how to improve on it. This article addresses the challenge of
summarizing the large amount of work on dynamic analysis and test generation for JavaScript.

1 See, for example, http://cordova.apache.org/, http://electron.atom.io/, http://nwjs.io/, and http://nodejs.org/.

2 See, for example, http://www.y8.com/, http://c9.io/, http://gmail.com/, and https://www.onenote.com/.

ACM Computing Surveys, Vol. 50, No. 5, Article 66. Publication date: September 2017.
A Survey of Dynamic Analysis and Test Generation for JavaScript 66:3

Fig. 1. Overview of topics covered in this article. Each white box corresponds to a section, except for topics
in dashed boxes, which are out of the scope of this article.

We present a comprehensive survey that enables interested outsiders to get an overview of this
thriving research field. In addition to presenting past work, we also outline challenges that are still
to be addressed, guiding future work into promising directions.
Figure 1 outlines this article. At first, we summarize challenges imposed by JavaScript
(Section 2). Then, we discuss dynamic analyses that help achieve four goals:

— Correctness and reliability analyses detect correctness problems and other issues that affect
the reliability of a program (Section 3).
— Security and privacy analyses detect malicious and vulnerable code (Section 4).
— Performance analyses help improve the efficiency of the program (Section 5).
— Developer productivity analyses help programmers during the development process, for ex-
ample, during program understanding and program repair (Section 6).

In addition to analyses used by developers, Section 7 discusses empirical studies performed using
dynamic analysis. Test generation approaches, in particular approaches to generate input data and
sequences of events, are the focus of Section 8. Section 9 compares different ways to implement
the approaches surveyed in this article. Finally, Section 10 discusses open research challenges and
outlines directions for future work.
Since this article focuses on dynamic analysis and test generation, other related techniques, such
as purely static analysis, formalizations and extensions of JavaScript, and manual testing, are out
of scope. In practice, the line between static and dynamic analysis is fuzzy, because an analysis
may interpret the program in a way similar but not equal to its execution on a target platform.
We define dynamic analysis as an approach that (i) executes a program on a regular execution
platform, that is, a platform that is also used for executions that do not analyze the program, and
that (ii) reasons about individual executions and not about all possible executions of the program.
There exist several surveys on techniques for JavaScript but, to the best of our knowledge, none
covers dynamic analysis and test generation in detail. An inspiring survey by Mesbah (2015) dis-
cusses testing of JavaScript-based web applications, with a focus on manual testing, test oracles,
and UI-level testing. Our article does not cover manual testing techniques but focuses on auto-
mated techniques. Li et al. (2014b) also focus on manual testing but cover neither program analysis
nor test generation. Other related work (Garousi et al. 2013; Dogan et al. 2014) performs systematic
mapping studies, which focus on a broad classification of approaches. As a contribution over all

ACM Computing Surveys, Vol. 50, No. 5, Article 66. Publication date: September 2017.
66:4 E. Andreasen et al.

these existing surveys, we provide a detailed discussion of open research challenges and possible
directions for future work.

2 CHALLENGES FOR ANALYZING JAVASCRIPT

This section discusses properties that make JavaScript particularly intricate for program analysis
(Section 2.1), explains why these properties naturally lead to dynamic analysis (Section 2.2), and
outlines challenges dynamic analysis and test generation for JavaScript face (Section 2.3).

2.1 JavaScript: An Unusual Language

Today’s popularity of JavaScript surprises considering how JavaScript came to life: In 1995, a single
engineer, Brendan Eich, designed and implemented the first version in only 10 days (Severance
2012). Initially intended to implement small scripts that enhance web sites, JavaScript now powers
applications that reach way beyond what was anticipated. JavaScript is also unusual in its choice
of language features:
— Highly dynamic. JavaScript is dynamically typed and provides numerous dynamic language
features (Section 2.2).
— Event-driven. JavaScript uses an asynchronous execution model. Programs consist of event
handlers that are triggered by user events and system events.
— Prototype-based. JavaScript is object oriented but not class based. Instead, each object has a
chain of prototype objects to share and reuse code and data.
While some of these features are also present in other widely used languages—for example, Python
is also dynamically typed—those languages that have received most attention by the program
analysis community, Java and C, do not provide them.
Another distinguishing property of JavaScript is to be embedded into complex and heteroge-
neous environments. The most prevalent environment is a browser, where JavaScript interacts
with multiple other languages, in particular, HTML and CSS, and with the DOM. Beyond the
browser, server-side JavaScript code typically runs on the Node.js platform, whose API differs
from the browser API. Likewise, hybrid mobile applications are typically embedded into an envi-
ronment that allows for interactions with the mobile device, such as Apache Cordova. Analyzing
the interactions of a program with one or more of these environments is challenging. The challenge
is compounded by the fact that even environments of the same kind, such as different browsers,
often vary in subtle ways across implementations and versions.
Since its creation in 1995, JavaScript has grown considerably by adding new language features.
Due to backward compatibility concerns, many of the less-fortunate design choices made in the
early stages cannot be undone. This increasing complexity of the language also increases the effort
required to build analysis tools.

2.2 Why is Dynamic Analysis So Widely Used for JavaScript?

The highly dynamic nature of JavaScript makes it less amenable for static analysis but a well-
suited target for dynamic analysis. In the following, we discuss some of the dynamic features of
the language and their implications for program analysis.
Types. JavaScript is dynamically typed, that is, there are no type annotations in the source code.
The language uses “duck typing,” meaning that whether an object is suitable for an operation de-
pends only on the properties of the object but not on its nominal type. Furthermore, JavaScript
heavily uses implicit type conversions, where a runtime value is converted from one type to an-
other to enable otherwise impossible operations. All these features make it difficult for a purely

ACM Computing Surveys, Vol. 50, No. 5, Article 66. Publication date: September 2017.
A Survey of Dynamic Analysis and Test Generation for JavaScript 66:5

static analysis to approximate types with reasonable precision, a challenge addressed by various
static analyses (Thiemann 2005; Jensen et al. 2009; Guha et al. 2011) not covered in this article.
Properties and Functions. JavaScript code may dynamically add and remove properties of an
object, making it difficult to statically determine which properties are available at a particular
program point and to what these properties refer. Since functions are values in JavaScript, the
difficulties of statically reasoning about properties also apply to functions. An additional challenge
is that a program may dynamically compute a property name before accessing the property. The
dynamic nature of properties and functions cause severe difficulties for static analysis (Sridharan
et al. 2012; Andreasen and Møller 2014).
Code Loading and Code Generation. Many JavaScript applications load parts of the code at run-
time, for example, via the <script> tag of an HTML page, which causes the browser to re-
trieve code from a server, or programmatically via an XMLHttpRequest, which loads code asyn-
chronously. Furthermore, JavaScript allows a program to interpret a string as code, and thereby to
generate and dynamically load code, for example, via the built-in eval function. These two prop-
erties challenge the common assumption of static analysis that the code of the program is readily
available.
While these language features challenge static analysis, they reduce to non-issues in dynamic
analysis. Because a dynamic analysis observes concrete values and concrete types, it does not suffer
from dynamic typing and dynamic usages of properties and functions. Likewise, dynamic analysis
is oblivious to dynamic code loading, because it analyzes the program’s execution once the source
code has been loaded. On the flip side, dynamic analysis is inherently limited by the executions it
observes and therefore cannot provide soundness guarantees for all possible executions. If sound-
ness is a requirement, for example, for verifying the absence of security vulnerabilities, then static
analysis is the more suitable choice. In this article, we focus on dynamic analyses.

2.3 Challenges for Dynamic Analysis and Test Generation

Even though the dynamic features of JavaScript make the language amenable for dynamic analysis,
some challenges remain.
“No crash” Philosophy. JavaScript follows a “no crash” philosophy, aiming for running a program
without showing any obvious signs of misbehavior, such as a crash, to the user. For example,
JavaScript remains silent while executing obviously incorrect operations, such as multiplying an
array with a string. The absence of obvious signs of misbehavior makes it difficult for a dynamic
analysis to distinguish intended from unintended behavior. Nevertheless, various ways to identify
misbehavior have been proposed, as discussed in Sections 3, 4, 5, and 8.
Nondeterminism. The heavy interactions of JavaScript applications with users and the network
lead to a high degree of hard-to-control inputs. Since dynamic analysis focuses on individual exe-
cutions triggered by specific inputs, it may miss behavior triggered by other possible inputs. This
input dependence is compounded by several non-traditional sources of nondeterminism, for ex-
ample, asynchronously scheduled events whose execution order is nondeterministic. As a result,
repeatedly analyzing a program execution driven by a single input may lead to multiple behaviors.
We discuss techniques to handle nondeterminism and input dependence, in particular dynamic
data race detectors for JavaScript (Section 3) and record-and-replay systems (Section 6).
Event-driven. Due to the event-driven nature of JavaScript programs, only specific sequences of
input events may be able to reach a particular region of code. Creating such sequences of events
is different from traditional input data generation. Test generation (Section 8) addresses this chal-
lenge.

ACM Computing Surveys, Vol. 50, No. 5, Article 66. Publication date: September 2017.
66:6 E. Andreasen et al.

Despite its dynamic nature and sometimes unusual semantics, JavaScript has several properties
that make testing and analyzing programs easier than in many other languages. First, the language
is essentially single-threaded and therefore does not suffer from concurrency issues caused by
shared-memory multi-threading.3 Second, since source code is the only widely used format to
distribute JavaScript programs, there is no need to handle compiled code. Code obfuscation, which
is commonly used to distribute proprietary code, does not significantly affect program analyses,
because obfuscation is semantics preserving. Finally, testing and analyzing partial code is easier
than in statically compiled languages, since an incomplete program may still run.

3 CORRECTNESS AND RELIABILITY

Correctness and reliability are two of the most important challenges faced by JavaScript develop-
ers. Correctness here means that a program conforms to its specification. Reliability means the
extent to which a software system delivers usable services when those services are demanded.
Because of the “no crash” philosophy and the fact that typical JavaScript programs do not come
with a correctness specification, there is no clear definition of what constitutes correct execution.
Even runtime errors, for example, dereferencing null, reading from an undeclared variable, or call-
ing a non-function value, may be benign, since uncaught exceptions terminate only the current
event handler but not the application. Moreover, JavaScript is memory-safe (i.e., it has automatic
garbage collection, arrays cannot be accessed out-of-bound, and there is no pointer arithmetic), so
programming errors cannot lead to memory corruption. All this makes the problem of ensuring
the correctness and reliability of JavaScript programs different from programs in other popular
languages, such as Java and C.
A significant amount of work tries to alleviate correctness issues by detecting them before de-
ploying the software. This section presents such techniques based on dynamic analyses. Specifi-
cally, we present approaches that address code smells and bad coding practices (Section 3.1), that
target cross-browser issues (Section 3.2), and that detect data races (Section 3.3).

3.1 Code Smells, Bad Coding Practices, and Program Repair

Code smells indicate potential quality issues in the program. To deal with code smells, the soft-
ware development community has learned over time guidelines and informal rules to help avoid
common pitfalls of JavaScript. Those rules specify which language features, programming idioms,
APIs, and so on, to avoid or how to use them correctly. Following these rules often improves soft-
ware quality by reducing bugs, increasing performance, improving maintainability, and preventing
security vulnerabilities.
Lintlike tools are static checkers that enforce code practices and report potential code smells.
Unfortunately, due to the dynamic nature of JavaScript, approaches for detecting code smells stat-
ically are limited in their effectiveness. Although there are some successful static checkers used in
real-world application development (e.g., JSHint, JSLint, ESLint, and Closure Linter), they are lim-
ited by the need to approximate possible runtime behavior and often cannot precisely determine
the presence of a code smell. To address this issue, JSNose (Fard and Mesbah 2013) and DLint (Gong
et al. 2015b) dynamically detect code smells missed by existing static lintlike tools.
JSNose (Fard and Mesbah 2013) combines static analysis and dynamic analysis to detect code
smells in web applications. The approach mostly focuses on code smells at the level of closures, ob-
jects, and functions. For example, JSNose warns about suspiciously long closure chains, unusually
large functions, the excessive use of global variables, and not executed and therefore potentially
dead code. In contrast, DLint (Gong et al. 2015b) detects violations of coding practices at the level

3 Data races exist nevertheless, as we discuss in Section 3.3.

ACM Computing Surveys, Vol. 50, No. 5, Article 66. Publication date: September 2017.
A Survey of Dynamic Analysis and Test Generation for JavaScript 66:7

of basic JavaScript operations, such as local variable reads and writes, object property reads and
writes, and function calls. The approach monitors these operations and detects instances of bad
coding practices by checking a set of predicates at runtime. Gong et al. also report an empirical
study that compares the effectiveness of DLint’s checkers and their corresponding static checkers
in JSHint. The result of the study suggests that dynamic checking complements static checkers:
Some violations of coding practices can be detected only by a dynamic checkers, for example, be-
cause the violation depends on a runtime type, whereas other violations are bound only by a static
checker, for example, because it depends the syntactic structure of the code.
TypeDevil (Pradel et al. 2015) addresses a particular kind of bad practice: type inconsistencies.
They arise when a single variable or property holds values of multiple, incompatible types. To find
type inconsistencies, the analysis records runtime type information for each variable, property,
and function and then merges structurally equivalent types. By analyzing program-specific type
information, TypeDevil detects problems missed by checkers of generic coding rules, such as DLint
and JSNose.
Some analyses not only detect problems but also help fixing them. One such analysis, Eval-
orizer (Meawad et al. 2012), replaces unnecessary uses of the eval function with safer alternatives.
Using this function is discouraged, because its subtle semantics is easily misunderstood, because it
has a negative performance impact, and because eval may enable attackers to execute untrusted
code. Evalorizer dynamically intercepts arguments passed to eval and transforms the eval call to
a statement or expression without eval, based on a set of rules. The approach assumes that a call
site of eval always receives the same or very similar JavaScript code as its argument.
Vejovis (Ocariza Jr. et al. 2014) generates fixes for bugs caused by calling the DOM query API
methods, for example, getElementById, with wrong parameters. The approach identifies possible
bugs by dynamically checking whether a DOM query returns abnormal elements, for example,
undefined, or whether there is an out-of-range access on the returned list of elements. Based
on the observed symptoms and the current DOM structure, Vejovis suggests fixes, such as passing
another string to a DOM method or adding a null/undefined check before using a value retrieved
from the DOM.

3.2 Cross-Browser Testing

Because supported language features and their implementation may differ across browsers, cross-
browser compatibility issues challenge client-side web applications. Such issues are caused by
ambiguities in the language specification or by disagreements among browser vendors. Incompat-
ibilities often lead to unexpected behavior.
CrossT (Mesbah and Prasad 2011) detects cross-browser issues by first crawling the web applica-
tion in different browsers to summarize the behavior into a finite-state model and then finding in-
consistencies between the generated models. WebDiff (Choudhary et al. 2010a, 2010b) structurally
matches components in the DOM trees generated in different browsers and then computes the
visual difference of screenshots of the matching components. In addition to comparing captured
state models and comparing visual appearances, CrossCheck (Choudhary et al. 2012) incorpo-
rates machine-learning techniques to find visual discrepancies between DOM elements rendered
in different browsers. Based on their previous work, Choudhary et al. further proposed X-PERT
(Choudhary et al. 2013, 2014b), which detects cross-browser incompatibilities related to the struc-
ture and content of the DOM and to the web site’s behavior. The tool compares the text con-
tent of matching components and the relative layouts of elements by extracting a relative align-
ment graph of DOM elements. As X-PERT addresses limitations of previous work by the same au-
thors, it seems to be the most comprehensive approach for detecting cross-browser issues. FMAP
(Choudhary et al. 2014a; Choudhary 2014) aims at detecting missing features between the desktop

ACM Computing Surveys, Vol. 50, No. 5, Article 66. Publication date: September 2017.
66:8 E. Andreasen et al.

version and the mobile version of a web application by collecting and comparing their network
traces. The approach assumes that the web application uses the same back-end functionality while
having specific customizations in the front-end.

3.3 Dynamic Race Detection

Even though JavaScript programs execute in a single thread and each event handler runs without
preemption, there may be races, as known from concurrent programs. The reason is that the or-
dering of responses to asynchronous requests, timer events, script and frame loading, and HTML
parsing is not fully controlled by the developer, which may introduce nondeterminism. For ex-
ample, a common mistake is to assume that all HTML parsing has completed before any user
event handlers are executed or that network communication is synchronous. As a consequence,
users with a slow network connection or a different browser than the developer may experience
UI glitches and data corruption. Because JavaScript provides no built-in mechanisms for concur-
rency control programmers resort to ad hoc synchronization, such as timer events that repeatedly
postpone processing until some flag is set by another event handler. The problem with races in
JavaScript was first described by Steen (2009) and Ide et al. (2009). An early static approach to
detect races (Zheng et al. 2011) faces not only the challenges with static analysis for JavaScript,
but also its practicability is limited by not taking the possible event orderings into account.
WebRacer, by Petrov et al. (2012), pioneered dynamic race detection for JavaScript. In particular,
they characterize the happens-before relation that models the causal relationship of events. The
results from WebRacer show that races are prevalent, even in widely used web sites. However, not
all races correspond to errors. First, ad hoc synchronization inevitably causes races although its
purpose is to prevent errors. Second, for event handlers that commute, different event orderings
may lead to the same state. Third, even when different event orderings lead to different states,
those differences may be benign, for example, if they do not affect the UI or network data. For
this reason, much of the later work has focused on improving precision. The follow-up tool Even-
tRacer (Raychev et al. 2013) introduced the notion of race coverage to filter many harmless races
caused, for example, by ad hoc synchronization.
Instead of looking for races per se, Wave (Hong et al. 2014) heuristically explores different event
schedules and checks whether the final DOM states differ. While exploring schedules, Wave re-
spects the happens-before relation and does not change the sequence of user events. The potential
advantage is that it directly targets the consequences of the races. However, as noted by Jensen
et al. (2015a), event handlers influence each other, so aggressively reordering events to provoke
races often leads to infeasible event sequences, in which case Wave falsely reports errors.
The R4 algorithm by Jensen et al. (2015a) aims to systematically explore different event sched-
ules to discover the consequences of races. It takes a sequence of events from an initial execution
as input. Unlike Wave, R4 uses conflict-reversal bounding and approximate replay for reducing
divergence from the initial execution. Moreover, by adapting dynamic partial order reduction, R4
avoids exploring many equivalent schedules. The approach significantly reduces the number of
false positives compared to Wave but overall still suffers from a non-negligible amount of false
positives.
Mutlu et al. (2015) argue that most races are harmless and focus on those that affect persistent
storage, either on the client or on the server. Instead of actually executing alternative event sched-
ules like Wave and R4 , their approach is to perform a lightweight static analysis on the possible
schedules for the given execution to see whether persistent storage may be affected.
Although the problem with races is by now well described and multiple detection techniques
have been developed, the accuracy of the available tools are not yet sufficient for production use.

ACM Computing Surveys, Vol. 50, No. 5, Article 66. Publication date: September 2017.
A Survey of Dynamic Analysis and Test Generation for JavaScript 66:9

It remains an open question how to automatically and effectively detect races that are harmful and
explain their consequences to developers.

4 SECURITY AND PRIVACY

JavaScript powers many applications with strong security and privacy requirements, such as on-
line banking, shared editing of private documents, and e-commerce. As a result, various program
analyses address security and privacy issues. These efforts are a response to the limitations of the
current security mechanisms: They either relax these mechanisms or enhance them by defending
against newly identified threats. As surveyed in previous work (Ryck et al. 2010), the goals of the
research community with respect to the security of JavaScript applications are manifold: separate
scripts either from each other or from the underlying system, mediate interactions between scripts
and communication of scripts with other entities, and fine-grained control of behavior. Consider-
ing the large amount of JavaScript-related work on security and privacy, this section focuses only
on the last category. Specifically, we discuss dynamic information flow analyses, an area that has
received considerable attention from the program analysis community, in particular for JavaScript.
Information flow analysis is a heavyweight technique that reasons about each executed instruc-
tion. The analysis assigns security labels to values and propagates these labels through the pro-
gram. A label expresses the security level of the value, for example, how secret a value is. The
analysis checks whether the flows of values complies with a security policy that specifies to where
values with a particular label are allowed to flow. Depending on the security policy, information
flow analysis can detect different kinds of vulnerabilities and malicious behavior. Unless explicitly
specified, the analyses discussed here can be used with various policies. For a general survey of in-
formation flow techniques for other languages, the reader is directed to Sabelfeld and Myers (2003).

4.1 Taint Analysis

At first, we discuss work on taint analysis, a lightweight form of information flow analysis that con-
siders only data flows. All analyses discussed in this subsection address cross-site scripting (XSS)
vulnerabilities. These are vulnerabilities where user-controlled inputs may modify the DOM, for
example, by adding a script tag with additional JavaScript code. Lekies et al. (2013) conduct a
study that detects thousands of DOM-based XSS vulnerabilities across the Alexa top 5,000 web
sites. To validate each detected vulnerability, an exploit is generated. The prevalence of sophisti-
cated DOM-based vulnerabilities observed in popular web sites demands more powerful mecha-
nisms for protecting the users. The traditional way to detect these problems is to use string-based
XSS filters. Stock et al. (2014) describe and address the inability of the existing XSS filters to de-
fend against DOM-based XSS attacks. They propose a dynamic taint engine that tracks the origin
of strings and prevents the interpretation of user-provided input into tokens other than literals.
This restrictive policy leads to a low false-positive rate, and it detects 5 times more true positives
than traditional, string-based XSS filters.
Hybrid taint analysis approaches combine static and dynamic analysis. Partial evaluation using
the recorded dynamic data is proposed by Tripp et al. (2014) as a way to increase the precision of a
static analysis. Their analysis rewrites DOM accesses into abstractions that the static taint analysis
can reason about. The approach significantly reduces false positives compared to a purely static
analysis. Another hybrid approach (Wei and Ryder 2013) guides static analysis with dynamically
recorded traces. In particular, this technique makes dynamically generated code visible to the static
analysis. The approach detects vulnerabilities that purely static taint analysis misses, while being
more efficient. However, it is unknown how the hybrid approach compares to purely dynamic
approaches. The two presented hybrid solutions differ in the information they collect at runtime
but share the idea of providing dynamically gathered information to a static analysis.

ACM Computing Surveys, Vol. 50, No. 5, Article 66. Publication date: September 2017.
66:10 E. Andreasen et al.

Once a taint analysis detects that a flow violates the given security policy, a major challenge is to
decide whether the violation is a security problem or a false positive. Saxena et al. (2010b) address
this challenge by combining a taint analysis with fuzzing to generate attack strings against poten-
tial validation vulnerabilities. The analysis is performed on a trace that summarizes the JavaScript
execution in a language with simpler semantics. The candidate exploits are further validated using
a browser-based oracle.

4.2 Information Flow Analysis

Dynamic information flow analysis for JavaScript has received significant attention, because sev-
eral language features make this problem particularly hard: how to reason about the large number
of non-JavaScript APIs (DOM and other web APIs), the prevalence of eval, prototype inheritance,
and the special scoping rules of with blocks. Existing work aims at tackling subsets of these prob-
lem, while keeping the overhead and the implementation effort at an acceptable level. One addi-
tional, language-independent challenge for dynamic information flow analyses is how to handle
flows caused by not executing a branch. In this work, we refer to this type of flows as hidden flows.
They represent an additional information flow channel created by the runtime labels specific to
purely dynamic analyses (Sabelfeld and Myers 2003).
Austin and Flanagan (2010) introduced the first purely dynamic information flow analysis, called
a monitor, for a core of JavaScript. It provides two monitoring strategies, No Sensitive Upgrade
(NSU) and Permissive Upgrade (PU), both ensuring non-interference. These strategies prevent im-
plicit leaks of information by stopping the execution of the program whenever the monitor reaches
an unsafe state. To enhance the permissiveness of these techniques, developers may insert upgrade
operations that aid the monitor from stopping the execution unnecessarily. Hedin and Sabelfeld
(2012) have generalized these ideas to a larger subset of JavaScript that includes objects, higher-
order functions, exceptions, and dynamic code evaluation. Hedin et al. (2014) implemented this
approach in a custom interpreter that supports the full language, with models for built-in APIs
and the browser environment. Chudnov and Naumann (2015) proposed a rewriting-based, inlined
monitor that implements the NSU policy for full ECMAScript 5 with web support. The evaluation
shows that the monitor is able to run one order of magnitude faster than the one by Hedin et al.
(2014) and that it can enforce information flow control on synthetic mashups. However, since the
inliner is implemented in Haskell, it cannot protect against dynamically generated code. The above
approaches rely on upgrade operations that improve the permissiveness of the information flow
analysis. An approach for automatically inserting upgrade statements (Birgisson et al. 2012) com-
bines testing and program rewriting for transforming hidden flows into explicit flows. The main
drawback is that the permissiveness of the monitor depends on the completeness test suite.
A completely different approach (Austin and Flanagan 2012) takes inspiration from secure multi-
execution (Devriese and Piessens 2010), a technique for enforcing security policies through mul-
tiple, concurrent runs of the program. The key idea are faceted values, that is, runtime objects that
contain multiple alternative values, one for each security level. Each action in a program is associ-
ated with a principal that corresponds to a security level. Whenever an output operation is reached,
the facet corresponding to the principal’s security level is passed to the sink. The technique han-
dles hidden flows by construction, and, for simple policies and large numbers of principals, it scales
better than the original work on secure multi-execution.
Bichhawat et al. (2014) describe a hybrid information flow analysis that handles hidden flows. To
deal with unstructured control flow, the approach performs an on-demand static analysis. Chugh
et al. (2009) propose another hybrid method in which a static information flow analysis is first
performed on the server. This stage outputs a set of constraints to be checked on the client side for
the dynamically loaded code, reducing the client-side overhead. This reduction comes at a small

ACM Computing Surveys, Vol. 50, No. 5, Article 66. Publication date: September 2017.