IT22511 FULL STACK WEB DEVELOPMENT

UNIT II NODE JS 6
Basics of Node JS – Installation – Working with Node packages – Using Node package
manager, Creating a simple Node.js application – Using Events.

BASICS OF NODE JS

Node.js was developed in 2009 by Ryan Dahl as an answer to the frustration caused by
concurrency issues, especially when dealing with web services. Google had just come out with
the V8 JavaScript engine for the Chrome web browser, which was highly optimized for web
traffic. Dahl created Node.js on top of V8 as a server-side environment that matched the client-
side environment in the browser.
The result is an extremely scalable server-side environment that allows developers to more easily
bridge the gap between client and server. The fact that Node.js is written in JavaScript allows
developers to easily navigate back and forth between client and server code and even reuse code
between the two environments.
Node.js has a great ecosystem with new extensions being written all the time. The Node.js
environment is clean and easy to install, configure, and deploy. Literally in only an hour or
two you can have a Node.js webserver up and running.

Who Uses Node.js?

Node.js quickly gained popularity among a wide variety of companies. These companies use
Node.js first and foremost for scalability but also for ease of maintenance and faster
development. The following are just a few of the companies using the Node.js technology:
■ Yahoo!
■ LinkedIn
■ eBay
■ New York Times
■ Dow Jones
■ Microsoft

What Is Node.js Used For?

Node.js can be used for a wide variety of purposes. Because it is based on V8 and has highly
optimized code to handle HTTP traffic, the most common use is as a webserver. However,
Node.js can also be used for a variety of other web services such as:
■ Web services APIs such as REST
■ Real-time multiplayer games
■ Backend web services such as cross-domain, server-side requests
■ Web-based applications
■ Multiclient communication such as IM

What Does Node.js Come With?

Node.js comes with many built-in modules available right out of the box. This book covers
many but not all of these modules:
■ Assertion testing: Allows you to test functionality within your code.
■ Buffer: Enables interaction with TCP streams and file system operations.
1
IT22511 FULL STACK WEB DEVELOPMENT

■ C/C++ add-ons: Allows for C or C++ code to be used just like any other Node.js module.
■ Child processes: Allows you to create child processes.
■ Cluster: Enables the use of multicore systems.
■ Command line options: Gives you Node.js commands to use from a terminal.
■ Console: Gives the user a debugging console.
Crypto: Allows for the creation of custom encryption.
■ Debugger: Allows debugging of a Node.js file.
■ DNS: Allows connections to DNS servers.
■ Errors: Allows for the handling of errors.
■ Events: Enables the handling of asynchronous events.
■ File system: Allows for file I/O with both synchronous and asynchronous methods.
■ Globals: Makes frequently used modules available without having to include them first.
■ HTTP: Enables support for many HTTP features.
■ HTTPS: Enables HTTP over the TLS/SSL.
■ Modules: Provides the module loading system for Node.js.
■ Net: Allows the creation of servers and clients.
■ OS: Allows access to the operating system that Node.js is running on.
■ Path: Enables access to file and directory paths.
■ Process: Provides information and allows control over the current Node.js process.
■ Query strings: Allows for parsing and formatting URL queries.
■ Readline: Enables an interface to read from a data stream.
■ REPL: Allows developers to create a command shell.
■ Stream: Provides an API to build objects with the stream interface.
■ String decoder: Provides an API to decode buffer objects into strings.
■ Timers: Allows for scheduling functions to be called in the future.
■ TLS/SSL: Implements TLS and SSL protocols.
■ URL: Enables URL resolution and parsing.
■ Utilities: Provides support for various apps and modules.
■ V8: Exposes APIs for the Node.js version of V8.
■ VM: Allows for a V8 virtual machine to run and compile code.
■ ZLIB: Enables compression using Gzip and Deflate/Inflate.

INSTALLATION

To easily install Node.js, download an installer from the Node.js website at http://nodejs.org.
The Node.js installer installs the necessary files on your PC to get Node.js up and running. No
additional configuration is necessary to start creating Node.js applications.

Looking at the Node.js Install Location

If you look at the install location, you will see a couple of executable files and a node_modules
folder. The node executable file starts the Node.js JavaScript VM. The following list describes
the executables in the Node.js install location that you need to get started:

2
IT22511 FULL STACK WEB DEVELOPMENT

■ node: This file starts a Node.js JavaScript VM. If you pass in a JavaScript file location,
Node.js executes that script. If no target JavaScript file is specified, then a script prompt is
shown that allows you to execute JavaScript code directly from the console.
■ npm: This command is used to manage the Node.js packages discussed in the next section.
■ node_modules: This folder contains the installed Node.js packages. These packages act as
libraries that extend the capabilities of Node.js.

Verify Node.js Executables

Take a minute and verify that Node.js is installed and working before moving on. To do so, open
a console prompt and execute the following command to bring up a Node.js VM:
node
Next, at the Node.js prompt execute the following to write "Hello World" to the screen.
>console.log("Hello World");
You should see "Hello World" output to the console screen. Now exit the console using
Ctrl+C in Windows or Cmd+C on a Mac.
Next, verify that the npm command is working by executing the following command in the
OS console prompt:
npm version
You should see output similar to the following:
{ npm: '3.10.5',
ares: '1.10.1-DEV',
http_parser: '2.7.0',
icu: '57.1',
modules: '48',
node: '6.5.0',
openssl: '1.0.2h',
uv: '1.9.1',
v8: '5.1.281.81',
zlib: '1.2.8'}

Selecting a Node.js IDE

If you are planning on using an Integrated Development Environment (IDE) for your Node.js
projects, you should take a minute and configure that now as well. Most developers are particular
about the IDE that they like to use, and there will likely be a way to configure at least for
JavaScript if not Node.js directly. For example, Eclipse has some great Node.js plugins, and the
WebStorm IDE by IntelliJ has some good features for Node.js built in.
That said, you can use any editor you want to generate your Node.js web applications. In reality,
all you need is a decent text editor. Almost all the code you will generate will be .js, .json, .html,
and .css. So pick the editor in which you feel the most comfortable writing those types of files.

WORKING WITH NODE PACKAGES

One of the most powerful features of the Node.js framework is the ability to easily extend
it with additional Node Packaged Modules (NPMs) using the Node Package Manager (NPM).
That’s right, in the Node.js world, NPM stands for two things.

3
IT22511 FULL STACK WEB DEVELOPMENT

What Are Node Packaged Modules?

A Node Packaged Module is a packaged library that can easily be shared, reused, and installed in
different projects. Many different modules are available for a variety of purposes. For example,
the Mongoose module provides an ODM (Operational Data Model) for MongoDB, Express
extends Node’s HTTP capabilities, and so on.
Node.js modules are created by various third-party organizations to provide the needed features
that Node.js lacks out of the box. This community of contributors is active in adding and
updating modules.
Node Packaged Modules include a package.json file that defines the packages. The package.json
file includes informational metadata, such as the name, version author, and contributors,as well
as control metadata, such as dependencies and other requirements that the Node Package
Manager uses when performing actions such as installation and publishing.

Understanding the Node Package Registry

The Node modules have a managed location called the Node Package Registry where packages
are registered. This allows you to publish your own packages in a location where others can use
them as well as download packages that others have created.
The Node Package Registry is located at https://npmjs.com. From this location you can view the
newest and most popular modules as well as search for specific packages, as shown in Figure
3.1.

Figure 3.1 The official Node Package Modules website

USING NODE PACKAGE MANAGER

The Node Package Manager you have already seen is a command-line utility. It allows you to
find, install, remove, publish, and do everything else related to Node Package Modules. The
Node Package Manager provides the link between the Node Package Registry and your

4
IT22511 FULL STACK WEB DEVELOPMENT

development environment. The simplest way to really explain the Node Package Manager is to
list some of the commandline options and what they do. Table 3.1 lists the Node Package
Manager commands.

Searching for Node Package Modules

You can also search for modules in the Node Package Registry directly from the command
prompt using the npm search <search_string> command. For example, the following command
searches for modules related to openssl and displays the results as shown in Figure 3.2:
npm search openssl

5
IT22511 FULL STACK WEB DEVELOPMENT

Figure 3.2 Searching for Node.js modules from the command prompt

Installing Node Packaged Modules

To use a Node module in your applications, it must first be installed where Node can find it.
To install a Node module, use the npm install <module_name> command. This downloads the
Node module to your development environment and places it into the node_modules folder
where the install command is run. For example, the following command installs the express
module:
npm install express
The output of the npm install command displays the dependency hierarchy installed with the
module. For example, the following code block shows part of the output from installing the
express module.
C:\express\example
`-- express@4.14.0
+-- accepts@1.3.3
| +-- mime-types@2.1.11
| | `-- mime-db@1.23.0
| `-- negotiator@0.6.1
+-- array-flatten@1.1.1
+-- content-disposition@0.5.1
+-- content-type@1.0.2
+-- cookie@0.3.1
+-- cookie-signature@1.0.6
+-- debug@2.2.0
| `-- ms@0.7.1 ...

6
IT22511 FULL STACK WEB DEVELOPMENT

The dependency hierarchy is listed; some of the methods Express requires are cookie-signature,
range-parser, debug, fresh, cookie, and send modules. Each of these was downloaded during the
install. Notice that the version of each dependency module is listed.
Node.js has to be able to handle dependency conflicts. For example, the express module requires
cookie 0.3.1, but another module may require cookie 0.3.0. To handle this situation, a separate
copy for the cookie module is placed in each module’s folder under another node_modules
folder.
To illustrate how modules are stored in a hierarchy, consider the following example of how
express looks on disk. Notice that the cookie and send modules are located under the express
module hierarchy, and that since the send module requires mime it is located under the send
hierarchy:
./
./node_modules
./node_modules/express
./node_modules/express/node_modules/cookie
./node_modules/express/node_modules/send
./node_modules/express/node_modules/send/node_modules/mime

Using package.json
All Node modules must include a package.json file in their root directory. The package.json file
is a simple JSON text file that defines the module including dependencies. The package. json file
can contain a number of different directives to tell the Node Package Manager how to handle the
module.
The following is an example of a package.json file with a name, version, description, and
dependencies:
{
"name": "my_module",
"version": "0.1.0",
"description": "a simple node.js module",
"dependencies" : {
"express" : "latest"
}
}
The only required directives in the package.json file are name and version. The rest depend on
what you want to include. Table 3.2 describes the most common directives:

7
IT22511 FULL STACK WEB DEVELOPMENT

great way to use package.json files is to automatically download and install the dependencies for
your Node.js app. All you need to do is create a package.json file in the root of your project code

8
IT22511 FULL STACK WEB DEVELOPMENT

and add the necessary dependencies to it. For example, the following package. json requires the
express module as a dependency.
{
"name": "my_module",
"version": "0.1.0",
"dependencies" : {
"express" : "latest"
}
}
Then you run the following command from root of your package, and the express module is
automatically installed.
npm install
Notice that no module is specified in the npm install. That is because npm looks for a
package.json file by default. Later, as you need additional modules, all you need to do is add
those to the dependencies directive and then run npm install again.

CREATING A SIMPLE NODE.JS APPLICATION

Now you have enough information to jump into a Node.js project and get your feet wet. In this
section, you create your own Node Packaged Module and then use that module as a library in a
Node.js application.
The code in this exercise is kept to a minimum so that you can see exactly how to create a
package, publish it, and then use it again.

Creating a Node.js Packaged Module

To create a Node.js Packaged Module you need to create the functionality in JavaScript, define
the package using a package.json file, and then either publish it to the registry or package it for
local use.
The following steps take you through the process of building a Node.js Packaged Module using
an example called censorify. The censorify module accepts text and then replaces certain words
with asterisks:
1. Create a project folder named .../censorify. This is the root of the package.
2. Inside that folder, create a file named censortext.js.
3. Add the code from Listing 3.1 to censortext.js. Most of the code is just basic JavaScript;
however, note that lines 18–20 export the functions censor(), addCensoredWord(), and
getCensoredWords(). The exports.censor is required for Node.js applications using this module
to have access to the censor() function as well as the other two.

9
IT22511 FULL STACK WEB DEVELOPMENT

4. Once the module code is completed, you need to create a package.json file that is used to
generate the Node.js Packaged Module. Create a package.json file in the .../censorify folder.
Then add contents similar to Listing 3.2. Specifically, you need to add the name, version, and
main directives as a minimum. The main directive needs to be the name of the main JavaScript
module that will be loaded, in this case censortext.
Note that the .js is not required, Node.js automatically searches for the .js extension.

10
IT22511 FULL STACK WEB DEVELOPMENT

5. Create a file named README.md in the .../censorify folder. You can put whatever read me
instructions you want in this file.
6. Navigate to the .../censorify folder in a console window and run the npm pack command to
build a local package module.
7. The npm pack command creates a censorify-0.1.1.tgz file in the .../censorify folder. This is
your first Node.js Packaged Module.

Publishing a Node.js Packaged Module to the NPM Registry

In the previous section you created a local Node.js Packaged Module using the npm pack
command. You can also publish that same module to the NPM repository at http://npmjs.com/.
When modules are published to the NPM registry, they are accessible to everyone using the
NPM manager utility discussed earlier. This allows you to distribute your modules and
applications to others more easily.
The following steps describe the process of publishing the module to the NPM registry. These
steps assume that you have completed steps 1 through 5 from the previous section:
1. Create a public repository to contain the code for the module. Then push the contents of the
.../censorify folder up to that location. The following is an example of a Github repository URL:
https://github.com/username/projectname/directoryName/ch03/censorify
2. Create an account at https://npmjs.org/signup.
3. Use the npm adduser command from a console prompt to add the user you created to the
environment.
4. Type in the username, password, and email that you used to create the account in step 2.
5. Modify the package.json file to include the new repository information and any keywords that
you want made available in the registry search as shown in lines 7–14 in Listing 3.3.

11
IT22511 FULL STACK WEB DEVELOPMENT

6. Publish the module using the following command from the .../censor folder in the console:
npm publish
Once the package has been published you can search for it on the NPM registry and use the
npm install command to install it into your environment.
To remove a package from the registry make sure that you have added a user with rights to the
module to the environment using npm adduser and then execute the following command:
npm unpublish <project name>
For example, the following command unpublishes the censorify module:
npm unpublish censorify
In some instances you cannot unpublish the module without using the --force option. This option
forces the removal and deletion of the module from the registry. For example:
npm unpublish censorify –force
Using a Node.js Packaged Module in a Node.js Application

This section provides an example of actually using a Node.js module inside your Node.js
applications. Node.js makes this simple: All you need to do is install the NPM into your
application structure and then use the require() method to load the module.

The require() method accepts either an installed module name or a path to a .js file located
on the file system. For example:
require("censorify")
require("./lib/utils.js")
The .js filename extension is optional. If it is omitted, Node.js searches for it.
The following steps take you through that process so you can see how easy it is:

12
IT22511 FULL STACK WEB DEVELOPMENT

1. Create a project folder named .../readwords.

2. From a console prompt inside the .../readwords folder, use the following command to
install the censorify module from the censorify-0.1.1.tgz package you created earlier:
npm install .../censorify/censorify-0.1.1.tgz
3. Or if you have published the censorify module, you can use the standard command to
download and install it from the NPM registry:
npm install censorify
4. Verify that a folder named node_modules is created along with a subfolder named
censorify.
5. Create a file named .../readwords/readwords.js.
6. Add the contents shown in Listing 3.4 to the readwords.js file. Notice that a require()
call loads the censorify module and assigns it to the variable censor.
Then the censor variable can be used to invoke the getCensoredWords(), addCensoredWords(),
and censor() functions from the censorify module.

7. Run the readwords.js application using the node readwords.js command and view the output
shown in the following code block. Notice that the censored words are replaced with **** and
that the new censored word gloomy is added to the censorify module instance censor.
C:\nodeCode\ch03\readwords>node readwords
[ 'sad', 'bad', 'mad' ]
Some very *****, ***** and ***** text.
[ 'sad', 'bad', 'mad', 'gloomy' ]
A very *** day.

USING EVENTS.

Node.js applications are run in a single-threaded event-driven model. Although Node.js

implements a thread pool in the background to do work, the application itself doesn’t have any
concept of multiple threads. “Wait, what about performance and scale?” you might ask. At first it
may seem counterintuitive, but once you understand the logic behind the Node.js event model it
all makes perfect sense.
Comparing Event Callbacks and Threaded Models
In the traditional threaded web model, a request comes in to the webserver and is assigned to an
available thread. Then the handling of work for that request continues on that thread until the
request is complete and a response is sent.
Figure 4.1 illustrates the threaded model processing two requests, GetFile and GetData. The
GetFile request first opens the file, reads the contents, and then sends the data back in a response.
All this occurs in order on the same thread. The GetData request connects to the DB, queries the
necessary data, and then sends the data in the response.

13
IT22511 FULL STACK WEB DEVELOPMENT

Figure 4.1 Processing two requests on individual threads using the threaded model
The Node.js event model does things differently. Instead of executing all the work for each
request on individual threads, work is added to an event queue and then picked up by a single
thread running an event loop. The event loop grabs the top item in the event queue, executes it,
and then grabs the next item. When executing code that is no longer live or has blocking I/O,
instead of calling the function directly, the function is added to the event queue along with a
callback that is executed after the function completes. When all events on the Node.js event
queue have been executed, the Node application terminates.
Figure 4.2 illustrates the way Node.js handles the GetFile and GetData requests. The GetFile and
GetData requests are added to the event queue. Node.js first picks up the GetFile request,
executes it, and then completes by adding the Open() callback function to the event queue.
Next, it picks up the GetData request, executes it, and completes by adding the Connect()
callback function to the event queue. This continues until there are no callback functions to be
executed. Notice in Figure 4.2 that the events for each thread do not necessarily follow a direct
interleaved order. For example, the Connect request takes longer to complete than the Read
request, so Send(file) is called before Query(db).

Figure 4.2 Processing two requests on a single event-driven thread using the Node.js event model

14
IT22511 FULL STACK WEB DEVELOPMENT

Blocking I/O in Node.js

The Node.js event model of using the event callbacks is great until you run into the problem of
functions that block waiting for I/O. Blocking I/O stops the execution of the current thread and
waits for a response before continuing. Some examples of blocking I/O are
■ Reading a file
■ Querying a database
■ Socket request
■ Accessing a remote service
The reason Node.js uses event callbacks is not to have to wait for blocking I/O. Therefore, any
requests that perform blocking I/O are performed on a different thread in the background.Node.js
implements a thread pool in the background. When an event that requires blocking I/O is
retrieved from the event queue, Node.js retrieves a thread from the thread pool and executes the
function there instead of on the main event loop thread. This prevents the blocking I/O from
holding up the rest of the events in the event queue.

The function executed on the blocking thread can still add events back to the event queue to be
processed. For example, a database query call is typically passed a callback function that parses
the results and may schedule additional work on the event queue before sending a response.
Figure 4.3 illustrates the full Node.js event model including the event queue, event loop, and the
thread pool. Notice that the event loop either executes the function on the event loop thread itself
or, for blocking I/O, it executes the function on a separate thread.

The Conversation Example

To help you understand how events work in Node.js versus traditional threaded webservers,
consider the example of having different conversations with a large group of people at a party.
You are acting the part of the webserver, and the conversations represent the work necessary to
process different types of web requests. Your conversations are broken up into several segments
with different individuals. You end up talking to one person and then another. Then you go back
to the first person and then to a third person, back to the second, and so on.
This example has many similarities to webserver processing. Some conversations end quickly,
like a simple request for a piece of data in memory. Others are broken up into several segments
as you go back and forth between individuals, similar to a more complex server-side
conversation.
Still others have long breaks when you are waiting for the other person to respond, similar to
blocking I/O requests to the file system, database, or remote service.
Using the traditional webserver threading model in the conversation example sounds great at first
because each thread acts like you. The threads/clones can talk back and forth with each person,
and it almost seems as though you can have multiple conversations simultaneously.
There are two problems with this model.

15
IT22511 FULL STACK WEB DEVELOPMENT

Figure 4.3 In the Node.js event model, work is added as a function with callback to the event
queue, and then picked up on the event
loop thread. The function is then executed on the event loop thread in the case of non-blocking,
or on a separate thread in the case of blocking.
First, you are limited by the number of clones. What if you only have five clones? To talk with a
sixth person, one clone must completely finish its conversation. The second problem is the
limited number of CPUs (or “brains”) that the threads (“clones”) must share. This means that
clones sharing the same brain have to stop talking/listening while other clones are using the
brain. You can see that there really isn’t a benefit to having clones when they freeze while the
other clones are using the brain.
The Node.js event model acts more like real life when compared to the conversation example.
First, Node.js applications run on a single thread, which means there is only one of you, no
clones. Each time a person asks you a question, you respond as soon as you can. Your
interactions are completely event driven, and you move naturally from one person to the next.
Therefore, you can have as many conversations going on at the same time as you want by
bouncing between individuals. Second, your brain is always focused on the person you are
talking to since you aren’t sharing it with clones.
So how does Node.js handle blocking I/O requests? That is where the background thread pool
comes into play. Node.js hands blocking requests over to a thread in the thread pool so that it has
minimal impact on the application processing events. Think about when someone asks you a
question that you have to think about. You can still interact with others at the party while trying
to process that question in the back of your mind. That processing may impact how fast you
interact with others, but you are still able to communicate with several people while processing
the longer-lived thought.

16