Network Programming and Applications

 The Application Layer

 Socket Programming
 Application Protocols: DNS, FTP
 Application Protocols: SMTP, HTTP
 Remote Procedure Calls
 RPC Applications: NFS, NIS
 Web Applications: Search, Active Contents
 Distributed Objects: CORBA
 Client-Server Applications
Typical network app has two application
pieces: client and server transport
network
link
Client: physical Reply
 initiates contact with server; requests network
service link
Request physical
Server:
 provides requested service to client
Middleware: application
 provides a set of higher-level API to transport
ease the task of programming network
link
distributed applications
physical
 Distributed Application Design

Middleware support for distributed applications has two approaches:

 Communication Oriented:
– begin with the communication protocol
– design a message format and syntax
– design client and server by specifying how each reacts to incoming
messages and how each generates outgoing messages
– Example: sockets
– can be restrictive for the application designer
 Application Oriented:
– design a conventional application
– divide the application into multiple parts (client/server)
– add communication protocols that allow each to execute on a separate
machine
– example: RPC
– can be restrictive for programming
 Remote Procedure Call (RPC)

 Suggested by Birell and Nelson in 1984

 Goal: make distributed programming as simple as possible
– distributed programming should be similar to normal
sequential programming
 Distributed programming: to get something done, send a
message to the server, i.e., do I/O
 Sequential programming: to get something done, call a
function
 RPC tries to bridge this gap
 RPC: Central Idea

 Allow programs to call procedures located on other

machines
– when program on machine A calls a procedure on machine B, the
calling process on A is suspended
– the execution of the called procedure takes place on B
– when the procedure returns, the process on A resumes
 Information exchange takes place via procedure arguments
and return value
 RPC Functionality

Client (caller) Server (callee)

Request Message
call procedure and (contains remote procedure’s parameters)
wait for reply
receive request and start
procedure execution

procedure executes

send reply and wait

for next request
resume execution Reply Message
(contains result of procedure execution)
 RPC: Advantages

 Extends the conventional procedure call to the client/server

model
 Remote procedures accept arguments and return results
 Makes it easy to design and understand programs
 Helps programmer to focus on the application instead of
the communication protocol
 Allows a client to execute procedures on other computers
 Simplifies the task of writing client/server programs

RPC forms the foundation for many distributed utilities used

today, like NFS and NIS
 IPC v/s RPC

 IPC (Inter Process Communication)

– based on message passing
– uses low level communication primitives, send and receive
– has overhead of multiple system calls and copying of each
message; from process space to kernel space and vice-versa
– needs to be specific for each application; increases programming
effort
 RPC (Remote Procedure Call)
– provides a higher level interface
– provides programmers with a familiar procedural interface
– avoids the overhead of multiple system calls
– has a well-defined mechanism that supports compile-time type
checking and automatic interface generation
 RPC: Design Issues

 Transparency:
– Syntactic Transparency:
• RPC should have same syntax as local procedure call
– Semantic Transparency:
• RPC semantics should be identical to local procedure call
 Standard Representation:
– client machine may be little-endian and server machine may be big-endian
– one machine may be using ASCII and the other EBCDIC
– representation of floating point numbers on the two machines may be
different
– n client architectures and m server architectures => n * m versions
– need for eXternal Data Representation (XDR) for all data types

Need for stub procedures to do the necessary translation

 RPC: Overview

Client Server

Client Procedure Called Procedure

arguments results

Client Stub Server Stub

Network transport Network Transport

Network
 RPC: Communication Issues

 Call semantics:
– Possibly or maybe: No retry of call; no certainty of results
– At-least once: Retry implemented but no duplicate filtering
– Exactly once: Retry and duplicate filtering are implemented
– Others: At-most once; last-one call semantics etc

 Communication protocol:
– Request (R): No value returned; no acknowledgement
– Request-Reply (RR): Replies are used as acknowledgements
– Request-Reply-Acknowledge (RRA): Client specifically
acknowledges each reply from server
 RPC: Parameter Passing

stubs do marshalling (packing) of arguments and results

 call-by-reference: passing pointers (addresses) does not make sense;
client and server execute in different address spaces
 call by copy-in/copy-out: pass value of the variable pointed to; copy
the returned value back into the variable
void myfunc (int *x, int *y) { (*x)++; (*y)++; }
main() { int a = 1; myfunc (&a, &a);}
– normal execution: value of a after the call will be 3
– if myfunc is a remote procedure, to which arguments are passed by copy-
in/copy-out, the result will be 2!
 communication cannot be through global variables
 all arguments are marshalled into a structure and a single argument is
passed
 call-by-value is the most common semantics
 RPC: Implementation

Client Machine Server Machine

Client Server
Call Return Return execute Call

args results results args

Client Stub Server Stub
Pack Unpack Pack Unpack

RPC Runtime RPC Runtime

Dispatch
Send wait Receive Send Receive

return packets

call packets
 Client Stub Functions

 Pack the arguments into a message (also known as

parameter marshalling)
 Send the message to the server and wait for reply
 Upon reply, extract the return value from the message
 Do a normal return
 To generate the client stub, one needs to know the
argument and return value types
 Client program will have one stub for each remote
procedure
 Server Stub Functions

 Wait for a message

 Extract information about which server procedure has been
called (one server may ``export'' several procedures)
 Unpack the parameters and call the requested procedure
 Upon procedure return, pack the return value in a message
 Send message back to client
 To generate the server stub, one needs to know the
parameter and return value types of the exported
procedure
 Typically there is one server stub per exported procedure;
sometimes a single stub may serve many procedures
 RPC Runtime Functions

 Relieves client/server stubs from handling low-level

network communication
 Handles transmission of messages across the network
between client and server
 Performs the necessary application-level retransmissions,
acknowledgements, encryption etc
 RPC: Client-Server Binding

 Server exports the interface name and registers the service

port with the portmapper (binder); Server also registers the
RPC program numbers, and versions
 Client imports the interface and performs a lookup with the
portmapper to get the service port
 Client and server open a communication path to execute
remote procedures

 Binder may be available at a fixed host address or

client/server may use broadcast to locate the binder
 Binding may be at compile-time, link-time or call-time
 RPC Working
Client Machine Server Machine

Portmapper

2 1

Client 3 Server
Program Program

Protocol port # (16 bits)  RPC port # (32 bits)

 RPC: Exceptions and Recovery
 Exception handling:
– Client stub raises exception
– Client routine provides procedure for handling the exception
 Crash recovery:
– Server crash:
• retransmit request
• procedure may get executed twice when server recovers!
• at-least-once or at-most-once semantics
• stateful v/s stateless servers
– Client crash:
• computation being done in a server on behalf of such a client
(called an orphan) is useless
• replies may confuse the client if it reboots quickly
 Most implementations leave the tasks of authentication and
data encryption to the user
 Network Programming and Applications

 The Application Layer

 Socket Programming
 Application Protocols: DNS, FTP
 Application Protocols: SMTP, HTTP
 RPC Programming
 RPC Applications: NFS, NIS
 Web Applications: Search, Active Contents
 Distributed Objects: CORBA
 Sun RPC

 First commercial implementation of RPC was in SunOS

 de facto standard; Part of almost all Unix systems now
 Parameter passing only by copy-in (no copy-out)
 Return value is the only way of passing information back
 At least once semantics, no handling of orphans
 RPC programs do not use well known protocol ports.
Instead they use dynamic binding (portmapper) that allows
each RPC program to choose an arbitrary unused port
 Uses a data representation format known as XDR
(eXternal Data Representation)
 The XDR Library
 The XDR library provides a conventional way of translating built-in C
data types and pointer-based structures into an external bit-string
representation
 The XDR library has primitive and complex filters:
– Primitive XDR Filters:
xdr_int
xdr_char
xdr_long
xdr_float
xdr_void
xdr_enum
– Complex XDR Filters:
xdr_array : variable-length array with arbitrary element size
xdr_bytes : variable-length array of bytes
xdr_opaque : Fixed length data (uninterpreted)
xdr_reference : Object references
xdr_string : Null-terminated character arrays
 RPC Programming

There are three levels at which you can write RPC programs:
 Using library stubs: User makes simple library routines
available for others which hides low-level details
 Using RPCGEN: RPCGEN generates client and server
stubs automatically. There are several details that cannot
be easily controlled(for example, the number of retries in
case of timeout)
 Using RPC RunTime library: This is provided by SUN
and has the maximum flexibility and efficiency. This is
however relatively difficult to use
Most of the programming is done using RPCGEN
 RPCGEN: Protocol Compiler

 Not much is gained if the programmer has to write the

client and the server stubs
 Protocol specification (procedure names, argument types,
return type etc.) is specified in RPCGEN language (IDL)
 RPCGEN generates client, server stubs, and XDR routines
which do parameter packing and unpacking
 Programmer has to write:
– Client main program
– Implementation of the procedure (for the server)
– Compile the client program with the client stub, and procedure
implementation with the server stub, into two independent client
and server programs
 RPC Specification

RPC
Compiler

Shared Filters
Client and Server
Stub Header Files Stub
Compile Compile
and and
Link RPC and Data Link
Client Server
Representation
Functions Functions
Libraries
Client Server
Executable Executable
 Compiling the Protocol Definition

rpcgen file.x

client stub: file_clnt.c

server stub: file_svc.c
XDR filters: file_xdr.c

Parameter Marshalling
– standard representation for basic data types such as int, char, float..
– standard ways of packing compound types
• For example: arrays will be packed as: size, e_0, e_1, e_2,…
– given these conventions, the stubs on either side can pack/unpack
information correctly
 RPC Example: StatelessFS.x
/* Interface definition for a stateless file service (StatelessFS.x) */
const File_Name_Size = 16
const Buffer_Size = 1024

typedef string FileName<File_Name_Size>;

typedef long Position;
typedef long Nbytes;

struct Data { long n; char buffer[Buffer_Size]; };

struct readargs {FileName filename; Position position; Nbytes n; };
struct writeargs {FileName filename; Position position; Data data; };

program Stateless_FS {
version Stateless_FS_Vers {
Data READ (readargs) = 1; /* first procedure */
Nbytes WRITE (writeargs) = 2; /* second procedure */
} = 1; /* version number */
} = 0x20000000; /* unique identifier; program number */
 RPC Example: client.c
/* Client program for stateless file service (client.c) */
#include <stdio.h>
#include <rpc/rpc.h>
#include “Stateless_FS.h”
main (argc, argv)
int argc; char **argv;
{
CLIENT *client_handle;
char *server_host_name = “akash”;
readargs read_args;
writeargs write_args;
Data *read_result;
Nbytes *write_result;

client_handle = clnt_create (server_host_name, Stateless_FS,

Stateless_FS_Vers, “udp”); /* Get a client handle; creates a socket */
if (client_handle == NULL) {
clnt_pcreateerror (server_host_name);
return(1); /* Cannot contact server */
};
 RPC Example: client.c (contd)

….
/* Prepare parameters and make an RPC to read procedure */
read_args.filename = “example”;
read_args.position = 0;
read_args.n = 500;
read_result = read_1 (&read_args, client_handle);
….
/* Similar code for RPC to write procedure */
….
clnt_destroy (client_handle); /* destroy client handle; close socket */
}
 Client Calls
 clnt_create():
CLIENT *clnt_create(host, prognum, versnum, protocol)
char *host; u_long prognum, versnum; char *protocol;
– creates the CLIENT structure for the specified server host, program, and
version number
– can use tcp or udp protocol
 clnt_destroy():
clnt_destroy(clnt)
CLIENT *clnt;
– deallocates the memory associated with CLIENT.
 clnt_control():
– used to change or retrieve the characteristics of an RPC, such as the
timeout value, socket descriptors, and status
 clnt_call():
enum clnt_stat clnt_call(clnt, procnum, inproc, in, outproc, out, timeout)
– used by the client to make an RPC call
– It is blocking and uses a timeout value
 RPC Example: server.c
/* Server program for stateless file service (server.c) */
#include <stdio.h>
#include <rpc/rpc.h>
#include “Stateless_FS.h”

/* READ PROCEDURE */
Data *read_1 (args)
readargs *args;
{
static Data result; /* Must be declared as static */

/* Statements for reading args.n bytes from the file args.filename starting
from position args.position, and for putting the data read in &result.buffer
and the actual number of bytes read in result.n */

return (&result); /* Return the result as a single argument */

}
 RPC Example: server.c (contd)

….
/* WRITE PROCEDURE */
Nbytes *write_1 (args)
writeargs *args;
{
static Nbytes result;

/* Statements for writing args.data.n bytes from the buffer &args.data.buffer

into the file args.filename starting from position args.position, and for
putting the actual number of bytes written in result */

return (&result);
}
 Server Calls

 svctcp_create():
SVCXPRT *svctcp_create(sock, sendz, recvz)
int sock; u_int sendz, recvz;
– Creates and returns a TCP RPC service transport at a particular socket
– TCP-based RPC uses buffered I/O

 svcudp_create():
SVCXPRT *svcudp_create(sock);
int sock;
– Creates and returns a pointer to a UDP/IP- based server transport
– sock may be either an open socket or descriptor or RPC_ANY_SOCK
– The socket is bound to an arbitrary local UDP port
 Generating the Client and Server

• Generating the Client:

• Compile the client code:
• cc -c -o client.o -g client.c
• Compile the client stub:
• cc -c StatelessFS_clnt.c
• Compile the XDR filters:
• cc -c StatelessFS_xdr.c
• Build the client executable:
• cc -o Client client.o StatelessFS_clnt.o StatelessFS_xdr.o

 Generating the Server:

• Compile the service procedures:
• cc -c -o server.o server.c
• Compile the server stub:
• cc -c StatelessFS_svc.c
• Build the server executable:
• cc -o Server server.o StatelessFS_svc.o StatelessFS_xdr.o
 High-Level RPC (server side)
 int registerrpc (prognum, versnum, procnum, procname, inproc, outproc)
u_long prognum, versnum, procnum;
char *(*procname) () ;
xdrproc_t inproc, outproc;
– Registerrpc tells the server’s portmapper that a procedure is ready to
provide a service
– One server executable may install multiple programs, with different or
even multiple versions of each program
– Registerrpc sets up UDP/IP communications only; To register with
TCP/IP lower-level RPC calls need to be used
– inproc and outproc are the XDR filters that data passes through when
going on or off the network

 void svc_run();
– Starts the process of waiting on the end of a socket, listening for a valid
connection
– This function can be used with TCP/IP and UDP/IP transport
 High-Level RPC Example: server.c
#include <stdio.h>
#include <rpc/rpc.h>
#include “date.h”
#include <time.h>
char **str_date(long *); long *bin_date(void);
main(){
registerrpc (DATEPROG,DATEVERS,BIN_DATE, bin_date, xdr_void,
xdr_u_long);
registerrpc (DATEPROG,DATEVERS,STR_DATE, str_date, xdr_u_long,
xdr_wrapstring);
svc_run(); /* never returns */
fprintf (stderr,"ERROR, svc_run returned\n"); return 0;
}
long *bin_date (void) {
static long lresult; lresult = time(NULL); return &lresult;
}
char **str_date (long *t) {
static char *result; result = ctime(t); return &result;
}
 High-Level RPC (client side)

 int callrpc(host, prognum, versnum, procnum, inproc, in, outproc, out)

char *host;
u_long prognum, versnum, procnum;
char *in;
xdrproc_t inproc;
char *out;
xdrproc_t outproc;

– Callrpc is called by the client to find and execute a UDP service on a host
– It checks with the portmapper and sets up the appropriate I/O channel
– in and out are pointers to the remote procedure argument and return value
– inproc and outproc are the XDR filters
 High-Level RPC Example: client.c
#include <stdio.h>
#include <rpc/rpc.h>
#include “date.h”
long bin_date();
char *str_date(long);
int main (int argc, char *argv[]) {
long lresult; char *sresult; int retval;
if (argc != 2) {fprintf(stderr,"usage: %s hostname\n",argv[0]);
exit(0);}
retval = callrpc (argv[1], DATEPROG, DATEVERS, BIN_DATE,
xdr_void, 0, xdr_u_long, &lresult);
if (retval != 0) {clnt_perror(retval); exit(0);}
retval = callrpc (argv[1], DATEPROG, DATEVERS, STR_DATE,
xdr_u_long, &lresult, xdr_wrapstring, &sresult);
if (retval != 0) {clnt_perror(retval); exit(0);}
printf("Time on host %s = %ld, %s\n", argv[0], lresult, sresult);
return 0;
}
 Critique of RPC

 SUN RPC lacks location transparency

 SUN RPC is not transport independent; transport protocols
are limited to TCP/UDP
 SUN RPC supports only at-least-once semantics, which
may not be acceptable for some applications

 Usefulness of RPC is limited to applications that can be

modeled as Client-Server