Scribd
Upload
120 views11 pages

Lexical Analysis Examples

The document discusses lexical analysis and provides examples. Lexical analysis involves breaking down source code into tokens. A lexer or lexical analyzer reads character streams, identifies valid tokens like keywords, identifiers, operators, and removes non-tokens like whitespace, comments and preprocessor directives. The document provides an example C program that is analyzed by a lexer, showing the tokens identified like keywords "int", identifiers "x", "y", operators like "," and examples of non-tokens.

Uploaded by

akanksha naidu
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
120 views11 pages

Lexical Analysis Examples

The document discusses lexical analysis and provides examples. Lexical analysis involves breaking down source code into tokens. A lexer or lexical analyzer reads character streams, identifies valid tokens like keywords, identifiers, operators, and removes non-tokens like whitespace, comments and preprocessor directives. The document provides an example C program that is analyzed by a lexer, showing the tokens identified like keywords "int", identifiers "x", "y", operators like "," and examples of non-tokens.

Uploaded by

akanksha naidu
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 11

Lexical Analysis Examples

Lexical Analyser- Formal Definition


• Programs that perform lexical analysis are called lexical analysers or
lexers.
• A lexer contains tokenizer or scanner.
• If the lexical analyser detects that the token is invalid, it generates an
error.
• It reads character streams from the source code, checks for legal
tokens, and pass the data to the syntax analyser when it demands.
Example of Lexical Analysis, Tokens,
Non-Tokens
Lexical analyser performs below given tasks:
• Helps to identify token into the symbol table
• Removes white spaces and comments from the source program
• Correlates error messages with the source program
• Helps you to expands the macros if it is found in the source program
• Read input characters from the source program
Example
Consider the following code that is fed to Lexical Analyzer
#include <stdio.h>
int maximum(int x, int y)
{ // This will compare 2 numbers
if (x > y)
return x;
else
{ return y; }
}
Examples of Tokens created
Lexeme Token
int Keyword
maximum Identifier
( Operator
int Keyword
x Identifier
, Operator
int Keyword
Y Identifier
) Operator
{ Operator
If Keyword
Examples of Non tokens

Type Examples
Comment // This will compare 2 numbers
Pre-processor directive #include <stdio.h>
Pre-processor directive #define NUMS 8,9
Macro NUMS
Whitespace /n /b /t
C Tokens
• Keywords (eg: int, while),
• Identifiers (eg: main, total),
• Constants (eg: 10, 20),
• Strings (eg: “total”, “hello”),
• Special symbols (eg: (), {}),
• Operators (eg: +, /,-,*)
C TOKENS EXAMPLE PROGRAM:
int main()
{
int x, y, total;
x = 10, y = 20;
total = x + y;
printf ("Total = %d \n", total);
}
Tokens
where,
• main – identifier
• {,}, (,) – delimiter
• int – keyword
• x, y, total – identifier
• main, {, }, (, ), int, x, y, total,10,20 – tokens
Examples
int main()
{ // 2 variables
int a, b;
a = 10;
return 0;
}

'int' 'main' '(' ')' '{' 'int' 'a' ',' 'b' ';' 'a' '=' '10'
';' 'return' '0' ';' '}'
Examples Continued- Count the no of tokens
and what are the tokens
int main()
{
int a = 10, b = 20;
printf("sum is :%d",a+b);
return 0;
}

int max(int i);

You might also like