But the pattern P 2 will be taken into account as it was listed first in lex program.

By Dinesh Thakur Lex is a tool in lexical analysis phase to recognize tokens using regular expression.

Well be covering the following topics in this tutorial: Use of Lex Structure of Lex Programs Conflict Resolution in Lex Lookahead Operator Design of Lexical Analyzer DFAs for Lexical Analyzers Use of Lex lex.l is an a input file written in a language which describes the generation of lexical analyzer.

The lex compiler transforms lex.l to a C program known as lex.yy.c.

C compiler to a file called a.out.

The output of C compiler is the working lexical analyzer which takes stream of input characters and produces a stream of tokens.

The attribute value can be numeric code, pointer to symbol table or nothing.

Structure of Lex Programs Lex program will be in following form declarations translation rules auxiliary functions Declarations This section includes declaration of variables, constants and regular definitions.

Translation rules It contains regular expressions and code segments.

Form: Pattern Action Pattern is a regular expression or regular definition.

Auxiliary functions This section holds additional functions which are used in actions.

These functions are compiled separately and loaded with lexical analyzer.

Lexical analyzer produced by lex starts its process by reading one character at a time until a valid match for a pattern is found.

Conflict Resolution in Lex Conflict arises when several prefixes of input matches one or more patterns.

This can be resolved by the following: Always prefer a longer prefix than a shorter prefix.

If two or more patterns are matched for the longest prefix, then the first pattern listed in lex program is preferred.

Lookahead Operator Lookahead operator is the additional operator that is read by lex in order to distinguish additional pattern for a token.

Lexical analyzer is used to read one character ahead of valid lexeme and then retracts to produce token.

At times, it is needed to have certain characters at the end of input to match with a pattern.

In such cases, slash () is used to indicate end of part of pattern that matches the lexeme.

In some languages keywords are not reserved.

So the statements IF (I, J) 5 and IF(condition) THEN results in conflict whether to produce IF as an array name or a keyword.

To resolve this the lex rule for keyword IF can be written as, IF (.

Design of Lexical Analyzer Lexical analyzer can either be generated by NFA or by DFA.

Structure of Generated Analyzer Architecture of lexical analyzer generated by lex is given in Fig.

Lexical analyzer program includes: Program to simulate automata Components created from lex program by lex itself which are listed as follows: o A transition table for automaton.

Steps to construct automaton Step 1: Convert each regular expression into NFA either by Thompsons subset construction or Direct Method.

Step 2: Combine all NFAs into one by introducing new start state with s-transitions to each of start states of NFAs N i for pattern P i Step 2 is needed as the objective is to construct single automaton to recognize lexemes that matches with any of the patterns.

A 1 for pattern P l abb action A 2 for pattern P 2 ab action A 3 for pattern P 3 For string obb, pattern P 2 and pattern p 3 matches.

Lex Compiler Online