Crafting Your First Compiler: A Beginner's Guide to Flex and Bison

Building a compiler from scratch might sound intimidating, but Unix legends Flex and Bison make it incredibly accessible. Whether you want to design a custom configuration file format, build a math engine, or create your own programming language, this power duo has you covered.

In this guide, we will break down how Flex and Bison work together and build a working terminal-based calculator from scratch.

---

Understanding the Dynamic Duo

A compiler handles a massive influx of text data. To make sense of it, Flex and Bison break the workload into two critical steps:

• Flex (The Lexical Analyzer / Scanner): Reads your source code character by character and groups them into meaningful units called Tokens. For example, if it sees 42, it flags it as a NUMBER. If it sees +, it flags it as a PLUS sign.
• Bison (The Parser): Receives the tokens from Flex and checks them against structural grammar rules (Context-Free Grammar). It ensures that a sequence like NUMBER PLUS NUMBER makes grammatical sense, and then executes the logic behind it.

[ Raw Text ] ──> ( Flex / Scanner ) ──> [ Tokens ] ──> ( Bison / Parser ) ──> [ Execution / Output ]

Step-by-Step Guide: Building a Simple Calculator

Let’s build a lightweight tool that parses and solves addition problems instantly. We will need two source files: lexer.l (for Flex) and parser.y (for Bison).

Step 1: Define the Grammar (parser.y)

First, create the Bison file. This is where we declare our tokens and write the structural rules of our language.

%{
#include <stdio.h>
#include <stdlib.h>

/* Forward declarations for the compiler */
void yyerror(const char *s);
int yylex(void);
%}

/* Define the tokens our parser expects from Flex */
%token NUMBER PLUS NEWLINE

%%

/* Grammar Rules */
calculate:
    | calculate expression NEWLINE { printf("Result: %d\n", $2); }
    ;

expression:
    NUMBER                    { $$ = $1; }       /* A single number represents itself */
    | expression PLUS NUMBER  { $$ = $1 + $3; }  /* Addition logic */
    ;

%%

/* Error handling routine */
void yyerror(const char *s) {
    fprintf(stderr, "Error: %s\n", s);
}

int main(void) {
    printf("Enter a math expression (e.g., 5+12) and press Enter:\n");
    return yyparse(); /* Triggers the parsing loop */
}

Step 2: Extracting Tokens (lexer.l)

Next, create the Flex file. We use Regular Expressions (Regex) to spot valid patterns in the input text and hand them over to Bison.

%{
#include "parser.tab.h" /* Import token definitions generated by Bison */
#include <stdlib.h>
%}

%%

[0-9]+  { yylval = atoi(yytext); return NUMBER; } /* Extract numeric value */
"+"     { return PLUS; }
\n      { return NEWLINE; }
[ \t]   { /* Ignore spaces and tabs */ }
.       { printf("Unknown character encountered: %s\n", yytext); }

%%

int yywrap(void) {
    return 1;
}

---

Compiling and Running the Code

To bring this code to life, you need Flex, Bison, and a C compiler (like GCC) installed on your machine. Open your terminal and execute the following sequence:

# 1. Run Bison to generate parser source code and header file
bison -d parser.y

# 2. Run Flex to generate the lexical scanner code
flex lexer.l

# 3. Compile all generated C components together via GCC
gcc parser.tab.c lex.yy.c -o calculator

# 4. Fire up your new calculator!
./calculator

The Live Test

Type 15+30 into the running prompt and hit Enter. Your terminal will instantly return:

Result: 45

---

Wrap Up & Next Steps

Flex and Bison seamlessly convert raw user text into highly structured data operations. By expanding on this blueprint, you can slowly scale up to handling subtractions, multiplication precedence, or variables.