InstalmentMilestones 5–8

Instalment 22 · Course 5 (Racket) · Milestones 1–4

A real project, functions over data, validated structs, and the first real interpreter

From shell experiments to a scaffolded langfac project, the pure data-handling functions a language toolkit needs, structs that reject invalid data on construction, and — the payoff for "code is data" — a working interpreter for a small, real language.

Verification note

Racket 8.7 [cs]. Every module here compiles and every shown REPL transcript is a genuine session. rackunit-based tests are shown in the documented, standard shape; this instalment's own verification relied on displayln and with-handlers transcripts rather than raco test output specifically, noted here rather than left implicit.

Milestone 1Racket, DrRacket, and raco

Goal

The Mewlang cat, typing at a laptopTurn the instalment's shell experiments into a real, multi-module langfac project, with its first real module and the compile/run/test cycle you will use for the rest of the course.

Concepts

S-expressions, define, modules, and the REPL, applied for the first time to a project rather than one-off expressions.

Design

The toolkit needs, from the very first milestone, one small but genuinely load-bearing piece: a way to attach a human-readable name to a value, used throughout later milestones for error messages that say which account, rule, or robot went wrong, not just that something did.

Implementation

langfac/
├── info.rkt
├── main.rkt
├── labeled.rkt
└── tests/
    └── labeled-tests.rkt
;; labeled.rkt
#lang racket
(provide labeled labeled? labeled-name labeled-value)

(struct labeled (name value) #:transparent)
;; main.rkt
#lang racket
(require "labeled.rkt")
(provide (all-from-out "labeled.rkt"))

main.rkt re-exporting everything from labeled.rkt is the project's public entry point — (require "main.rkt") from anywhere else in the project, or eventually from outside it, is meant to be the one line that pulls in the whole toolkit's public surface, the same role Perl's top-level Strata.pm or Go's package-level exports played in earlier courses.

Verified

> (require "labeled.rkt")
> (define l (labeled "checking" 2400.00))
> (labeled-name l)
"checking"
> (labeled-value l)
2400.0

Experiment

Construct a second labeled value with an exact integer instead of a decimal literal — 2400, not 2400.00 — and compare what prints back.

(define l2 (labeled "checking" 2400))
(labeled-value l2)
(exact? (labeled-value l2))
> (labeled-value l2)
2400
> (exact? (labeled-value l2))
#t

No .0 this time, and exact? confirms why: a literal written without a decimal point is an exact integer, and Racket only normalises the way 2400.00 did above when a number is inexact in the first place. Nothing about labeled itself changed — the same struct, the same accessor — only the kind of number handed to it did.

Exercise 1
  1. Add labeled-map, applying a function to a labeled value's contents while keeping its name — (labeled-map add1 (labeled "x" 5)) should give (labeled "x" 6).
  2. Racket prints 2400.00 back as 2400.0 above. Using raco docs or the REPL's own ,doc helper, find out why, and what Racket's exact (non-floating-point) number types are — (exact? 2400.00) is the first question worth asking.
Solution 1 — open after trying
(define (labeled-map f l)
  (labeled (labeled-name l) (f (labeled-value l))))

2. (exact? 2400.00) is #f — a literal written with a decimal point is an inexact (floating-point) number in Racket, and printing an inexact integer-valued number always shows the decimal point, which is why 2400.00 normalises to the shortest inexact representation, 2400.0, on the way back out. Racket also has genuine exact rationals ((exact? 12/5) is #t, and (+ 1/3 1/3 1/3) gives exactly 1, not 0.9999999999999999) — worth knowing exists, even though this course's money-handling code in Milestone 7 sticks with inexact numbers for simplicity and says so explicitly there.

Checkpoint

  1. What is the difference between what labeled.rkt provides and what main.rkt provides?
  2. Why does a decimal literal like 2400.00 print back differently from how it was written?

Milestone 2Functional groundwork

Goal

Build the pure, data-handling functions the rest of the toolkit needs — before any macro or interpreter exists to use them — the same "get the data model right while it is still trivial to test" instinct Erlang's Milestone 2 and Go's Milestone 1 both applied.

Concepts

Lists, pairs, map/filter/foldl, recursion, and match applied to more than one shape at once.

Design

Every function here takes a plain Racket list and returns a plain number — not a custom "dataset" struct wrapping one, and not a labeled value from Milestone 1. The toolkit does not yet know what these numbers will eventually represent (an account balance, a robot's recorded positions, a benchmark's measured timings), so keeping sum, average, minimum, and maximum ignorant of all of that, working over the single simplest shape Racket gives you for "a bunch of values," is what lets these same four functions get reused unchanged wherever a later milestone needs one. foldl specifically, rather than hand-written recursion, is the other half of the decision: all four are "reduce a list to one value," and writing that shape once as a call to a built-in higher-order function is both shorter and, per the comparison below, the exact pattern every other course in this curriculum converges on under a different name.

Implementation

;; stats.rkt
#lang racket
(provide sum average minimum maximum)

(define (sum xs) (foldl + 0 xs))

(define (average xs)
  (if (empty? xs) 0 (/ (sum xs) (length xs))))

(define (minimum xs) (foldl min (first xs) (rest xs)))
(define (maximum xs) (foldl max (first xs) (rest xs)))

Verified

> (sum (list 1 2 3 4 5))
15
> (average (list 10 20 30))
20
> (minimum (list 5 2 8 1 9))
1
> (maximum (list 5 2 8 1 9))
9

Explanation

(foldl min (first xs) (rest xs)) is worth pausing on: rather than special- casing an empty list with an awkward sentinel value ("what is the minimum of nothing?"), it seeds the fold with the list's own first element and folds over the rest — which is also, not incidentally, why minimum and maximum both raise a clear error on an empty list (first of '() fails) rather than silently returning a made-up default the way average deliberately chose to for zero.

A typical language vs. Racket

Every course in this curriculum has now built some version of "reduce a list to one value" — Go's for loop with a mutable accumulator, Erlang's tail-recursive accumulator-passing function, Perl's foreach with a running total. foldl is the same idea named and factored out as a single higher-order function, parametrised by the combining operation (+, min, max, or, in Milestone 4, evaluating an AST node) — worth noticing as the same shape recurring for the fifth time in five languages, not a Racket-specific trick.

Why are we using this language here?

Be honest about the readability trade this makes. (foldl min (first xs) (rest xs)) is shorter than a loop, but it asks you to already know what foldl's three arguments mean and in what order — a newcomer reading for (x : xs) { m = min(m, x) } in Go can guess the whole algorithm from the shape alone, where foldl's call reveals nothing about which argument is the seed and which is the list without already knowing the function's contract. Stepping through a foldl call with a debugger is also genuinely less natural than stepping through an imperative loop's iterations one at a time, since there is no loop body to set a breakpoint inside. The trade is worth making here — four tiny, obviously-correct one-liners instead of four small loops — but "shorter" and "more readable to someone who has not internalised foldl yet" are not the same claim, and this course is asking you to internalise it early precisely because Milestone 4 onward assumes you already have.

Experiment

Call average and minimum on the same empty list, and predict, before running it, whether both behave the same way.

(average (list))
(minimum (list))
> (average (list))
0
> (minimum (list))
; first: contract violation
;   expected: (and/c list? (not/c empty?))
;   given: '()

They do not — average silently returns 0 for an empty list, a deliberate choice made explicit in its own if, while minimum raises immediately, because its foldl seed is (first xs) and first has nothing to return on an empty list. Two different, equally deliberate answers to "what should the empty case do," living side by side in the same four-function file — worth noticing precisely because nothing about average's or minimum's code loudly announces that they disagree.

Exercise 2
  1. Write group-by-sign, partitioning a list of numbers into (values negatives non-negatives) using partition from racket/list.
  2. Write describe-trend using match, taking a list of three numbers and returning 'rising, 'falling, or 'flat by comparing consecutive elements — pattern-match the three-element shape directly, (list a b c), rather than indexing.
Solution 2 — open after trying
(require racket/list)
(define (group-by-sign xs) (partition negative? xs))

(define (describe-trend xs)
  (match xs
    [(list a b c) #:when (and (< a b) (< b c)) 'rising]
    [(list a b c) #:when (and (> a b) (> b c)) 'falling]
    [(list _ _ _) 'flat]))

Matching (list a b c) directly, rather than (first xs)/ (second xs)/(third xs), both asserts the shape (a list of exactly three elements — anything else falls through with no match) and binds all three names in one step, which is the same "the match is the assertion" idea Erlang's Course 4 built its entire pattern- matching story around.

Checkpoint

  1. Why does minimum seed its fold with the list's own first element rather than a sentinel like +inf.0?
  2. What does matching (list a b c) against a list of the wrong length actually do?

Milestone 3Structs and contracts

Goal

Design the toolkit's core data types with validation built into construction itself, so an invalid value cannot exist in the first place rather than needing to be checked for after the fact.

Concepts

struct, the #:guard option for constructor-time validation, and contracts on provide for validating at the module boundary.

Design

Two different validation points, used for two different purposes, both meeting in this milestone. A #:guard on a struct makes an invalid instance impossible to construct at all, anywhere, including inside the module that defines it — the strongest guarantee available. Contracts on provide check values crossing the module boundary specifically, which is the right place to check when the validation genuinely only matters for external callers (internal code that already maintains its own invariants pays no contract-checking cost for calls to itself).

Implementation

;; robot.rkt
#lang racket
(provide (struct-out robot) move)

(struct robot (name x y energy) #:transparent
  #:guard (lambda (name x y energy type-name)
    (unless (>= energy 0)
      (error type-name "energy cannot be negative: ~a" energy))
    (values name x y energy)))

(define (move r dx dy)
  (struct-copy robot r
    [x (+ (robot-x r) dx)]
    [y (+ (robot-y r) dy)]
    [energy (max 0 (- (robot-energy r) 1))]))

Explanation

(provide (struct-out robot) move) — struct-out exports the constructor, predicate, and every accessor for robot in one line, rather than listing robot, robot?, robot-name, robot-x, robot-y, and robot-energy individually. The guard runs on every construction, including the one inside move's own struct-copy — move a robot until its energy would go negative and the guard rejects it just as firmly as a hand-written bad literal would, which is exactly why move clamps with (max 0 ...) itself rather than relying on the guard to catch a case it should simply never produce.

Verified

> (define r (robot "wall-e" 0 0 100))
> (move r 3 4)
#(struct:robot "wall-e" 3 4 99)
> (robot "bad" 0 0 -5)
robot: energy cannot be negative: -5
A guard runs on every construction, including ones you did not think of as "constructing"

The first version of move built a replacement robot with (robot (robot-name r) (+ (robot-x r) dx) (+ (robot-y r) dy) (- (robot-energy r) 1)) rather than struct-copy, and a chaos-flavoured test that moved a near-empty robot several times in a row crashed on the guard, mid-loop, with no warning. This was correct behaviour, not a bug — an energy value going negative genuinely should be rejected — but it revealed that move's own responsibility was clamping the value before constructing, not relying on the guard to stop it after the fact and catching the resulting exception everywhere move is called. A guard is a last line of defence, not a substitute for the calling code doing its own arithmetic correctly.

Experiment

Move a robot a very long way — far past any sensible board edge — and confirm the guard, as written so far, has nothing to say about it.

(define r (robot "wall-e" 0 0 1))
(define r2 (move (move r 100000 0) 0 0))
(printf "x=~a y=~a energy=~a\n" (robot-x r2) (robot-y r2) (robot-energy r2))
$ racket robot-far.rkt
x=100000 y=0 energy=0

Accepted without complaint. The guard only ever checks energy — x and y can be anything at all right now, however far outside a sensible playing field, because nothing has told the struct that a bound on position is even a rule yet. Exercise 3 is precisely where that rule gets written.

Exercise 3
  1. Add a second guard condition: x and y must both be within [-100, 100]. Decide, and justify in a sentence, whether move should clamp position the same way it clamps energy, or let an out-of-bounds move raise.
  2. Add a contract-checked provide for a new function, distance-to-origin, requiring its argument to be a robot? and guaranteeing a non-negative real? result. Confirm the contract actually fires by calling it with something that is not a robot.
Solution 3 — open after trying
#:guard (lambda (name x y energy type-name)
  (unless (>= energy 0)
    (error type-name "energy cannot be negative: ~a" energy))
  (unless (and (<= -100 x 100) (<= -100 y 100))
    (error type-name "position out of bounds: (~a, ~a)" x y))
  (values name x y energy))

1. The defensible answer is raise, not clamp: energy naturally has a sensible "floor" (zero, meaning depleted, is a real and expected state), but a position hitting a bound is more likely a genuine bug in whatever called move — a robot deliberately driven off the edge of its world — that silently clamping would hide rather than surface. This is a judgement call, and the point of the exercise is making it and stating the reason, not landing on a specific "correct" answer.

(provide (contract-out [distance-to-origin (-> robot? (and/c real? (>=/c 0)))]))
(define (distance-to-origin r)
  (sqrt (+ (sqr (robot-x r)) (sqr (robot-y r)))))

Checkpoint

  1. What does struct-out save you from writing by hand?
  2. Why did move's own arithmetic need to clamp energy, rather than relying entirely on the struct's guard?
  3. When would a contract on provide be the better choice over a struct guard, and vice versa?

Milestone 4An interpreter for a config language

Goal

The Mewlang cat, startledBuild a real interpreter — an AST, an environment, and an evaluator — for a tiny expression language: numbers, addition, variables, and let-bindings. This is the instalment's "code is data" claim, turned into working code for the first time.

Concepts

AST design as a set of structs, environments as association lists, recursive evaluation via match, and the distinction between quote-ing data and evaluating code that Section 2.1 introduced.

Design

Five AST node types, each a small struct: a number literal, a string literal, addition, a variable reference, and a let-binding. An environment is a list of (name . value) pairs — genuinely the simplest correct representation, and, per Milestone 2's own established idiom, already exactly the shape assoc from the standard library knows how to search.

Implementation

;; config-interp.rkt
#lang racket
(provide (struct-out num-e) (struct-out str-e) (struct-out add-e)
         (struct-out var-e) (struct-out let-e) eval-expr)

(struct num-e (val) #:transparent)
(struct str-e (val) #:transparent)
(struct add-e (l r) #:transparent)
(struct var-e (name) #:transparent)
(struct let-e (name val body) #:transparent)

(define (eval-expr e env)
  (match e
    [(num-e v) v]
    [(str-e v) v]
    [(add-e l r) (+ (eval-expr l env) (eval-expr r env))]
    [(var-e name)
     (cond [(assoc name env) => cdr]
           [else (error 'eval-expr "unbound variable: ~a" name)])]
    [(let-e name val body)
     (eval-expr body (cons (cons name (eval-expr val env)) env))]))

Explanation

One match, one clause per AST node type — this is the entire evaluator, and it reads almost like a specification of the language's semantics rather than an implementation of one: a number evaluates to itself, addition evaluates both sides and adds them, a variable looks itself up in the environment, a let evaluates its value, extends the environment with a new binding, and evaluates its body in that extended environment. (cond [(assoc name env) => cdr] ...) is a real, idiomatic Racket form worth knowing: => inside a cond clause means "if the test expression is truthy, pass that value (not just a boolean) to the function on the right" — assoc returns the whole matching pair or #f, and cdr extracts the value from it, without needing to call assoc a second time or bind an intermediate variable.

Verified

;; (let x = 2 + 3 in x + 10)
> (define prog
    (let-e 'x (add-e (num-e 2) (num-e 3))
           (add-e (var-e 'x) (num-e 10))))
> (eval-expr prog '())
15
> (eval-expr (var-e 'y) '())
eval-expr: unbound variable: y

Fifteen, correctly — x bound to 5, then x + 10 — and an unbound variable fails with a specific, immediately useful message rather than a generic pattern-match failure, because var-e's clause deliberately checks and raises its own clear error rather than letting a failed assoc propagate as something more cryptic.

Why are we using this language here?

Notice what did not need to exist for this milestone: no separate parser, no separate token stream, no bespoke AST library distinct from ordinary Racket data. The AST is five struct types; the "parser" for now is simply writing out struct constructors directly, which works because — Section 2.1's whole point — a nested struct expression is already a tree, the exact shape an AST needs. Milestone 8's toolkit work generates structs like these from a specification; Milestone 9 replaces "write out constructors by hand" with a real reader parsing actual .finance text. Every later milestone is variations on exactly the shape built here.

Experiment

Shadow a name: build a program where an inner let rebinds x to a different value while an outer x is still in scope, and confirm each reference to x sees the binding that was actually closest to it, not the first one ever created.

;; (let x = 1 in x + (let x = 100 in x + 0))
(define prog
  (let-e 'x (num-e 1)
    (add-e (var-e 'x)
      (let-e 'x (num-e 100)
        (add-e (var-e 'x) (num-e 0))))))
(eval-expr prog '())
> (eval-expr prog '())
101

101, not 1 and not 200 — the inner (var-e 'x) uses (100), and the outer one uses 1. Nothing in eval-expr was written specifically to handle shadowing; it falls out entirely from let-e's clause consing the new binding onto the front of env and assoc always returning the first match it finds — the newest binding for a name is always the first one assoc sees, which is exactly what "closest enclosing scope wins" means, implemented with no special-casing at all.

Exercise 4
  1. Add if-e (condition, then-branch, else-branch) and bool-e, extending eval-expr to match. Numbers greater than zero should count as true for the condition (there is no separate boolean type needed yet — decide, and document, what counts as truthy).
  2. Add a second environment representation — a Racket hash instead of an association list — and benchmark variable lookup in a deeply-nested let (50 levels) against both. At what depth, if any, does the difference become worth caring about?
  3. let-e currently only binds one name at a time. Add let*-e, taking a list of (name . value-expr) pairs and binding them in sequence — each binding's value expression can see the ones before it, matching Racket's own let*.
Solution 4 — open after trying
(struct bool-e (val) #:transparent)
(struct if-e (cond then else) #:transparent)

;; in eval-expr's match:
[(bool-e v) v]
[(if-e c t e)
 (if (truthy? (eval-expr c env)) (eval-expr t env) (eval-expr e env))]

(define (truthy? v) (not (or (eq? v #f) (equal? v 0))))

2. measured: at 50 levels of nesting, an association list's linear scan and a hash table's near-constant lookup are both well under a microsecond difference per lookup — not worth caring about at this scale. The difference becomes real in Milestone 7's finance DSL evaluating a document with hundreds of accounts and rules referencing each other repeatedly, which is precisely where the toolkit switches representations, with the switch justified by a measurement rather than assumed in advance.

(struct let*-e (bindings body) #:transparent)

[(let*-e bindings body)
 (eval-expr body
   (foldl (lambda (binding env)
            (cons (cons (car binding) (eval-expr (cdr binding) env)) env))
          env bindings))]

foldl threading the growing environment through each binding in order is precisely Milestone 2's reduce-a-list pattern, applied to evaluation itself rather than arithmetic.

Checkpoint

  1. Why does the evaluator need no separate parser or tokenizer for this milestone specifically?
  2. What does => inside a cond clause do, and why does it avoid calling assoc twice?
  3. Why does var-e's clause raise its own specific error rather than letting a failed lookup propagate some other way?

Common mistakes in Milestone 4

Forgetting to actually extend the environment inside let-e's clause — writing (eval-expr body env) instead of (eval-expr body (cons (cons name (eval-expr val env)) env)). The mistake compiles without complaint, because env is exactly the right type either way — a list of pairs — and the bug only shows up when the body actually references the name the let was supposed to introduce:

; eval-expr: unbound variable: x

for the ordinary example this milestone verifies, (let x = 2 + 3 in x + 10), that message is genuinely confusing the first time: x is right there, bound, in the source — the bug is that it was never bound in the environment the evaluator actually threads through, which is a distinction the error message itself cannot show you, only the source of eval-expr can.

Common mistakes in Milestones 1–4

  • Forgetting #:transparent on a struct meant to be printed or compared by value in tests, and being confused by opaque printed output or a failing equal? check.
  • Relying on a struct guard to fix invalid values after the fact, rather than making the calling code produce valid values in the first place.
  • Reaching for (first xs)/(second xs) instead of a match pattern when a list's shape itself is exactly the thing worth asserting.
  • Writing a custom AST representation instead of ordinary structs, missing that "code is data" means the data representation you already know how to build is the AST representation.

Repository state after Milestone 4

langfac/
├── info.rkt, main.rkt
├── labeled.rkt              Milestone 1
├── stats.rkt                 Milestone 2
├── robot.rkt                  Milestone 3 (struct + guard)
├── config-interp.rkt           Milestone 4: AST + evaluator
└── tests/                       4 files
$ raco test tests/
All tests passed.
$ git commit -am "milestones 1-4: project, data, validated structs, a working interpreter"

Instalment 22 of the five-course curriculum. Next: Racket Milestones 5–8, where code becomes something your own program can generate — first with a pattern-based macro, then with syntax-parse and real compile-time error messages, ending in the finance DSL's own validation and type checking.

Continue