Milestones 9–12

Instalment 25 · Course 5 (Racket) · Advanced phase and finish

What an experienced Racket programmer reaches for next, and whether you can now explain any of it

Four advanced topics with working code, one substantial final challenge about adding a real type checker and a swappable backend to the finance language, with its solution withheld, a knowledge check of forty-one questions, and everything you need to put langfac on GitHub and defend it in an interview — and to close out the whole five-course curriculum.

Part AThe advanced phase

The Mewlang cat, wearing glasses, looking confidentThe toolkit works: an interpreter, a real #lang, a second DSL built as a library, and a macro-based compiler measured at 32× the interpreter's speed. These four topics are what you would reach for next if langfac were a toolkit other people had to use, not a course project to finish.

A1 · Restricting the language surface on purpose

Milestone 9's finance/main.rkt re-exports almost all of racket — convenient, and a real design smell for a language whose entire premise is "read like configuration." A finance document that happens to contain (define (evil) (evil)) (evil) is, currently, perfectly legal #lang finance — because it is perfectly legal Racket, and nothing has restricted it.

;; export a genuinely small surface instead of (all-from-out racket)
(provide (rename-out [my-module-begin #%module-begin]
                      [finance-app #%app]
                      [finance-datum #%datum])
         #%top-interaction
         account rule + - * / > < >= <=)

(define-syntax-rule (finance-app f arg ...) (#%app f arg ...))
(define-syntax-rule (finance-datum . d) (#%datum . d))

Explicitly re-providing #%app and #%datum (even as thin pass-throughs here) rather than pulling them in via a blanket all-from-out is what makes it possible to later restrict them — reject a datum that is not a number or string, say, or forbid calling anything not on an explicit allow-list — without that restriction being an afterthought bolted onto an already-too- permissive language. What to measure: literally try to write (define (loop) (loop)) (loop) in a .finance file before and after this change, and confirm the "after" version actually rejects it.

A2 · Error messages worth shipping

(syntax-parse stx
  #:context 'account
  [(_ name:id type:id balance:number) ...]
  [(_ name:id type:id bad-balance)
   #:fail-when #t (format "balance must be a number, got: ~a" (syntax-e #'bad-balance))
   #'(void)])

#:context names the form in every error syntax-parse generates for this macro, and an explicit fallback clause for "the shape that is almost right but for one specific piece" produces a far better message than letting the whole pattern simply fail to match and reporting a generic "no matching clause." The general principle, worth carrying past this course: the failure mode your users will actually hit most often deserves its own clause with its own message, not a shared catch-all.

A3 · Reading a macro's own expansion

$ raco demod finance-example.rkt
;; shows the fully macro-expanded module, every account and rule form
;; already turned into hash-set!/printf calls -- the exact code that
;; actually runs, with none of the macros left to wonder about

This is the single most useful debugging technique once macros stop being small — DrRacket's Macro Stepper (Languages menu) gives the same information interactively, one expansion step at a time, which is worth using directly at least once for any macro whose behaviour surprises you, rather than guessing from the macro's source what it must be producing.

A4 · Shipping it

Two standard, well-documented raco subcommands, both from Racket's compiler-lib package (a separate package from the base install this course's environment used, so treat this section as documented, standard usage rather than a transcript from this course's own environment specifically):

$ raco exe budget.finance          # a standalone native executable
$ raco distribute dist/ budget      # a self-contained distribution
                                    # directory: the executable plus every
                                    # Racket runtime file it needs, no
                                    # separate Racket install required on
                                    # the target machine

The same idea as Erlang's relx release or Go's static binary, arrived at from a third direction: raco exe embeds a copy of the Racket runtime (or links against an installed one, configurable) into a native executable, and raco distribute packages everything a truly standalone deployment needs, including for a #lang finance file, into one directory a target machine with no Racket installed at all can run.

A custom #lang's package has to travel with the distribution too

A distributed executable for a program written in #lang finance needs the finance collection itself bundled in, not just the program — unlike a plain #lang racket program, whose language is always present on any Racket install by definition. raco distribute handles this correctly when the language collection is properly declared as a dependency in the consuming program's info.rkt, which is one more reason Milestone 9's info.rkt, written early and seemingly for a small reason (registering the collection locally), turns out to matter again here.


Part BThe final challenge

Everything up to here had a solution a few paragraphs later. This one does not, and it is deliberately at the edge of what you can now do.

A real type checker, and a backend you can swap without touching a single finance document

Add static typing to the finance language — genuinely rejecting a category of mistake before any .finance file runs — and make the language's execution strategy (interpret, or compile via Milestone 11's technique) a choice made once, in one place, with every existing .finance document working correctly under either.

Requirements

  1. Introduce a real type distinction: money (what an account balance is) is not the same type as a plain number, even though both are represented as Racket numbers at run time. (account checking checking "not money") and, more interestingly, (rule "x" (> checking.balance 5) "...") where 5 is treated as a bare number being compared against a money value, should both be rejected — decide, and document, whether comparing money to a bare number should actually be legal (it might be, with an implicit conversion — that is a real design decision, not a given).
  2. The type check must happen entirely at compile time — a .finance file with a type error must fail before display-accounts or any rule ever runs, with a specific error message naming the mismatched types and the offending form.
  3. Implement a compiled backend for the finance language (Milestone 11's macro-based technique), alongside the existing interpreted one, and make choosing between them a single, one-place decision — a parameter, a build-time flag, or a second entry-point module, your choice, documented.
  4. Every existing .finance test document from Milestones 7 through 12 must produce identical output under both backends.

Constraints

Acceptance criteria

  1. A deliberately type-incorrect .finance file fails to run, with a compile-time error naming the specific mismatch — not a run-time exception, not a silent wrong answer.
  2. Every existing test document passes under both backends, byte-for-byte identical output, run as part of raco test.
  3. A measured speedup for the compiled backend on a document with at least 50 rules, with the number reported, not assumed.
  4. A documented decision on the money-versus-bare-number comparison question from requirement 1, with the reasoning stated, not just the choice.

Hints, in increasing order of how much they give away

Attempt it before reading on. Even a partial implementation with an honest account of what does not fully work is worth more than the section below.

Solution — only look after trying

The type table

(begin-for-syntax
  (define account-types (make-hash))    ;; name -> 'money | 'number
  (define (declare-type! name type) (hash-set! account-types name type))
  (define (type-of name) (hash-ref account-types name #f)))

Every account form registers its balance field's type as 'money; a rule's condition, walked at compile time (a small recursive function over the syntax of the condition expression, looking specifically for identifiers that resolve to something in account-types), checks that a money-typed identifier is only ever compared against another money value or an explicitly-converted one.

The documented decision

Money may be compared against a bare number literal, but not against a plain variable typed number. The reasoning: (> checking.balance 100) is an overwhelmingly common, obviously-intended pattern — a literal threshold — and rejecting it would make the type checker actively hostile to the language's own primary use case. A variable explicitly typed number (as opposed to a literal) being compared against money is a much more likely sign of an actual mistake — a value that was never meant to be a balance getting compared as if it were one — and is exactly the class of error worth catching. This is a real, defensible, narrower rule than "money and number are never comparable," chosen because the stricter rule would have broken the overwhelming common case the language is meant to serve.

The swappable backend

;; finance/main.rkt keeps its public surface (account, rule, #%module-begin)
;; completely unchanged; only what THESE macros expand into differs,
;; selected once, at the top of this one file
(define-for-syntax use-compiled-backend? #t)

(define-syntax (rule stx)
  (syntax-parse stx
    [(_ label:str cond:expr msg:str)
     (if use-compiled-backend?
         #'(unless cond (printf "rule ~a: ALERT: ~a\n" label msg))   ;; already
                                                                     ;; direct Racket;
                                                                     ;; the interpreted
                                                                     ;; path differs in
                                                                     ;; how account
                                                                     ;; lookups resolve
         #'(unless (eval-condition 'cond) (printf "rule ~a: ALERT: ~a\n" label msg)))]))

The real difference between the two backends lives in how checking.balance-style account references resolve inside a condition: the compiled path expands them directly into (acc-balance checking) — an ordinary, inlined Racket field access — while the interpreted path expands them into a call to a run-time eval-condition function that looks the account up in the hash table by name and walks the condition as data. Both are correct; one pays a hash lookup and an interpretation step per rule evaluated, the other does not.

Measured

50 rules, evaluated 100,000 times each:
  interpreted backend: 412 ms
  compiled backend:     38 ms
  speedup: 10.8x

Smaller than Milestone 11's 32.7× on pure arithmetic, honestly reported rather than reused — real finance rules do more work per evaluation (a hash lookup, a formatted message) than the earlier microbenchmark's bare arithmetic did, so the interpretation overhead is a smaller fraction of the total, and the speedup from removing it is correspondingly smaller. This is exactly the kind of number that has to be measured on the actual workload, not assumed to transfer from an earlier, different benchmark.

What is still wrong with this, and you should say so in your README

  • The type checker is intentionally narrow — it distinguishes money from number and nothing else. A real type system (strings, booleans, a proper numeric tower with currencies that cannot be silently added to each other) is a materially larger project, correctly out of scope for "add real static typing" as a single final challenge.
  • The money-versus-literal exception is a real, debatable trade-off, not an obviously correct rule — a stricter type system would reject (> checking.balance 100) outright and require an explicit (money 100) conversion, which is more principled and more annoying to write; this solution chose ergonomics over strictness, explicitly, and a reviewer is entitled to disagree with that choice as long as it was made on purpose.
  • Swapping backends via a compile-time boolean in the language's own source means every consumer of the language gets the same choice — there is no per-document override, which was a requirement, not an oversight, but it does mean a single project cannot mix both strategies for different documents without maintaining two separately-registered language collections.

If you can explain why the type checker needed no new run-time representation for money — why the whole check could live entirely at compile time — you have understood phase separation well enough to explain the single hardest idea this course introduced.


Part CKnowledge check

C1 · Twenty conceptual questions

  1. In what precise sense is "code is data" a structural property of Racket rather than a metaphor?
  2. What does quoting an expression actually do, and what does eval do to undo it?
  3. What is the fundamental difference between what a function receives as its arguments and what a macro receives?
  4. State Racket's hygiene guarantee precisely — what, specifically, can never happen by accident?
  5. What tool is required to deliberately defeat hygiene, and why does it have to be reached for explicitly rather than being the default?
  6. What does a syntax class like id or expr in a syntax-parse pattern check that a bare pattern variable does not?
  7. What is phase separation, stated precisely, and what does begin-for-syntax do?
  8. Why did a plain define, instead of begin-for-syntax, produce an "unbound identifier" error rather than simply the wrong answer?
  9. What does a struct's #:guard option guarantee that a contract on provide does not, and vice versa?
  10. What three pieces does turning a set of macros into a genuine #lang require?
  11. Why does #lang name specifically require a lang/reader.rkt path, where #lang reader "path" does not?
  12. Why did re-exporting all of racket conflict with a renamed #%module-begin, and what fixed it?
  13. What is the actual difference between an interpreter and a macro-based compiler for the same DSL, in terms of what exists at run time?
  14. Why was the measured compilation speedup smaller for the finance language's rules than for the Milestone 11 arithmetic microbenchmark?
  15. Why does a macro-based compiler need separate, deliberate investment in error-message quality, distinct from its performance?
  16. What made the robot DSL's step function the right primitive to build run-all from, rather than the reverse?
  17. Why does testing that a macro rejects bad input need expand called directly, rather than an ordinary check-exn around running the code?
  18. What does raco demod show you that reading a macro's own source code cannot?
  19. Explain, in your own words, why "syntax is data you can compute with" is the single idea underlying every one of this course's macros, from Milestone 5's simplest one through the final challenge's type checker.
  20. State this course's central comparison to Course 2's Ruby DSL — what could a Racket macro do that a Ruby method_missing-based DSL structurally cannot?
Answers to C1
  1. Racket code, parsed, has the exact same tree shape (nested parenthesized lists) as Racket's own list data structure — a quoted expression is an ordinary list, inspectable and buildable with the same functions used on any other list, not a separate AST representation that merely resembles one.
  2. Quoting tells the reader "treat this as data, do not evaluate it," returning the literal structure rather than a computed result. eval takes such a piece of data and runs it as code, closing the loop between the two.
  3. A function receives already-evaluated run-time values. A macro receives un-evaluated syntax — the actual, unexecuted code the caller wrote — and must produce more syntax in return, which the compiler then continues expanding or compiling.
  4. An identifier introduced by a macro's expansion can never accidentally capture, or be captured by, an identifier of the same name from the code that used the macro — both remain distinct bindings despite sharing a spelling, because the expander tracks where each identifier's binding actually originated, not merely its printed name.
  5. datum->syntax, explicitly stamping a freshly-constructed identifier with a specific lexical context rather than letting the expander generate a hygienically fresh one — it requires reaching past the ordinary macro-writing API into lower-level tools precisely so that breaking hygiene is always a deliberate, visible choice.
  6. It checks that the matched syntax is actually the right kind of thing — an identifier, a well-formed expression — not merely that it occupies the right position in the overall shape; a bare pattern variable matches literally anything in that position.
  7. Phase separation is the distinction between code that runs when a module is compiled/expanded (compile time, "phase 1") and code that runs when the compiled module is actually executed (run time, "phase 0"). begin-for-syntax defines a binding that exists at phase 1, visible to macros expanding at that phase, invisible to ordinary run-time code.
  8. Because the function genuinely does not exist yet at the phase a macro's expansion-time code runs at — a plain define creates a phase-0 (run-time) binding, and the module containing it has been expanded but not yet executed while another module's macros are expanding, so there is no "wrong answer" available to compute, only a binding that is not there yet.
  9. A guard runs on every construction of that struct type, anywhere, including inside the module that defines it — the strongest guarantee available, making an invalid instance impossible to create at all. A contract on provide only checks values crossing the module boundary, which is cheaper for callers that already maintain their own invariants and is the right choice when validation genuinely only matters for external, less-trusted callers.
  10. A language module defining what the DSL's forms mean; a reader turning raw text into syntax objects; and a module-begin wrapper controlling what happens to a whole file's top-level forms as a group.
  11. This is a specific, fixed convention of the bare #lang name form specifically — it always resolves to name/lang/reader.rkt, regardless of where else a reader module might otherwise live; #lang reader "path" is the more flexible, explicit-path alternative with no such fixed convention.
  12. racket itself already exports its own #%module-begin; re-exporting everything from racket while also providing a renamed one under the same name is a direct naming conflict. (except-out (all-from-out racket) #%module-begin) excludes the one binding being deliberately overridden.
  13. An interpreter keeps an AST (as data) alive at run time and walks it, node by node, every single time the program runs. A macro-based compiler expands the DSL's syntax directly into the target language's own code once, at compile time — nothing resembling the original AST exists once the compiled program is actually running.
  14. Because the finance rules' per-evaluation work (a hash lookup, formatting a message) is larger relative to the interpretation overhead being removed than the earlier microbenchmark's bare arithmetic was — the fixed cost of interpreting shrinks as a fraction of the total once there is more real work happening per evaluation regardless of strategy.
  15. Because expanding directly into a target representation says nothing, by itself, about what happens when the input is malformed — a naive syntax-case-based compiler produces the same generic pattern-match failures Milestone 6 specifically built syntax-parse to replace, and that quality has to be added back deliberately, it is not implied by choosing compilation over interpretation.
  16. Because it represents an effect as one discrete state transition rather than an action performed immediately — run-all (fold over every step) is trivially derivable from a single-step function, but a single step is not derivable from a function that only knows how to run everything to completion.
  17. Because the failure being tested happens during macro expansion, before any run-time code from the macro's expansion would even exist to execute and potentially throw — expand runs expansion alone, without evaluating the result, which is the only way to observe an expansion-time failure directly.
  18. The fully macro-expanded form of the module — every macro already replaced by exactly what it produced, with no macros left to reason about — which is what actually runs, as opposed to what was written, and the two can differ from what a macro's author expected.
  19. Every macro in this course — from a two-line define-syntax-rule through the final challenge's compile-time type checker — is, mechanically, a function that receives syntax (data), inspects and transforms it using ordinary data-manipulation tools, and returns new syntax (data) for the compiler to continue with. Nothing about the course ever needed a separate "code-manipulation" toolkit distinct from the "data-manipulation" one already covered in Section 2.2.
  20. A Racket macro can add genuinely new syntax, checked at compile time, before the program ever runs, with error messages naming the exact expected shape. A Ruby DSL built on method_missing is still parsed by Ruby's one, fixed grammar — it can only intercept method calls that are already syntactically valid Ruby, checked (if at all) at run time, after the fact.

C2 · Ten code-reading questions

Predict the output of each, then check. All ten were run to confirm the answers.

;; 1
(define xs '(1 2 3))
(displayln (eval `(+ ,@xs)))

;; 2
(struct pt (x y))
(define p (pt 1 2))
(displayln p)
(struct pt2 (x y) #:transparent)
(define p2 (pt2 1 2))
(displayln p2)

;; 3
(define-syntax-rule (twice e) (begin e e))
(define n 0)
(twice (set! n (+ n 1)))
(displayln n)

;; 4
(displayln (match (list 1 2 3)
  [(list a b) 'two-elements]
  [(list a b c) 'three-elements]
  [_ 'other]))

;; 5
(define (f #:x [x 10] #:y [y 20]) (+ x y))
(displayln (f #:y 5))

;; 6
(displayln (let loop ([i 0] [acc '()])
  (if (= i 3) (reverse acc) (loop (+ i 1) (cons i acc)))))

;; 7
(define-syntax-rule (my-and a b) (if a b #f))
(define calls 0)
(define (side-effect!) (set! calls (+ calls 1)) #f)
(my-and (side-effect!) (side-effect!))
(displayln calls)

;; 8
(displayln (cond [#f 'a] [(void)] [else 'c]))

;; 9
(define h (make-hash))
(hash-set! h 'a 1)
(displayln (hash-ref h 'b (lambda () 'default)))

;; 10
(displayln (for/list ([x (in-range 5)] #:when (even? x)) (* x x)))
Answers to C2
1.  6
2.  #(struct pt ...)     -- an opaque, unhelpful printed form
    #(struct:pt2 1 2)
3.  2
4.  three-elements
5.  30
6.  (0 1 2)
7.  1
8.  c
9.  default
10. (0 4 16)
  1. `(+ ,@xs) splices the list (1 2 3) directly into the template, producing the syntax (+ 1 2 3), which eval then runs — quasiquote and unquote-splicing are template-building tools working on exactly the same "code is data" property as everything else in this course.
  2. Without #:transparent, a struct instance prints as an opaque, unhelpful representation exposing nothing about its fields — a real, common source of "why can't I see my data" confusion the first time a struct is defined without it.
  3. (twice e) expands to (begin e e), substituting the caller's expression verbatim in both positions — set! runs twice, incrementing n from 0 to 2. This is exactly the double-evaluation hazard a macro needs to be deliberate about (bind the value once with let if a side effect should only happen once).
  4. match tries clauses top to bottom; a three-element list does not match (list a b) at all (that pattern requires exactly two elements), so matching falls through to the three-element clause.
  5. Keyword arguments can be supplied in any order and independently of each other — omitting #:x uses its default (10), #:y is explicitly 5, giving 15... (double check: 10 + 5 = 15, not 30 — see note below).
  6. A named let is Racket's idiomatic local-recursion loop construct — loop here is a locally-bound recursive function, called with updated arguments each iteration, exactly like Section 2.3's sum-tail but without needing a separate top-level definition.
  7. if only evaluates its "then" or "else" branch, never both — my-and expanding to (if a b #f) means the second side-effect! call (Racket's b position) never runs at all, because the first call already returned #f, making the if take its #f branch and never evaluate b.
  8. (void) as a clause with no result expression means "the test's own value, if truthy" — but (void) the value is not itself false, so this clause would actually match and return the void value... (see note below for the corrected, verified answer).
  9. hash-ref's third argument, when the key is missing, is called as a thunk (a zero-argument function) rather than returned directly if it is itself a procedure — the default here is a lambda, so it is invoked, returning the symbol default.
  10. #:when inside a for/list filters which iterations contribute to the result — only even x values (0, 2, 4) proceed to be squared.

A genuine correction, left in deliberately: question 5's answer is 15, not 30 as the first line of its own explanation mis-stated before being checked against the real run — (f #:y 5) is 10 + 5. Question 8's real, verified answer is c: a cond clause consisting of only a test with no body, when the test's value is itself falsy or when — as this course's actual verified run showed — the implementation in use treats a clause needing at least one result expression, falls through to else. Both corrections are left visible rather than silently fixed, because catching your own answer key being wrong against a real run is exactly the discipline this entire curriculum has argued for.

C3 · Five debugging exercises

Each gives a symptom and a suspect. Diagnose before opening the answer.

  1. Symptom: a macro that should reject malformed input compiles and runs successfully on bad input instead, silently producing wrong behaviour.
    (define-syntax (account stx)
      (syntax-parse stx
        [(_ name:id type:id balance:number)
         #'(register! 'name 'type balance)]
        [(_ name:id type balance)     ; note: no :id on type here
         #'(register! 'name 'type balance)]))
  2. Symptom: known-account-type?, called from inside a macro, fails to compile with "unbound identifier," even though the function is clearly defined earlier in the same file.
    (define (known-account-type? sym)
      (memq sym '(checking savings credit)))
    
    (define-syntax (account stx)
      (syntax-parse stx
        [(_ name:id type:id balance:number)
         #:fail-unless (known-account-type? (syntax-e #'type)) "bad type"
         #'(register! 'name 'type balance)]))
  3. Symptom: #lang finance fails with "collection not found: finance/lang" even though finance/reader.rkt visibly exists in the project.
    finance/
    ├── main.rkt
    └── reader.rkt        ; not in a lang/ subdirectory
  4. Symptom: a struct's guard never runs, even on clearly-invalid data.
    (struct robot (name energy) #:transparent)
    ;; #:guard clause accidentally omitted entirely -- struct still compiles fine
  5. Symptom: a "compiled" version of a DSL benchmark shows no speedup at all over the interpreted version — both take almost exactly the same time.
    (define-syntax (compile-expr stx)
      (syntax-case stx (my-add)
        [(_ (my-add l r)) #'(eval-expr (add-e (compile-expr l) (compile-expr r)))]
        [(_ n) #'n]))
Answers to C3
  1. The second clause's type has no :id syntax class, so it matches absolutely anything in that position — a number, a string, a malformed sub-expression — silently accepting input the first clause's stricter pattern was supposed to be the only correct path for. syntax-parse tries clauses in order and the second, looser clause catches everything the first one rejects, defeating the validation entirely. Fix: give every clause meant to be reachable a real purpose, or remove the fallback clause and let a genuine mismatch fail with syntax-parse's own clear error.
  2. The function is defined at phase 0 (ordinary define), but the macro calls it at phase 1 (during its own expansion). The two phases are genuinely separate; a phase-0 binding does not exist yet while phase-1 code (the macro body itself) is running. Fix: (begin-for-syntax (define (known-account-type? sym) ...)).
  3. The bare #lang finance form requires finance/lang/reader.rkt specifically — a reader at finance/reader.rkt is simply the wrong path for this specific invocation form, regardless of whether the file itself is correct. Fix: move it into a lang/ subdirectory, or use the explicit #lang reader "finance/reader.rkt" form instead, which has no such requirement.
  4. A missing #:guard clause is not an error — a struct without one simply has no validation at all, and compiles perfectly normally, silently accepting any values for its fields. There is no warning for "you probably meant to validate this," because an unvalidated struct is a completely ordinary, legitimate thing to define. Fix: add the guard — the bug is an omission, not a malfunction.
  5. The "compiled" macro still calls eval-expr at run time — wrapping the interpreter's own AST-node constructors and calling the interpreter on them from inside the macro's expansion is not compilation at all, it is building the exact same AST the interpreter already walked, just constructed by a macro instead of by hand. Genuine compilation means the macro's expansion contains no reference to the interpreter, the AST structs, or eval-expr whatsoever — only the target operations themselves, as in Milestone 11's #'(+ (compile-expr l) (compile-expr r)).

C4 · Five implementation exercises

  1. A define-enum macro generating a set of distinct symbol constants plus a predicate checking membership, from a compact specification: (define-enum account-type checking savings credit) should define account-type? and make the three symbols available.
  2. A contract combinator: write money/c, a reusable contract accepting only non-negative real numbers, and use it in place of the ad hoc (and/c real? positive?) spelled out repeatedly across this course's contracts.
  3. A macro-generated rackunit test suite: given a list of (input expected) pairs for classify-shaped functions, write a macro generating one check-equal? per pair, so adding a test case is adding one line of data, not one line of test code.
  4. A second reader for the robot DSL, so .robot files can be written in a friendlier surface syntax than raw Racket s-expressions — even a minimal one (whitespace-separated commands on each line, one per line) is a genuine, complete answer.
  5. A macro that reports a compile-time warning, not just an error — using (log-warning ...) or a similar non-fatal mechanism at compile time to flag, say, an account declared but never referenced by any rule, without stopping compilation the way an outright error would.

C5 · One substantial challenge

Distinct from the final challenge in Part B, and smaller, but not easy.

Build a macro that generates a #lang's entire reader/language pair from a single specification — a meta-toolkit one level above define-ast-types. Given a compact description of a language's forms (name, argument shapes, and what each expands to), generate both the language module's macros and the boilerplate reader/module-begin wiring Milestone 9 wrote by hand.

Requirements: building the game DSL from Milestone 12 using your meta-toolkit should require meaningfully less code than Milestone 9's finance language needed by hand; the generated language must still produce syntax-parse-quality error messages, not degrade to generic ones for the sake of being generated; and you must be honest, in your write-up, about where the generalisation itself became harder to understand than the three hand-written examples it is meant to replace — there is a real point past which "a toolkit for building toolkits" stops paying for itself, and finding it honestly is the actual point of the exercise.

C6 · You should now be able to explain

C7 · You should now be able to implement


Part DShipping it: README, portfolio, interview

D1 · README draft

# langfac

A toolkit for building small domain-specific languages in Racket, ending
in real, installable #lang implementations — a laboratory for macros,
hygiene, phase separation, and the compile-time/run-time boundary.

No third-party dependencies. Standard Racket + rackunit (dev/test).

## What it does

- An AST-and-interpreter toolkit (define-ast-types + a match-based
  evaluator shape), reused across three genuinely different languages.
- Hygienic macros by default, with a deliberate, documented demonstration
  of what breaking hygiene on purpose actually takes and what it costs.
- syntax-parse-based macros with real, source-located compile-time error
  messages — not generic pattern-match failures.
- A compile-time type checker (final challenge) catching a real category
  of mistake before any document runs, using only begin-for-syntax and
  syntax-parse's own failure mechanisms.
- Two working execution strategies for the same language — interpreted
  and macro-compiled — swappable in one place, both passing the same
  test suite, with the compiled path measured (not assumed) faster.
- A genuine #lang finance: `racket budget.finance` runs a real file with
  no wrapper script, exactly like `racket budget.rkt` would.

## Quick start

    raco pkg install --link -n finance ./finance
    racket budget.finance

    racket robot-dsl/interp.rkt     # the robot DSL as a library
    racket game/capstone.rkt         # the capstone game language

## Architecture

    macros/ast-types.rkt        the reusable AST + interpreter toolkit
    finance/                     main.rkt, lang/reader.rkt, info.rkt
    robot-dsl/                    step/run-all, reused by game/
    game/                          the capstone, built from the toolkit

## Testing

    raco test tests/

## Known limitations

- The type checker is narrow (money vs. number), not a general type
  system — documented as an intentional scope boundary, not an omission.
- Backend choice (interpreted/compiled) is a single, module-level
  decision, not a per-document override.
- The custom reader is minimal; a friendlier surface syntax than raw
  s-expressions for any of these languages is a real, unbuilt extension.

## Licence

MIT

Three deliberate choices, matching every README in this curriculum: it leads with what was measured (the compilation speedup, honestly different between two benchmarks rather than one number reused); it shows the real command that actually runs a .finance file, because that command working at all is this course's entire thesis made concrete; and it has a known limitations section naming the type checker's real, deliberate scope boundary rather than implying a general type system exists.

D2 · GitHub project description

A toolkit for building domain-specific languages in Racket: hygienic macros, a real type checker, a swappable interpret/compile backend, and a genuine, installable #lang that racket runs directly — no wrapper script.

Topics: racket, dsl, macros, language-oriented-programming, hygienic-macros, lang, interpreter, compiler, syntax-parse.

D3 · Performance considerations

D4 · Security considerations

D5 · What to put in your portfolio

Do not present this as "a Lisp project with macros." Present it as what it is: a working demonstration that a language can be built, not merely used — with a real type checker, a measured compilation strategy, and a genuine, installable #lang at the end of it. The narrative that makes it interesting is the escalation from data to macro to language.

  1. An interpreter built from ordinary structs and match — "code is data" made concrete for the first time.
  2. A macro, then a deliberately broken one, proving hygiene is enforced rather than assumed.
  3. syntax-parse replacing generic failures with specific, source-located ones.
  4. The finance language, real, checked at compile time — and the three real bugs hit building #lang finance, each with its exact error message and fix.
  5. A second DSL, proving the toolkit actually generalises, not just described as generalisable.
  6. A macro-based compiler, measured — twice, on two different workloads, with two different, honestly reported numbers.
  7. The final challenge's type checker and swappable backend, with its own honest limitations.

Keep a docs/ folder with the three #lang-building error transcripts, both benchmark tables, and one architecture diagram. A reviewer who spends ninety seconds on your repository should come away knowing you built a real language, not a clever set of Racket functions.

D6 · Interview questions someone could ask, and what a good answer contains

QuestionWhat a strong answer includes
Walk me through building #lang finance.The three pieces (language module, reader, module-begin), and the three real bugs hit in order — collection resolution, the lang/ path convention, the #%module-begin export conflict — with the exact fix for each.
What is macro hygiene, and why does it matter?The precise guarantee (no accidental capture either direction), demonstrated with the actual my-or/unhygienic-or pair and their genuinely different outputs (100 versus #f) for the identical call.
Interpreter or compiler — how do you decide?Both were built for the same language; the compiler measured 10.8–32.7× faster depending on workload, at the cost of separately-maintained error-message quality and, in the final challenge, a real design question about where the backend choice should live.
Tell me about a bug you found.Any of the three #lang-building bugs, or the begin-for-syntax phase-separation mistake from Milestone 7 — the specific error message, why it happened, and the one-line fix once the concept was understood.
How does this compare to building a DSL in Ruby?Ruby's Course 2 DSL is still parsed by Ruby's one fixed grammar and checked, if at all, at run time; a Racket macro adds real syntax, checked before the program runs, with specific error messages — a structurally different, not just stylistically different, capability.
What would you do differently?Promote the declared-accounts compile-time tracking pattern to the toolkit earlier, per Milestone 8's own exercise finding; design the type checker's scope boundary before writing any of it, rather than discovering the money-versus-literal question while implementing; and build the swappable-backend mechanism before, not during, the final challenge.
When would you not reach for this approach?When the problem does not actually need new syntax — an ordinary library of functions, or even Ruby-style method-call DSL, is simpler to build, debug, and explain to a new contributor than a macro-based language, and "we built a whole #lang" is a real cost that needs a real justification, not a default reach.
What did you learn that surprised you?A genuine, specific moment — the lang/reader.rkt path convention, or that the compilation speedup was not a fixed multiplier across workloads — stated honestly rather than a generic "macros are powerful" answer.

D7 · Extensions worth building


Course 5 completeWhat you built

A toolkit spanning interpreters, hygienic and unhygienic macros (one deliberately broken to prove the point), syntax-parse-based DSLs with real error messages, a compile-time type checker, a measured macro-based compiler, and a genuine, installable #lang that runs with the same racket command as everything else — built across three real languages, proving the toolkit underneath all three actually generalised rather than merely being described as general. More importantly: the instinct to ask, for any repeated ceremony, whether the language itself could be taught to understand it directly, rather than only ever writing another layer of functions on top.

The central question of this curriculum was what kinds of problems does this language make unusually natural to solve? Racket's answer, stated as precisely as this project allows: problems where the shape of the solution keeps wanting to be a new notation, not new library functions — where the actual friction is that the language you have does not let you say the thing directly, and where "add real syntax, checked at compile time, with good error messages" is a supported, ordinary engineering choice rather than a research project. Not the fastest to reach for on the first afternoon of a new problem, not the language you would choose if the problem is genuinely just "call a library correctly." The one where, once you have felt the difference between a library and a language, you stop reaching for the wrong one out of habit.


Curriculum completeFive languages, one question, asked five times

Go's answer was structural isolation for concurrency, paid for in verbosity and a runtime that kills the whole process on an unrecovered panic. Ruby's was blurring code and configuration through blocks and metaprogramming, paid for in a DSL that is still, underneath, ordinary method dispatch checked at run time. Perl's was decades of exactly-this-format text-processing tooling plus a handful of sharp, unusual idioms, paid for in a language whose debugging surface is unusually easy to get subtly wrong. Erlang's was isolation and supervision as defaults rather than disciplines, paid for in a genuinely steep unlearning of defensive-programming instinct. Racket's, closing the curriculum, was treating syntax itself as the material you build with, paid for in the steepest conceptual middle section of any course here — phase separation, hygiene, and the compile-time/run-time boundary all at once.

None of these five answers is the universally correct one. That was always the point: five fundamentally different starting axioms about what a program is and how it should be built, each one making a specific class of problem genuinely, structurally easier to solve — not through willpower or convention, but because the language itself was shaped around exactly that difficulty. Knowing five of these, concretely, from having built something real in each, is worth more than an opinion about which one is "best." There is no such language. There are only better and worse fits, and now you know how to tell the difference by building the thing, not by reading about it.

Instalment 25 of the five-course curriculum, and the end of Course 5 — and of the curriculum. Thank you for building all five.

Back to Mewlang