Measuring the magic, compiling it away, and checking whether any of it stuck

Part AThe advanced phase

A1 · The object model as a debugging tool

Metaprogramming makes "where did this method come from?" a real question. Ruby answers it directly, and these four calls will save you hours:

Automation::Steps::Filter.instance_method(:matching).owner        # which module defined it
Automation::Steps::Filter.instance_method(:matching).source_location # file and line
Automation::Steps::Filter.ancestors.first(5)                      # the lookup chain
object.singleton_methods                                          # methods on this object alone

For a method generated by define_method inside a class macro, source_location points at the macro, not at the option :matching line that triggered it. That is worth knowing before it confuses you: to make generated methods self-documenting you have to record the call site yourself, exactly as Milestone 4 did for steps.

The lookup chain, demonstrated:

module A; def who = "A"; end
module B; def who = "B"; end
class C
  include A
  prepend B
  def who = "C"
end
p C.new.who, C.ancestors.first(4)

"B"
[B, C, A, Object]

prepend goes before the class, include goes after it. So a prepended module wins over the class's own method (which is why it works for instrumentation), and an included module loses to it (which is why include provides defaults rather than overrides). Memorising the order is less useful than remembering that ancestors will always tell you.

A2 · Measure before you metaprogram

Milestone 7 replaced method_missing with generated methods and I claimed it was faster. Here is the number, 300,000 calls each:

                       user     system      total        real
define_method      0.019719   0.000818   0.020537 (  0.020542)
method_missing     0.040257   0.000662   0.040919 (  0.040936)

Twice as slow, which is the shape you would predict: method_missing runs only after a full method lookup has failed, so you pay for the search and then for the dispatch. Two honest observations about that result.

It is only 20 nanoseconds per call. For a DSL evaluated once at startup this is irrelevant, and choosing generated methods was justified by typo detection and introspection, not speed. If someone tells you method_missing is "too slow" for a configuration DSL, ask them for the number.

Benchmark ships with Ruby. benchmark/ips (iterations per second, with statistical comparison) is the better tool and is a gem; the standard Benchmark.bm above is enough to settle a 2× question. Use the real profiler (stackprof, also a gem) before optimising anything in anger.

A3 · Lazy pipelines for data that does not fit

Our runner passes an Array between steps, so a pipeline over 10 million records needs 10 million records in memory. Ruby's answer is Enumerator::Lazy: chain map and select without materialising anything, and pull only what you consume.

def records = (1..2_000_000).lazy.map { |i| { id: i, topic: i.even? ? "AI" : "bio" } }

# eager, over 200,000 records
(1..200_000).map { |i| { id: i, topic: i.even? ? "AI" : "bio" } }
            .select { |r| r[:topic] == "AI" }
            .map { |r| r[:id] }
            .first(5)

# lazy, over 2,000,000 records
records.select { |r| r[:topic] == "AI" }.map { |r| r[:id] }.first(5)

[2, 4, 6, 8, 10]
[2, 4, 6, 8, 10]
eager over 200k: 0.3803s | lazy over 2M: 0.0002s

Same answer; the lazy version was roughly two thousand times faster over ten times as much data, because it built exactly ten records. Chained eager calls allocate a full intermediate array per step; lazy chains fuse into one pass and stop as soon as the consumer is satisfied.

To support this in the gem, a step declares that it streams:

class Filter < Automation::Plugin
  streaming true      # promises: takes an Enumerable, returns an Enumerable,
                      # never calls #size, #sort or #to_a on its input

  def call(records) = records.select { |r| matches?(r) }
end

The runner then wraps the initial payload in .lazy when every step up to the first non-streaming one declares it, and materialises at the first step that does not. The metadata pattern from Milestone 7 makes this a five-line feature: the declaration is already the mechanism.

The catch to document: a lazy payload means failures move. An exception from fetch no longer happens during the fetch step; it happens later, when save_to finally pulls a value, so your beautiful per-step error reporting blames the wrong step. Streaming and precise error attribution are in genuine tension, and the usual resolution is to materialise at well-chosen boundaries rather than everywhere or nowhere.

A4 · Concurrency, honestly

CRuby has a global VM lock, so Ruby threads do not run Ruby code in parallel. They do run in parallel during I/O, because the lock is released around blocking calls:

def slow_io(n) = sleep(0.05) || n

serial   = Benchmark.realtime { 4.times { |i| slow_io(i) } }
parallel = Benchmark.realtime { 4.times.map { |i| Thread.new { slow_io(i) } }.each(&:join) }

serial 0.201s | threaded 0.069s

Three times faster on a single core, because all four sleeps overlapped. Our pipeline steps are HTTP calls and file writes, so this is exactly the case that benefits, and a fan_out step that runs N sub-pipelines concurrently is worth having.

Build it with a bounded pool rather than a thread per item, for the same reason Go's Milestone 11 bounded its workers:

def fan_out(items, concurrency: 8)
  queue = Queue.new
  items.each { |i| queue << i }
  results = Array.new(items.size)

  workers = concurrency.times.map do
    Thread.new do
      while (index = queue.pop(true) rescue nil)
        results[index] = yield(items[index])
      end
    end
  end
  workers.each(&:join)
  results
end

Queue is thread-safe and ships with Ruby; pop(true) is non-blocking and raises ThreadError when empty, which is how the workers know to stop. Writing to distinct indices of a preallocated array is safe here for the same reason it was in Go: separate slots, no resizing.

A5 · Allocation discipline

Ruby's garbage collector is good and allocation is still the main cost in most Ruby hot loops. With GC disabled to keep the measurement stable, 10,000 iterations each:

{:unfrozen_literal => 10005,
 :frozen_literal   => 2,
 :symbol           => 2,
 :data_node        => 30005,
 :hash_node        => 20002}

• A string literal allocates a new object every time it is evaluated (10,005 objects), while a frozen one is deduplicated to a single object (2). That is what the frozen_string_literal magic comment buys, in one measurement.
• Symbols allocate nothing, which is why identifiers in this project are symbols.
• A Data node with a Hash inside costs about three objects (node, hash, and the keyword machinery) against two for a bare Hash. Immutability and value equality are not free, and at 30,000 objects per 10,000 nodes they are cheap enough that the trade is easy.

Note the measurement technique. My first attempt, without GC.disable, reported negative allocations for the string case, because the collector ran mid-measurement and freed more than the loop created. Naive allocation counting is noise unless you disable the GC or use ObjectSpace.trace_object_allocations, and a benchmark that can report a negative count is a benchmark you should not trust in either direction.

A6 · Refinements, types, and docs

Refinements are scoped monkey-patching: a change to a core class that exists only in the file or module that asks for it.

module StepSugar
  refine Symbol do
    def to_step = "step(#{self})"
  end
end

:fetch.to_step              # => NoMethodError

module Scoped
  using StepSugar
  def self.demo = :fetch.to_step
end

Scoped.demo                 # => "step(fetch)"
:fetch.to_step              # => NoMethodError, still

This is the responsible way to add methods to classes you do not own, and it is why using exists. We did not use refinements in the DSL for a reason worth stating: they are lexically scoped, so they do not apply inside a block that was defined elsewhere, which is precisely how our DSL blocks are evaluated. A refinement-based DSL would work in the file that declared it and mysteriously fail everywhere else.

Types for a dynamic gem

Ruby 3 ships RBS, a separate signature language, and steep type-checks against it:

# sig/automation.rbs
module Automation
  class Registry
    def register: (Symbol | String, ?_Callable, ?spec: Hash[Symbol, untyped]?, ?doc: String?)
                  { (untyped, *untyped) -> untyped } -> self
    def fetch: (Symbol | String, ?location: String?) -> _Callable
    def known: () -> Array[Symbol]
  end

  interface _Callable
    def call: (untyped, *untyped, **untyped) -> untyped
  end
end

Signatures live outside the code, so they can be added incrementally and do not clutter a DSL. Be realistic about coverage: the parts of this gem that are ordinary objects (registry, config, store, AST) type-check well; the parts built on method_missing and generated methods cannot be described at all, because their signatures do not exist until run time. A gem like this one is exactly where static typing gives least, which is worth knowing when someone proposes adding it everywhere.

For documentation, YARD reads @param/@return comments and, more importantly here, lets you document methods that do not exist:

# @!method fetch(from:, limit: nil)
#   Fetch a JSON array of records from an HTTP endpoint.
#   @param from [String] URL returning a JSON array
#   @param limit [Integer, nil] keep at most this many records

That @!method directive is how you stop dynamic definitions from being invisible to your users. It is also a smell detector: if you cannot write the directive, your users cannot discover the method.

CI and supply chain

# .github/workflows/ci.yml (the shape, not the whole file)
strategy:
  matrix:
    ruby: ["3.2", "3.3", "3.4"]
steps:
  - uses: ruby/setup-ruby@v1
    with: { ruby-version: "${{ matrix.ruby }}", bundler-cache: true }
  - run: bundle exec rake test
  - run: bundle exec rubocop
  - run: gem install bundler-audit && bundle audit check --update

Test the Ruby versions your gemspec claims to support, or do not claim them. bundler-audit checks your lockfile against known advisories, which for a gem with zero runtime dependencies is a short conversation and remains worth automating.

Part BThe final challenge

Compile the pipeline instead of interpreting it

Everything so far interprets: the runner walks the AST at run time, looks each step up in a registry, and calls it. Now write a compiler that turns a pipeline into standalone Ruby source which runs with no interpreter, no registry lookups, and no dependency on this gem.

$ automation compile examples/research.rb --output research_compiled.rb
$ ruby research_compiled.rb          # no `require "automation"` anywhere

Requirements

1. Automation::Compiler.new(registry).compile(pipeline) returns a String of valid Ruby defining a single method run(input = nil) and calling it.
2. Step implementations are inlined into the generated source. A step registered as a block or lambda has its source extracted; a step registered as a Plugin subclass has its class definition emitted.
3. Retries, dry-run support and when_failed handling are generated as ordinary Ruby control flow, not as a middleware chain.
4. The generated file has no runtime dependency on the gem and passes ruby -c (syntax check) and RuboCop.
5. Compilation fails loudly, naming the step, when a step's implementation cannot be inlined.

Constraints

• No eval in the generated code. The point is source you can read and commit.
• Generated code must include a header comment naming the source file, the pipeline and the gem version.
• String interpolation of user values must be safe: an option containing "; system('rm -rf /'); " must not become executable code.
• The compiler must not run any step.

Acceptance criteria

1. Differential testing. For at least five pipelines, interpreted and compiled runs produce identical payloads and identical failure behaviour. Write this as a test that generates, writes, shells out to ruby, and compares.
2. Speed. The compiled version is measurably faster on a pipeline of ten trivial steps run 10,000 times. Report the ratio and be honest if it is small.
3. Safety. A test proves that a hostile option value ends up as data, not code.
4. Failure. Compiling a pipeline whose step was registered as a C-implemented method or an object without extractable source raises a clear error naming the step.

Hints

• Proc#source_location gives you a file and line. Getting the source text from that is the hard part: read the file and find the block's extent. Look at how the method_source gem does it, and consider whether requiring plugin classes (whose source is a whole file) rather than blocks makes the problem easier.
• For safe interpolation, do not build strings from user values at all. Emit value.inspect, which for every type we allow produces a literal that evaluates back to an equal value. Test it with a string full of quotes and interpolation markers.
• Generate a method per step (step_1_fetch) rather than one enormous method: the stack traces are better and it is easier to read.
• The retry logic you already wrote as middleware becomes a begin/rescue/retry template. Write the target code by hand first, then work backwards to the generator.

This is the most valuable exercise in the Ruby course, because it is the bridge to Course 5. Racket does this at compile time, with the language's own macro system, and produces better error messages while doing it. Building the clumsy version by hand is what makes the elegant version legible.

module Automation
  class Compiler
    class NotCompilable < Error; end

    def initialize(registry) = @registry = registry

    def compile(pipeline)
      steps = pipeline.steps.each_with_index.map { |step, i| compile_step(step, i) }

      <<~RUBY
        # Generated by automation #{VERSION} from #{pipeline.location}
        # Pipeline: #{pipeline.name}. Do not edit; regenerate instead.
        # frozen_string_literal: true

        #{inlined_definitions(pipeline).join("\n\n")}

        #{steps.join("\n\n")}

        def run(input = nil)
        #{run_body(pipeline).gsub(/^/, "  ")}
        end

        puts run.inspect if $PROGRAM_NAME == __FILE__
      RUBY
    end

    private

    def compile_step(step, index)
      <<~RUBY
        def step_#{index}_#{step.name}(input)
          #{call_expression(step)}
        end
      RUBY
    end

    # Values become literals via #inspect, never via interpolation.
    def literal(value)
      case value
      when String, Symbol, Numeric, TrueClass, FalseClass, NilClass then value.inspect
      when Array then "[#{value.map { |v| literal(v) }.join(', ')}]"
      when Hash then "{#{value.map { |k, v| "#{literal(k)} => #{literal(v)}" }.join(', ')}}"
      else raise NotCompilable, "cannot compile a #{value.class} option value"
      end
    end
  end
end

  def assert_compiles_identically(pipeline, input = nil)
    interpreted = Automation::Runner.new(registry: test_registry).run(pipeline, input)

    Dir.mktmpdir do |dir|
      file = File.join(dir, "compiled.rb")
      File.write(file, Automation::Compiler.new(test_registry).compile(pipeline))

      assert system("ruby", "-c", file, out: File::NULL), "generated code is not valid Ruby"

      compiled = `ruby #{file}`.chomp
      assert_equal interpreted.payload.inspect, compiled
    end
  end

Part CKnowledge check

C1 · Twenty conceptual questions

1. Explain what instance_eval changes and what it leaves alone. Name the four kinds of identifier and how each behaves inside the block.
2. Why is BasicObject the right superclass for a DSL builder, and what does it cost?
3. What is a Binding, and what is block.binding.receiver used for here? What is the conservative alternative?
4. Give three reasons the DSL builds an AST instead of executing as it parses.
5. How does a dynamic language produce build-time errors with file and line numbers?
6. When should you prefer define_method over method_missing, and when is method_missing genuinely necessary?
7. What two methods must always be defined together, and what breaks otherwise?
8. Explain the class-macro pattern using option as the example. What are the two effects of one call?
9. Why must inherited call super?
10. Compare include, prepend, extend and using in one sentence each.
11. Why is a proc's return dangerous when you store user callbacks, and what do you use instead?
12. When does Ruby set Exception#cause? Why did the runner not get one?
13. What is the middleware fold, and why is the list reversed before folding?
14. Why does the runner return a RunResult rather than raising, and what is the cost of that choice?
15. What makes from_h safe for untrusted pipelines, and what would make it unsafe again?
16. Why must policies be idempotent, and how do you detect that they are not?
17. Why does the project produce successor pipelines instead of mutating running ones?
18. What does the frozen_string_literal comment actually do, and what did it measure at?
19. Explain Ruby's GVL in terms of what threads can and cannot speed up.
20. For a DSL gem, what counts as a breaking change under semantic versioning?

C2 · Ten code-reading questions

Predict the output. All ten were run; answers below.

# 1
module A; def who = "A"; end
module B; def who = "B"; end
class C
  include A
  prepend B
  def who = "C"
end
p C.new.who, C.ancestors.first(4)

# 2
def run_block = [yield(1, 2, 3), block_given?]
p run_block { |a, b| [a, b] }

# 3  (no frozen_string_literal comment in this file)
s = "abc".dup
s << "d"
p s, s.frozen?, "abc".frozen?

# 4
Point = Struct.new(:x, :y)
pt = Point.new(1, 2)
pt.x = 9
D = Data.define(:x, :y)
d = D.new(x: 1, y: 2)
p pt.to_a, d.with(x: 9), d.respond_to?(:x=)

# 5
def f(a, *rest, k: 1, **opts) = [a, rest, k, opts]
p f(1, 2, 3, k: 4, z: 5)
h = { k: 9, z: 8 }
p f(1, **h)
p f(1, h)

# 6
@a = false
@a ||= "set"
@b = nil
@b ||= "set"
p @a, @b

# 7
p([1, 2, 3].reduce([]) { |acc, n| acc << n * 2 })
p([1, 2, 3].reduce(0) { |acc, n| acc + n if n.odd? })

# 8
class Gen
  limit = 3
  define_method(:with_closure) { limit }
  def with_def = defined?(limit) ? limit : "limit is not visible here"
end
p Gen.new.with_closure, Gen.new.with_def

# 9
class Ghost
  def method_missing(n, *) = n.to_s.start_with?("go_") ? :ok : super
end
g = Ghost.new
p g.go_home, g.respond_to?(:go_home)

# 10
class Widget; end
Widget.class_eval { def hello = "instance method" }
Widget.instance_eval { def hi = "class method" }
p Widget.new.hello, Widget.hi

1.  "B"  /  [B, C, A, Object]
2.  [[1, 2], true]
3.  "abcd"  /  false  /  false
4.  [9, 2]  /  #<data D x=9, y=2>  /  false
5.  [1, [2, 3], 4, {:z=>5}]
    [1, [], 9, {:z=>8}]
    [1, [{:k=>9, :z=>8}], 1, {}]
6.  "set"  /  "set"
7.  [2, 4, 6]  then NoMethodError (nil accumulator)
8.  3  /  "limit is not visible here"
9.  :ok  /  false
10. "instance method"  /  "class method"

C3 · Five debugging exercises

1. Symptom: a pipeline built in a test has one step instead of two, no error, no warning.

   summarize = stub_step(:summarize) { |items| items.first(2) }
   pipeline = build_pipeline("t") do
     fetch from: "x"
     summarize
   end

2. Symptom: explain reports every step's location as define.rb:113 instead of the user's file.

   def __where__
     frame = ::Kernel.caller_locations(1, 1).first
     "#{::File.basename(frame.path)}:#{frame.lineno}"
   end

3. Symptom: a when_failed handler prints step fetch failed: step fetch failed: timeout, doubled.

   handler.callable.call(error, step)   # error is the StepFailed wrapper

4. Symptom: after adding a policy, every define is slower than the last and eventually raises "rewrite produced 501 steps".

   Automation::Policies.register(:audit) do |pl|
     pl.rewrite { append step(:audit_log) }
   end

5. Symptom: a third-party step works when registered as a lambda and crashes the validator when registered as a custom object.

   params = @registry.fetch(step.name).parameters

C4 · Five implementation exercises

1. Conditional steps. Add only_if and unless options evaluated at run time against the context vars, so fetch from: url, only_if: :full_refresh is skipped unless the var is set. The result must record the step as :skipped, and explain must show the condition.
2. Sub-pipelines. Let a pipeline include another by name (include_pipeline "cleanup"), with cycle detection and a clear error naming the cycle.
3. Parallel fan-out. Implement a fan_out step that runs several sub-pipelines concurrently with a bounded thread pool, merges their payloads, and reports the slowest branch.
4. Persistence and resume. Checkpoint the payload after each step so an interrupted run can resume from the last completed step. Decide what makes a payload safely resumable and document the limits.
5. A second front end. Write a YAML loader producing the same AST as the Ruby DSL, with the same validation and source locations (YAML parsers can report line numbers). Prove with a test that the two front ends produce equal pipelines for equivalent input.

C5 · One substantial challenge

Build a language server for your DSL. Implement a minimal LSP server (stdin/stdout, JSON-RPC) that gives an editor: completion for step names with their documentation, hover showing a step's options and defaults, and diagnostics from the validator with correct ranges, for any file using your gem.

Requirements: it must derive everything from the registry rather than a hard-coded list, so a user's plugins appear automatically; diagnostics must update on save; and it must not execute the pipeline file more than once per change. Hints: the hard part is not the protocol (it is a few message types) but deciding when it is safe to load a user's file, given that loading is executing. Think about a subprocess, a timeout, and what to do when the file has a syntax error.

C6 · You should now be able to explain

• What instance_eval changes, and all four identifier behaviours inside a block.
• The Ruby object model: singleton classes, ancestor chains, and method lookup.
• When metaprogramming is the right tool and the specific costs it imposes on tooling.
• Why an AST is the load-bearing idea in a DSL, not the syntax.
• How a dynamic language produces build-time errors with source locations.
• The class-macro pattern, and therefore how Rails, RSpec and most Ruby libraries work.
• Blocks, procs and lambdas, including the return hazard.
• Exception hierarchies, wrapping, and the limits of cause.
• The middleware pattern and how the chain is built.
• Why executable configuration is remote code execution, and where the trust boundary goes.
• Lazy enumerators, and the tension between streaming and error attribution.
• The GVL: what threads can and cannot make faster.
• Why transformations produce successors rather than mutating.

C7 · You should now be able to implement

• An internal DSL with instance_eval, delegation to the caller, and collision-safe plumbing.
• An immutable AST with source locations, value equality and safe transformations.
• A validator that reports every problem at once with file and line.
• A plugin system with class macros, declared metadata and generated readers.
• An interpreter with a context, middleware, retries and failure handlers.
• Testing helpers other people can use: spies, isolated registries, domain assertions, contract tests.
• Self-description: explain, diagrams, diffs.
• A rewrite API and an idempotent policy layer.
• A gem with a CLI, honest exit codes, a changelog and zero runtime dependencies.
• A data-only loading path for untrusted input, with allow-lists and size bounds.

Part DShipping it: README, portfolio, interview

D1 · README draft

# automation

A Ruby DSL for describing automation pipelines, with the unusual property
that a pipeline is **data you can inspect, validate, diagram and rewrite**
before anything runs.

No runtime dependencies. Ruby 3.2+.

```ruby
Automation.define("research") do
  retry_on Automation::HttpError, times: 3, backoff: :exponential

  fetch     from: "https://example.com/papers.json", limit: 20
  filter    field: :topic, matching: "AI"
  summarize field: :abstract, max_words: 40
  save_to   collection: "knowledge_base"

  when_failed { |error, step| warn "#{step.name}: #{error.message}" }
end
```

    $ automation explain examples/research.rb
    $ automation run examples/research.rb --dry-run
    $ automation graph examples/research.rb > pipeline.mmd

## Why this exists

Every team eventually writes a YAML file that wants to be a program, then
invents conditionals, loops and variables badly. This starts from the other
end: it is Ruby, so loops and conditionals were always there, and the DSL is
a thin surface over an AST that the tooling can reason about.

## What you get

- **Errors before anything runs**, with your file and line:

      pipeline is invalid:
        - research.rb:7: step fetch is missing required option :from
        - research.rb:8: unknown step :summarise; did you mean summarize?
        - research.rb:9: step filter got unknown option :limit; it accepts: field, matching

- **Dry runs** that provably touch nothing (there is a test asserting the
  fake HTTP server receives zero requests).
- **Retries** with linear or exponential backoff, and failure handlers.
- **Inspection**: `explain`, Mermaid diagrams, and diffs between versions.
- **Rewriting**: pipelines are values, so transformations produce new ones.
- **Policies**: organisation-wide rules applied to every pipeline at define
  time (always log first, cap expensive options, require a retry policy).
- **A safe mode**: load pipelines from JSON, executing no code at all.
- **Testing helpers** for your own pipelines and plugins.

## Adding a step

```ruby
class Translate < Automation::Plugin
  step_name :translate
  doc "Translate a field into another language."

  option :field, default: :abstract
  option :to, required: true

  def call(records)
    records.map { |r| r.merge(field => Translator.call(r[field], to: to)) }
  end
end
Translate.register!
```

The declared options drive validation, `--help`, `explain` output and typo
suggestions. Nothing else needs to know your step exists.

## Trust

`automation run file.rb` **executes Ruby**. Only run pipeline files you
trust, exactly as you would with a Rakefile or a Gemfile.

For untrusted input use the data path, which builds the same AST from JSON
and executes no code:

    $ automation run --safe pipeline.json

The registry is the boundary: an untrusted pipeline can only compose verbs
you chose to register.

## Known limitations

- A local variable with the same name as a step silently shadows the verb
  inside a `define` block. Name spies and helpers distinctly.
- Instance variables do not resolve inside the DSL block (`self` is the
  builder). Bind them to locals first.
- `PStore` is the default store; it rewrites the whole collection per
  transaction and does not scale past a few megabytes. Swap in SQLite
  behind `Automation::Store`'s three methods.
- Steps registered as blocks cannot be compiled to standalone Ruby; use
  plugin classes if you need that.

## Licence

MIT

Three deliberate choices in that README. It leads with the property that is unusual (pipelines are inspectable data) rather than with "a DSL for pipelines", which describes fifty gems. It shows real error output, because that is what convinces an experienced reader that the tool was used rather than published. And the known limitations section lists the two DSL traps I fell into, which costs nothing and is the difference between a user trusting your judgement and discovering the traps alone at midnight.

D2 · GitHub project description

A Ruby DSL where pipelines are inspectable data: validated with file and line before anything runs, diagrammable, rewritable, and loadable from JSON with no code execution. No runtime dependencies.

Topics: ruby, dsl, metaprogramming, ast, pipeline, automation, workflow, gem, plugin-architecture, interpreter.

D3 · Performance considerations

• The DSL runs once, at load. Dispatch cost there is irrelevant: method_missing versus generated methods was 20 ns per call, and we chose generated methods for error messages, not speed.
• The interpreter's overhead is a registry lookup and a few lambda calls per step, which is noise next to any step doing real I/O. Compiling (Part B) buys artefact independence, not meaningful speed.
• Allocation is the usual Ruby cost. Frozen string literals took 10,000 allocations down to 2 in the measurement above; symbols allocate nothing; Data nodes cost about 3 objects each, which is a fair price for immutability and value equality.
• Large datasets want lazy enumerators. Measured 0.0002 s over 2M lazy records against 0.38 s over 200k eager ones for a first(5) query. Steps must declare that they stream.
• Threads help I/O-bound steps (0.201 s to 0.069 s for four concurrent sleeps) and never help CPU-bound ones. Bound the pool.
• PStore rewrites the whole collection per transaction. It is the right default and the wrong choice past a few megabytes.

D4 · Security considerations

• A pipeline file is a program. automation run file.rb is ruby file.rb with extra steps. Treat pipeline files exactly as you treat Rakefiles: repository code, reviewed, never accepted from users.
• The data path is the boundary. from_h builds the same AST from JSON with no callables. Bound it: a step limit, an identifier allow-list for names, and a recursive allow-list for option values. Reject handlers in data pipelines, because a handler is a block.
• Safety is a property of the registry, not of the loader. If someone registers a :shell step, the data path executes shell commands as designed. Document which verbs are safe to expose.
• Ruby has no sandbox. $SAFE was removed in 3.0 and never worked well. Do not attempt to sandbox untrusted Ruby in-process; use a separate process with OS-level limits, or do not execute it.
• URLs in options are a request forgery risk. Our HTTP client already restricts schemes to http and https; a production version should also block private address ranges unless explicitly allowed, and cap redirects (we do) and response size (we do not, and should).
• Publish with MFA. rubygems_mfa_required in the gemspec; review your Gemfile.lock; run bundle audit in CI.

D5 · What to put in your portfolio

Present this as a study of what it takes to make a DSL trustworthy, not as "a pipeline runner". The interesting narrative is the sequence of realisations:

1. A block plus instance_eval makes configuration read like a language, in about four lines.
2. Doing that naively breaks in three silent ways, each demonstrated with real output: caller methods swallowed as steps, verbs shadowed by Object methods, typos accepted as valid.
3. The fix that matters is not syntactic: build an AST and execute nothing, which then gives validation with file and line, dry runs, diagrams, diffs and rewriting.
4. Declared plugin metadata drives validation, documentation and typo suggestions from one declaration.
5. An executable configuration format is remote code execution, so there is a second, data-only front end sharing the same AST and engine.

Keep a docs/ folder with the explain output, a rendered Mermaid diagram, and the error-message screenshots. In an interview, the error messages are the artefact to lead with: "here is a dynamic language producing compile-time-quality diagnostics, and here is how".

Cut, if you want it tighter: the terminal-free extras (graph is charming but inessential), and the Milestone 2 Pipeline class, unless you keep it explicitly as the "before" half of the comparison and say so.

D6 · Interview questions someone could ask

D7 · Extensions worth building

• The compiler from Part B, and a --compile flag producing an auditable standalone file.
• The language server from C5, which makes the DSL feel like a first-class language in an editor.
• A scheduler: cron-like recurring runs with history, so explain can show a step's typical duration and failure rate from real data.
• Web UI rendering the Mermaid graph with live run status.
• A second front end (YAML or a tiny external syntax) proving the AST is genuinely the centre.

Course 2 completeWhat you built, and what it argues

A gem with twelve packages, a CLI, a plugin system, testing helpers for its users, two front ends sharing one AST, and 34 tests. More importantly, an argument you can make with evidence: the syntax is the least interesting part of a DSL. Anyone can make configuration look like a language in an afternoon with instance_eval. What makes it usable is everything that follows from refusing to execute while you parse.

Ruby's answer to the curriculum's central question, stated precisely: problems where the shape of the solution is best expressed as a small language, and where you want to grow that language at run time, from inside the program, without a build step or a parser. Ruby makes the front end nearly free and hands you the responsibility for everything a compiler would have done: checking names, reporting locations, keeping tooling informed, and testing enough to survive the absence of a type checker.

Course 5 takes the same ambition and moves it to compile time. Racket's macros turn a language definition into a function from syntax to syntax, checked before the program runs, with source locations carried automatically. When you get there, the thing to compare it against is Milestone 4's caller_locations frame counting and Part B's string-concatenating compiler. Both work. One is doing by hand what the other has as a primitive.