What an experienced Go programmer does next, and whether you can now explain any of it

Part AThe advanced phase

The colony works. These six topics are what you would reach for if this were a system you had to operate rather than a system you had to finish. Each is short, each has code you can drop in, and each names what to measure afterwards.

A1 · Worker pools and bounded concurrency

"One goroutine per ant" is right because ants are the unit of simulation. "One goroutine per unit of work" is wrong for most other things: parsing 50,000 files, calling an API for each of 10,000 records, or replaying a chaos schedule. Unbounded goroutines mean unbounded memory, unbounded file descriptors, and a remote service you have accidentally attacked.

The Go idiom for a bound is a buffered channel used as a counting semaphore. A slot in the buffer is a permit.

// Pool runs at most n tasks at once. The semaphore is a buffered channel:
// each slot in the buffer is a permit.
type Pool struct {
	sem chan struct{}
	wg  sync.WaitGroup
}

func NewPool(n int) *Pool { return &Pool{sem: make(chan struct{}, n)} }

// Go blocks until a permit is free, which is where backpressure happens.
func (p *Pool) Go(ctx context.Context, f func()) error {
	select {
	case p.sem <- struct{}{}:
	case <-ctx.Done():
		return ctx.Err()
	}
	p.wg.Add(1)
	go func() {
		defer p.wg.Done()
		defer func() { <-p.sem }()
		f()
	}()
	return nil
}

func (p *Pool) Wait() { p.wg.Wait() }

Three details that separate this from the versions you find on blogs. The acquire is a select with ctx.Done(), so a saturated pool can still be shut down. The release is a defer inside the goroutine, so a panicking task still returns its permit. And Go blocks rather than queueing, which means the caller feels the backpressure and stops producing work: a queue you cannot see is a queue that grows until you run out of memory.

With generics, the same idea becomes a reusable parallel map:

// MapConcurrent applies f to every element with bounded concurrency and
// returns results in input order. Generics make it reusable; the result
// slice is preallocated so workers write disjoint indices with no lock.
func MapConcurrent[T, R any](ctx context.Context, in []T, workers int,
	f func(context.Context, T) (R, error)) ([]R, error) {

	out := make([]R, len(in))
	errs := make([]error, len(in))

	pool := NewPool(workers)
	for i, v := range in {
		if err := pool.Go(ctx, func() {
			out[i], errs[i] = f(ctx, v)
		}); err != nil {
			break
		}
	}
	pool.Wait()

	for _, err := range errs {
		if err != nil {
			return out, err
		}
	}
	return out, ctx.Err()
}

[T, R any] declares two type parameters; any is the constraint meaning "any type at all". The important trick is not the generics, it is that writing to distinct indices of a preallocated slice from many goroutines is race-free. Each index is a separate memory location and the slice header never changes, so no synchronisation is needed. Appending to a shared slice would be a race; this is not. That distinction is worth internalising, because it turns a whole class of "collect the results" problems into lock-free code.

What to measure: throughput against worker count, and the memory high-water mark with and without the bound. The optimum worker count for CPU work is around GOMAXPROCS; for I/O-bound work it is much higher and should be found empirically.

A2 · A taxonomy of backpressure

You have now implemented three of these four without naming them. Naming them is what lets you choose deliberately.

The one you have not built is admission control: refusing work at the front door so that accepted work still completes on time. It is the difference between a system that degrades and a system that collapses, and the shape is three lines:

	if depth := len(e.reqs); depth > e.cfg.QueueSize*3/4 {
		e.M.Rejected.Inc()
		return response{err: ErrOverloaded}   // fail fast, do not queue
	}

The counter-intuitive part is that rejecting 10% of requests instantly can leave the other 90% faster than accepting 100% and serving them all slowly, because queueing delay compounds. When every request waits 8 ms behind a queue, clients time out and retry, which adds load, which lengthens the queue. Little's Law states it precisely: for a stable system, average queue length = arrival rate × average time in system. You cannot choose all three, so choose which one you will control.

What to measure: queue depth as a time series (not an average), latency percentiles at several arrival rates, and the point where p99 latency turns sharply upward. That knee is your capacity, and it is always lower than the throughput number you can quote.

A3 · Testing concurrency like you mean it

Five techniques, in increasing order of how much they will surprise you.

1. Run tests repeatedly under the race detector. go test -race -count=20 ./... in CI. A single pass proves almost nothing about interleaving, and twenty passes cost seconds.

2. Keep a deterministic oracle. This is why the sequential Sim from Milestone 3 was never deleted. It is slow, boring and exactly reproducible, which makes it the reference implementation: given the same seed and the same number of ticks, the concurrent engine should agree with it on invariants (food conserved, no negative cells, ants inside the grid) even though it cannot agree on exact positions. A slow, obviously-correct implementation is a testing asset, not dead code.

3. Test invariants, not outputs. Every genuine bug in this project was caught by conservation, not by an expected value. Concurrent systems have no expected output; they have properties that must hold.

4. Fuzz the behaviours. A Behaviour is untrusted input to the engine, so fuzz it:

func FuzzApplyAction(f *testing.F) {
	f.Add(5, 5, 1, 0, uint8(1))
	f.Fuzz(func(t *testing.T, x, y, dx, dy int, kind uint8) {
		e := NewEngine(Config{Width: 16, Height: 16, Ants: 1}, Forager{})
		act := Action{Kind: ActionKind(kind % 4), Dir: world.Position{X: dx, Y: dy}}
		// The engine must never panic, whatever an ant asks for.
		e.applyAction(request{id: 0, pos: world.Position{X: x, Y: y}, act: act})
	})
}

Run with go test -fuzz FuzzApplyAction. The fuzzer mutates inputs, keeps anything that reaches new code paths, and writes failing cases to testdata/fuzz/ where they become permanent regression tests. It found nothing in my version because every grid access is bounds-checked, which is precisely the kind of "nothing" you want evidence for.

5. Detect leaks explicitly. Our poll-and-compare test is the homemade version; go.uber.org/goleak is the real one and reports the leaked stacks. Add it to TestMain and every test in the package is covered at once.

A4 · Deterministic chaos

Milestone 8's chaos rolls dice inside each ant, which makes a failing run unreproducible in the one way that matters: you cannot replay it. The fix is to decide the failures up front.

// Schedule is a precomputed list of failures. Because it is built once from
// a seed, a chaotic run replays exactly, which is the difference between a
// bug you can fix and a bug you can only describe.
type Schedule struct {
	faults []Fault
	next   int
}

func BuildSchedule(seed uint64, ticks, ants int, ratePerTick float64) *Schedule {
	rng := rand.New(rand.NewPCG(seed, 0xA5A5))
	var faults []Fault
	for t := range ticks {
		if rng.Float64() >= ratePerTick {
			continue
		}
		faults = append(faults, Fault{
			Tick:  t,
			AntID: rng.IntN(ants),
			Kind:  FaultKind(rng.IntN(3)),
		})
	}
	return &Schedule{faults: faults}
}

// Due reports the faults scheduled at or before tick t. Called only by the
// owner goroutine, so it needs no synchronisation and no allocation.
func (s *Schedule) Due(t int) []Fault {
	start := s.next
	for s.next < len(s.faults) && s.faults[s.next].Tick <= t {
		s.next++
	}
	return s.faults[start:s.next]
}

Now a failing CI run reports "seed 8841, 400 ticks" and you reproduce it exactly on your laptop. The schedule can also be written to a file and checked in next to the bug it exposed.

This is one step short of what the state of the art does. FoundationDB and TigerBeetle run their entire system inside a deterministic simulator where time, scheduling, disk and network are all controlled by one seed, so every interleaving is reproducible and the fuzzer can search the space of schedules. That is a large engineering investment and it is why those systems are unusually trustworthy. Knowing it exists changes what you consider possible.

What to measure: that the same seed produces byte-identical failure output twice. If it does not, you still have a source of nondeterminism you have not captured.

A5 · Structured logs and the execution tracer

Counters tell you a rate; logs tell you a story. Go 1.21 added log/slog to the standard library, so structured logging no longer needs a dependency:

	logger := slog.New(slog.NewJSONHandler(os.Stdout, &slog.HandlerOptions{
		Level: slog.LevelInfo,
	}))
	slog.SetDefault(logger)

	slog.Info("colony started", "ants", cfg.Ants, "grid", cfg.Width, "seed", cfg.Seed)

	// A logger carrying context, created once per ant rather than per line.
	log := slog.With("ant", a.ID, "gen", a.Generation)
	log.Warn("restarting after crash", "pos", a.Pos.String(), "carrying", a.Carrying)

Key-value pairs rather than formatted strings, so a log aggregator can filter on ant=4471 without regular expressions. Two rules for a system like this one: never log per tick (three million requests a second will produce more log volume than simulation), and sample or aggregate anything that can happen per ant. A log line costs microseconds; a counter costs nanoseconds. Log the exceptional, count the routine.

Then there is the tool almost nobody uses and everybody should:

$ curl -o trace.out "127.0.0.1:9090/debug/pprof/trace?seconds=5"
$ go tool trace trace.out

This opens a browser view of what every goroutine did, when, on which processor, including scheduler latency, garbage collection pauses, syscall blocking and network waits. Where pprof tells you where time went, the tracer tells you why a goroutine was not running. For this project the interesting view is "Synchronization blocking profile", which shows the ants queued behind the owner as a wall of parked goroutines. No other mainstream language ships anything this good with the runtime.

A6 · Shipping it

# a stamped, stripped, static binary
go build -trimpath -ldflags "-s -w -X main.version=$(git describe --tags --always)" \
   -o bin/antfarm ./cmd/antfarm

# cross-compile: no toolchain to install, no container needed
GOOS=linux GOARCH=arm64 go build -o bin/antfarm-linux-arm64 ./cmd/antfarm

-trimpath removes your home directory from the binary, -s -w strip debug information (halving the size, at the cost of readable panic traces: keep them if you value stack traces more than megabytes), and -X sets a string variable at link time, which is the standard way to embed a version. Cross-compilation needs nothing installed, which is genuinely unusual and is why so much infrastructure tooling is written in Go.

FROM golang:1.25 AS build
WORKDIR /src
COPY . .
RUN CGO_ENABLED=0 go build -trimpath -ldflags "-s -w" -o /antfarm ./cmd/antfarm

FROM gcr.io/distroless/static-debian12
COPY --from=build /antfarm /antfarm
EXPOSE 8080 9090
USER 65534:65534
ENTRYPOINT ["/antfarm"]

CGO_ENABLED=0 produces a binary with no libc dependency, which is what makes the final image a few megabytes with no operating system in it. Distroless (or scratch) means no shell, no package manager and nothing for an attacker to pivot into.

Part BThe final challenge

Everything up to here had a solution a few paragraphs later. This one does not, and you should spend real time on it before opening the last section. It is deliberately at the edge of what you can now do, and it is the kind of thing that makes a portfolio project memorable.

Migrating ants across a sharded, failing world

Milestone 12 put the whole world in one remote process. Now split it: K world processes, each owning a region of the grid, with ants that walk from one region into another.

Requirements

1. Run K world nodes (start with 3), each owning a contiguous band of rows, configured statically at startup. Each exposes the Milestone 12 protocol for its own region.
2. Run an ant process with N ants. Each ant talks to whichever node owns the region it currently occupies.
3. When an ant's move crosses a region boundary, its state (ID, Carrying, Energy, Generation) must migrate to the new node's ant table. Exactly once. An ant may never exist on two nodes, and may never disappear.
4. Nodes fail. Kill any node at any moment, including mid-migration, and the colony must continue: ants in the dead region wait, ants elsewhere keep working, and a restarted node rejoins.
5. Detect failure with heartbeats. A node that has not been heard from within a configurable window is marked unavailable; ants that would enter its region bounce back instead.
6. Observability: per-node ant count, migrations per second, in-flight handoffs, failed handoffs, duplicate-suppression hits, and node up/down state, all on the existing /metrics endpoint.

Constraints

• No external dependencies. No etcd, no Redis, no message broker. Standard library only.
• No global lock and no single coordinator process. The nodes talk to each other directly.
• Handoff must be safe under message loss, message duplication and node death at any step. Assume the network can drop, delay and duplicate, but not corrupt.
• Bounded memory: no structure may grow without limit, including whatever you use to suppress duplicates.

Acceptance criteria

1. Conservation under chaos. A 60-second run with 3 nodes, 1,000 ants, one node killed and restarted every 10 seconds, ends with: food delivered + food on the ground + food carried + food lost equal to the starting total, where "lost" is explicitly accounted rather than unexplained.
2. No duplicates. At the end, the union of all node ant tables contains each ant ID exactly once. Assert it in a test.
3. Liveness. Food delivered during the chaotic run is at least 50% of a clean run of the same length. A system that survives by doing nothing does not pass.
4. Clean tooling. go test -race -count=5 ./... passes, the leak test still passes, and go vet is silent.
5. Documented protocol. A docs/handoff.md with a message diagram and a table of every failure point and what happens there.

Hints, in increasing order of how much they give away

• The hard part is not moving the ant. It is that "send the ant and delete it locally" has no safe ordering: delete first and a lost message loses the ant; send first and a lost acknowledgement duplicates it.
• You cannot make a network message exactly-once. You can make it at-least-once and make the receiver idempotent, which is indistinguishable from exactly-once to everyone downstream. What does the receiver need to remember, and for how long?
• Think of an ant as having exactly one owner at a time, and the handoff as transferring a lease. What state does the sender need between "I no longer own it" and "they confirmed they do"?
• The bounded-memory constraint is about the deduplication table. Migration IDs must be forgettable eventually. What makes it safe to forget one?
• Heartbeats give you failure detection, not failure knowledge. A node you cannot reach may be alive and reachable by someone else. What does your design do if two nodes disagree about whether a third is up?

Attempt it before reading on. Even a partial implementation with an honest failure analysis is worth more than the section below.

  Node A (source)                        Node B (destination)
  ───────────────                        ───────────────────
  ant crosses boundary
  mid := uuid()
  state := freeze(ant)      ─ HANDOFF(mid, ant) ─►
  mark ant "migrating"                           if seen(mid): reply ACK (dedupe)
  (still owned by A, frozen)                     else: insert ant, remember mid
                            ◄──── ACK(mid) ─────  reply ACK
  delete ant locally
  remember mid until B's
  ack-of-ack window closes

  retry HANDOFF until ACK (at-least-once)
  B suppresses duplicates by mid (idempotent)

type MigrationID struct {
	From uint64 // node ID, so IDs from different nodes cannot collide
	Seq  uint64 // monotonic per node
}

type handoff struct {
	ID    MigrationID
	Ant   sim.Ant
	Tick  uint64 // source node's logical clock, for the dedupe window
}

// receive is idempotent: the same handoff delivered ten times inserts one
// ant and sends ten acks.
func (n *Node) receive(h handoff) ack {
	n.mu.Lock()
	defer n.mu.Unlock()

	if _, dup := n.seen[h.ID]; dup {
		n.M.DedupeHits.Inc()
		return ack{ID: h.ID, OK: true}   // re-acknowledge, do not re-insert
	}
	n.seen[h.ID] = h.Tick
	n.ants[h.Ant.ID] = &h.Ant
	n.M.Migrations.Inc()
	return ack{ID: h.ID, OK: true}
}

// send retries until acknowledged or the destination is declared down.
func (n *Node) send(ctx context.Context, to NodeID, a *sim.Ant) error {
	h := handoff{ID: n.nextMigrationID(), Ant: *a, Tick: n.tick.Load()}
	n.markMigrating(a.ID)

	for attempt := 0; ; attempt++ {
		if !n.membership.Up(to) {
			n.unmarkMigrating(a.ID)      // bounce: keep the ant here
			n.M.FailedHandoffs.Inc()
			return ErrDestinationDown
		}
		if _, err := n.peer(to).Handoff(ctx, h); err == nil {
			n.delete(a.ID)               // only now is it safe to forget
			return nil
		}
		select {
		case <-time.After(backoff(attempt)):
		case <-ctx.Done():
			n.unmarkMigrating(a.ID)
			return ctx.Err()
		}
	}
}

Part CKnowledge check

C1 · Twenty conceptual questions

1. Define a data race precisely, in terms of the Go memory model. Why is "two goroutines write at once" an incomplete definition?
2. The race detector has no false positives but many false negatives. Explain both halves.
3. Why is an unbuffered channel send a synchronisation point, and what does that guarantee about memory written before it?
4. When is closing a channel the correct shutdown signal, and when is it a panic waiting to happen?
5. Why can a nil channel in a select be useful rather than a bug?
6. Explain why select chooses randomly among ready cases, and give a concrete bug that behaviour caused in this project.
7. Contrast a mutex-protected world with a single owner goroutine. Name two things the owner gives you that the mutex cannot, and one thing it does not give you.
8. Why must wg.Add not run concurrently with wg.Wait, and how did this project guarantee that?
9. Cancellation in Go is cooperative. What follows from that for a goroutine in a tight loop, and for one blocked on a socket read?
10. Why does a context need to be passed as a parameter rather than stored in a struct?
11. An interface value holding a nil pointer is not nil. Explain the representation that makes this true and the coding rule that avoids it.
12. When should a method use a pointer receiver, and what goes wrong with a map of struct values?
13. Why does append return a slice, and what is the classic aliasing bug?
14. Why is writing to distinct indices of a preallocated slice from many goroutines safe, while appending to a shared slice is not?
15. Compare panic/recover with exceptions. Where is recover legitimate, and why is a recover in every goroutine a bad idea?
16. Explain why an unrecovered panic in one goroutine kills the process, and what that implies for supervision design.
17. Give three distinct backpressure policies, the code shape of each, and a message type in this project suited to each.
18. Why did raising message loss from 2% to 25% cut throughput by 96% while the absolute number of dropped messages stayed flat?
19. Go has no atomic float64. Give two workarounds and the trade-off of each.
20. The same operation measured 29 ns locally and 10.7 µs over loopback TCP. What does that ratio imply for how you choose service boundaries?

C2 · Ten code-reading questions

Predict the output or behaviour of each, then check. All ten were run to confirm the answers.

// 1
a := []int{1, 2, 3, 4, 5}
b := a[1:3]
b = append(b, 99)
fmt.Println(a, len(b), cap(b))

// 2
func() {
	for i := range 3 {
		defer fmt.Print(i, " ")
	}
}()

// 3
var m map[string]int
fmt.Println(m["x"], len(m), m == nil)
// and then: m["x"] = 1

// 4
var nilCh chan int
select {
case <-nilCh:
	fmt.Println("received")
default:
	fmt.Println("default")
}

// 5
ch := make(chan int, 2)
ch <- 7
close(ch)
v1, ok1 := <-ch
v2, ok2 := <-ch
fmt.Println(v1, ok1, v2, ok2)

// 6
type OOB struct{ X int }
func (e *OOB) Error() string { return "oob" }
func find() *OOB { return nil }

var err error = find()
fmt.Println(err == nil, find() == nil)

// 7
type Ant struct{ Energy int }
func (a Ant) Drain() { a.Energy = 0 }

ant := Ant{Energy: 100}
ant.Drain()
fmt.Println(ant.Energy)

// 8
done := make(chan int, 3)
for i := range 3 {
	go func() { done <- i }()
}
sum := 0
for range 3 {
	sum += <-done
}
fmt.Println(sum)

// 9
type P struct{ X, Y int }
seen := map[P]int{{1, 2}: 5}
fmt.Println(P{1, 2} == P{1, 2}, seen[P{1, 2}], seen[P{9, 9}])

// 10
var wg sync.WaitGroup
for i := range 3 {
	go func() {
		wg.Add(1)
		defer wg.Done()
		work(i)
	}()
}
wg.Wait()
fmt.Println("done")

1.  [1 2 3 99 5] 3 4
2.  2 1 0
3.  0 0 true      then: panic: assignment to entry in nil map
4.  default
5.  7 true 0 false
6.  false true
7.  100
8.  3
9.  true 5 0
10. prints "done" immediately, usually before any work runs

C3 · Five debugging exercises

Each gives a symptom and a suspect. Diagnose before opening the answer.

1. Symptom: the simulation freezes after a few seconds. Every goroutine profile line shows ants blocked in roundTrip, and the owner goroutine shows blocked in handle at a channel send. Nothing crashes; the process sits at 0% CPU forever.

   c := &antClient{reply: make(chan response)}   // note: unbuffered

2. Symptom: fatal error: concurrent map writes, roughly once every twenty runs, always under load, never under a debugger.

   type Scout struct{ headings map[int]world.Position }
   
   func (s *Scout) Decide(a *Ant, sn Sense, rng *rand.Rand) Action {
   	if _, ok := s.headings[a.ID]; !ok {
   		s.headings[a.ID] = randomDir(rng)
   	}
   	return Action{Kind: ActMove, Dir: s.headings[a.ID]}
   }

3. Symptom: memory grows steadily at about 40 MB per minute. The heap profile shows nothing large; runtime.NumGoroutine() climbs from 1,500 to 40,000 over ten minutes.

   func (e *Engine) evaporate(ctx context.Context) {
   	for {
   		select {
   		case <-ctx.Done():
   			return
   		case <-time.After(e.cfg.EvaporateEvery):
   			e.reqs <- request{kind: reqEvaporate}
   		}
   	}
   }

4. Symptom: the conservation test fails intermittently with food increasing. Race detector clean. Only happens with chaos enabled.

   	case ActPickUp:
   		a.Carrying = true
   		if err := e.world.TakeFood(r.pos); err != nil {
   			return response{err: err}
   		}
   		return response{ok: true}

5. Symptom: after adding a second metrics endpoint, all counters read zero on the new one while the old one works.

   func startSecond(m metrics.Set) func() {      // note: by value
   	return serve(":9091", m.Handler(), "metrics2")
   }

C4 · Five implementation exercises

1. Deterministic replay. Add a recording mode that writes every nondeterministic decision (chaos events, RNG draws that matter, scheduling-visible ordering) to a file, and a replay mode that reproduces a run exactly from it. Acceptance: a chaotic run that fails the conservation test replays and fails identically ten times out of ten.
2. Region-aware behaviour. Implement a Behaviour that maintains a small per-ant memory of where food was found (last three positions, on the Ant struct) and biases search toward those areas when pheromone is absent. Measure whether it beats TrailFollower over ten seeds, and report both mean and win count.
3. Adaptive evaporation. Make the evaporation rate a function of colony state: faster when delivery rate is falling (trails are stale), slower when it is rising. Acceptance: it must beat the best fixed rate you found in Milestone 6's sweep on at least seven of ten seeds.
4. A multiplexed client. Exercise 12 from the last instalment: request IDs, one reader goroutine, many ants per connection, bounded memory, -race clean with 1,000 ants on one connection.
5. Snapshot and restore. Serialise the entire world and colony to a file and restore it, such that a restored run continues identically to one that was never interrupted (with chaos off). Acceptance: run 5 seconds, snapshot, run 5 more; versus run 10 seconds. Identical final statistics.

C5 · One substantial challenge

Distinct from the final challenge in Part B, and smaller, but not easy.

Build a scheduling fuzzer. Make the interleaving itself a controlled variable. Introduce a Scheduler abstraction that every goroutine in internal/sim consults at yield points (before a send, after a receive, around each tick), which in test mode advances goroutines in an order chosen by a seed. Then write a test that runs 10,000 seeds against the conservation invariant and reports any seed that breaks it.

Requirements: production mode must compile to zero overhead (the scheduler calls must be inlinable no-ops, verified with -gcflags=-m); the fuzzer must find the "select chose Done and discarded the reply" bug from Milestone 11 when you reintroduce it; and a failing seed must be replayable. Hints: an interface with two implementations, one of which has empty methods; the real difficulty is not the scheduler but finding the right yield points, and the answer is "wherever the program's behaviour could depend on ordering".

C6 · You should now be able to explain

• The Go memory model in terms of happens-before, and how channel operations and mutexes create it.
• Why goroutines are cheap and what the runtime does when one blocks.
• Channel semantics in full: buffered, unbuffered, nil, closed, directional.
• select, including random choice, default, and disabling a case with nil.
• Context trees, derived cancellation, and why cancellation is cooperative.
• The difference between concurrency and parallelism, with evidence from your own benchmarks.
• Ownership as a design tool: owned, shared-with-a-lock, atomic, and immutable-after-construction.
• Interfaces, implicit satisfaction, the nil-interface trap, and "accept interfaces, return structs".
• Error values, wrapping, errors.Is/As, and when a custom type beats a sentinel.
• Why panics are process-fatal, and what supervision therefore costs in Go.
• Backpressure policies and how to choose one per message type.
• How to read a CPU profile, a heap profile and a goroutine profile, and what each answers.
• Why timeouts amplify failure, and the standard mitigations.
• The cost of a process boundary, in nanoseconds you have measured yourself.
• Why exactly-once delivery is impossible and how at-least-once plus idempotency substitutes for it.

C7 · You should now be able to implement

• A concurrent simulation with per-agent goroutines and no data races.
• A single-owner service with a request/response protocol over channels.
• Graceful, ordered shutdown with contexts and WaitGroups, proven leak-free by a test.
• A supervisor with restart backoff, jitter and an intensity limit.
• Fault injection as a configurable subsystem, with a deterministic schedule.
• Lock-free counters and a bucketed latency histogram.
• A Prometheus-format metrics endpoint with pprof mounted safely.
• A server-sent-events stream that never blocks its producer.
• Sharded state with atomic reads and single-owner writes.
• A TCP server with one goroutine per connection, a connection registry and correct shutdown.
• A reconnecting client with jittered backoff and socket deadlines.
• Table-driven tests, benchmarks, fuzz targets and invariant tests.

Part DShipping it: README, portfolio, interview

D1 · README draft

# antfarm

A concurrent ant-colony simulation used as a laboratory for Go concurrency,
fault tolerance and observability. Thousands of ants, each an independent
goroutine, forage for food and lay pheromone trails in a world owned by a
single goroutine. Failures are injected on purpose; everything is measured.

No third-party dependencies. Standard library only.

## What it does

- Each ant runs in its own goroutine and communicates by message passing.
- Exactly one goroutine owns the world; there is no mutex in the engine.
- Ants lay pheromone trails on the way home, which measurably improves
  foraging (~25% more food delivered than a random walk, same seed and budget).
- Chaos injection: random crashes, dropped messages and slow workers, with a
  supervisor that restarts crashed ants and accounts for their lost cargo.
- Prometheus-format metrics, pprof, a terminal renderer and a browser view
  fed by server-sent events.
- A sharded engine with atomic reads that is 1.4-2.9x faster than the
  single-owner engine on a single core.
- The world can run in a separate process; ants reconnect when it restarts.

## Quick start

    go run ./cmd/antfarm -ants 2000 -grid 128x128 -food 10 -duration 10s \
        -metrics :9090 -view :8080

Then open http://localhost:8080 for the live view and
http://localhost:9090/metrics for the counters.

    go run ./cmd/antfarm -term -ants 500        # terminal renderer
    go run ./cmd/antfarm -chaos crash=0.001,drop=0.02,slow=0.005

## Flags

    -grid WxH              world size (default 64x64)
    -ants N                number of ants
    -food N                number of food sources
    -food-per-source N     units per source
    -behaviour forager|trail
    -seed N                same seed replays the same run
    -duration D            how long to run
    -chaos k=v,...         crash, drop, slow, slowfor
    -metrics addr          serve /metrics and /debug/pprof
    -view addr             serve the browser view
    -term                  draw in the terminal

## Architecture

    cmd/antfarm     flags, wiring, signals, HTTP servers
    internal/sim    ants, behaviours, engines, chaos, supervision
    internal/world  grids, food, pheromone, atomic grid
    internal/metrics counters, histograms, exposition, pprof
    internal/view   terminal renderer, SSE stream, browser client
    internal/netsim gob-over-TCP server and reconnecting client

Two engines are kept deliberately:

- `Engine` (single owner) is the correct, readable design. Read this first.
- `FastEngine` (sharded, atomic reads) is the profile-driven rewrite.

The mutex-based version from the concurrency milestone is also kept, in
`concurrent.go`, because the point of this project is the comparison.

## Testing

    go test ./...                 # everything
    go test -race -count=5 ./...  # what CI runs
    go test -bench . ./...        # benchmarks

Tests are invariant-based: the central one asserts that food is conserved
(delivered + on the ground + carried + lost equals the starting total) under
concurrency, chaos and shutdown. It has caught every real bug in this project.

## Measurements

All on one core, Go 1.22:

| Thing                             | Result                    |
|-----------------------------------|---------------------------|
| Sense, in-process (atomic reads)  | 29 ns                     |
| Sense, over loopback TCP          | 10.7 µs (368x)            |
| Round trip p50 / p99, 1500 ants   | 2 µs / 8 ms               |
| Requests handled, 4 s run         | 3,034,578                 |
| Trail following vs random walk    | +25% food delivered       |
| Sharded vs single owner           | 1.4-2.9x throughput       |

## Known limitations

- Ant state is in memory; a crashed process loses it by design.
- The network protocol has no request IDs, so one connection serves one ant.
- The distributed mode has no consensus, so a partition can split ownership.

## Licence

MIT

Three things that README does deliberately. It leads with measurements rather than adjectives. It keeps the "worse" implementations and explains why, which signals that the comparison was the point. And it has a known limitations section, which is the single strongest credibility signal a portfolio project can carry: it says you understand your own design well enough to know where it ends.

D2 · GitHub project description

A concurrent ant-colony simulation in Go, used as a laboratory for message passing, supervision, chaos injection and observability. Thousands of goroutines, no mutex in the engine, everything measured.

Topics: go, golang, concurrency, goroutines, channels, simulation, agent-based-model, chaos-engineering, observability, distributed-systems, pprof, ant-colony-optimization.

D3 · Performance considerations

• Sensing dominates traffic. Two-thirds of all messages were sense requests; eliminating them with atomic reads was worth more than every other optimisation combined.
• The owner is a serialisation point by design. Throughput ceiling is one goroutine's worth of world operations. Shard when measurement says so, not before.
• Preallocate everything sized by config. make([]*Ant, 0, n) versus growth is roughly twenty allocations and a hundred thousand copies at 50,000 ants.
• Zero allocations in the tick path is achievable and worth protecting with a -benchmem test.
• Instrumentation is not free: two time.Now() calls per round trip cost about 15% of CPU at three million round trips per second. Sample.
• Queue capacity is a latency policy. Small queues block producers, large queues hide latency until clients time out. Publish depth; alert on it.
• Set GOMAXPROCS and GOMEMLIMIT in containers.

D4 · Security considerations

• pprof is a remote information disclosure surface. Heap dumps can contain user data, cmdline leaks arguments, and profile lets anyone impose CPU load. Mount it explicitly (never via DefaultServeMux), bind it to localhost, and reach it through a tunnel.
• The gob protocol is unauthenticated and unencrypted. Anyone who can reach the port can command the world. Add TLS with crypto/tls and a shared credential before this leaves a laptop; gob will also happily decode adversarial input, so treat the port as trusted-network-only.
• Unbounded decoding is a denial-of-service vector. A hostile client can announce a huge structure and make the server allocate. Cap message size with an io.LimitReader around the connection.
• Every HTTP server needs timeouts. The zero-value http.Server has none; a slow client can hold connections indefinitely. ReadHeaderTimeout at minimum.
• The browser view echoes nothing user-supplied today. If you add a name or a label, remember that fmt.Fprint into HTML is an injection; use html/template, which escapes by context.
• Resource limits are policy. -ants 100000000 will exhaust memory. Validate flags, and cap what a remote caller can request (the frame cols parameter comes from a query string).

D5 · What to put in your portfolio

Do not present this as "an ant simulation". Present it as what it is: a study of one concurrency problem solved four ways, with measurements. The narrative that makes it interesting is the sequence of failures.

1. Naive shared state, broken, with the race detector output as evidence.
2. A mutex: correct, and 1.76× slower than single-threaded. Show the benchmark.
3. A single owner goroutine: no locks, failure becomes expressible, still serialised, and honest about that.
4. Profile-driven sharding with atomic reads: 1.4–2.9× faster, and a conservation bug at shutdown that the race detector could not see.
5. The same protocol over TCP, unchanged, and the 368× latency cost of the boundary.

Keep a docs/ folder with the profile output, the benchmark table and one architecture diagram. Screenshot the browser view for the README. A reviewer who spends ninety seconds on your repository should come away knowing you can measure, not just build.

What to cut if you want it tighter: the terminal renderer (charming, not load-bearing) and the sequential Sim, unless you keep it explicitly as a test oracle and say so.

D6 · Interview questions someone could ask, and what a good answer contains

D7 · Extensions worth building

• Multiple colonies competing for the same food, with territorial pheromone. The interesting part is that the world owner now arbitrates conflict.
• Evolution: behaviour parameters (bias, run length) inherited with mutation, selected by delivery rate. You have the measurement harness already.
• Obstacles and a real pathfinding baseline to compare emergent trails against A*. Quantifies how good the colony actually is.
• A WebAssembly build (GOOS=js GOARCH=wasm) so the simulation runs in the browser with no server.
• The deterministic simulator from C5, which is the most educationally valuable of these by a distance.

Course 1 completeWhat you built

2,961 lines of dependency-free Go across six packages, four engine designs with measurements comparing them, a supervisor, a chaos subsystem, a metrics and profiling endpoint, two viewers, and a network protocol that survives its server being killed. More importantly: a habit of measuring before optimising, and of testing invariants rather than outputs.

The central question of this curriculum was what kinds of problems does this language make unusually natural to solve? Go's answer, stated as precisely as this project allows: problems with many independent activities that need to communicate, where you want the concurrency to be visible in the code, the failures to be detectable at run time, and the result to be a single binary you can deploy and profile in production. Not the fastest, not the safest, not the most expressive. The one where a competent team can build a correct concurrent system quickly and then find out what it is actually doing.

Next instalment begins Course 2: Ruby, and the automation DSL. The change of gear is total. Nothing will be about throughput; everything will be about expressiveness, and the first question will be why a configuration file should be a program.