---
title: 'The Model–View–Controller Framework'
source: 'https://academia.sh/en/courses/mvc-framework'
language: en
updated: '2026-10-21T05:03:40+00:00'
license: 'CC BY-SA 4.0'
lessons: 13
outcomes:
    - 'Builds the request flow of a web application with model, view and controller layers'
    - 'Explains how conventions, routing and filters decide which code a request reaches'
    - 'Manages the data model with migrations, relationships and validations, and fixes the N+1 query problem by measuring it'
    - 'Distinguishes by measurement whether a value stays only for this request, for the next request, for the session, or is persistent'
    - 'Lightens the request with background jobs and caching, and applies authentication and secure defaults'
---

# The Model–View–Controller Framework

In a web framework every request starts from scratch, yet some values do not vanish at the end of the request: a flash message appears only on the next page, the signed-in user's id is present in every request from that browser, and a saved product can be read by another browser too. The course measures every mechanism with a single question — how many requests does this value stay for? Conventions, routing, controllers, views, object–relational mapping, migrations, validations, queries, authentication, background jobs, caching, testing and secure defaults all pass through the same frame.

## Framework Structure

A framework that derives template and file names from convention rather than configuration, routing that turns path patterns into a controller and an action, filters that run before the action and flash messages carried to the next request, and views compiled once and escaped on every render.

### Convention over Configuration

Source: https://academia.sh/en/courses/mvc-framework/convention-over-configuration

The measuring axis of the course is how many requests a value stays for. The lesson runs the core, sorts thirteen observations into four stays, and shows that the template name is not written, is derived from names, and stays for this request.

The previous course closed with declarative interfaces: `field :price, 0` was a call, and it added methods to a class. A web framework is built on lines like these, but it asks something new. On the web every request starts from zero: the controller is built fresh, the parameters are empty, the request-specific store is empty. Even so, some values do not vanish when the request ends. A notification message (flash) appears on the next page, the id of a signed-in user is present on every page, and a saved product is read by another browser as well.

The course measures every mechanism with a single question: **how many requests does this value stay for?** The name of the answer's category is the **stay**, and it has four values. **This request**: the value is read only in the request that wrote it. **Next request**: the value is read only in the same client's next request. **Session**: the value is read in all of the same client's later requests, and not read by another client. **Persistent**: the value is also read in another client's request. In the output these four names are written in ASCII: `this_request`, `next`, `session`, `persistent`.

#### Model, View, Controller

**Model–view–controller (MVC)** splits an application into three responsibilities. The **model** carries the data and the rules about the data. The **view** turns the data into the output the user will see. The **controller** receives the request, asks the model, and hands the result to the view. A fourth part stands in front of the three: **routing** turns the incoming path into the **action** of a particular controller.

A request's path follows this order: routing, controller, model, view. In the core of the course each of the four parts exists as a small counterpart: `route`, `ProductsController`, `Product` and `compile_template`. This core is not a framework, it is a model of a framework. Every measured block is plain Ruby; no ready-made framework is used.

#### The Core

The core is two files. The first is `stay.rb`, a small request loop and a helper that classifies the stay. This block is source; it has no output, it is written to disk, and the other files pull it in with `require_relative`.

```ruby
# stay.rb — a small request loop and a helper that classifies how many requests a value stays for

# Flash: a value written in this request is carried to the next request only.
class Flash
  def initialize(incoming)
    @incoming = incoming
    @fresh = {}
  end

  def [](key) = @fresh.fetch(key) { @incoming[key] }

  def []=(key, value)
    @fresh[key] = value
  end

  def passed_on = @fresh
end

# Every request gets a new context: params, flash and the request store start empty each time.
class Context
  attr_reader :app, :params, :session, :flash, :request_store

  def initialize(app, session)
    @app = app
    @params = {}
    @session = session
    @flash = Flash.new(session.delete(:flash) || {})
    @request_store = {}
  end
end

# The application store is the same across all requests and all clients;
# the session is chosen per client.
class App
  attr_reader :store

  def initialize
    @sessions = Hash.new { |h, client| h[client] = {} }
    @store = { products: {}, schema: { products: %i[name price] }, templates: {},
               queue: [], cache: {}, counter: 0 }
  end

  def request(client)
    context = Context.new(self, @sessions[client])
    result = yield context
    context.session[:flash] = context.flash.passed_on unless context.flash.passed_on.empty?
    result
  end
end

# Rule: request 1 (client A) writes the value; requests 2 and 3 (A) and request 4 (B) read it.
# If B sees it, persistent; if A's third request sees it, session; if only the second
# request sees it, next; if none sees it, this_request.
def stay(write, read)
  app = App.new
  first = app.request("A") { |b| write.call(b); read.call(b) }
  raise ArgumentError, "value could not be read in the request that wrote it" if first.nil?

  seen = [["A", 2], ["A", 3], ["B", 4]].map do |client, order|
    app.request(client) { |b| read.call(b) }.nil? ? "--" : "#{client}#{order}"
  end
  kind = if seen[2] != "--" then "persistent"
         elsif seen[1] != "--" then "session"
         elsif seen[0] != "--" then "next"
         else "this_request"
         end
  [kind, ["A1", *seen].join(" "), first]
end
```

The four classes work together. `Context` is new on every request and receives `params`, `flash` and `request_store` empty. `App` holds the `store` shared by all requests and the session chosen by client. `Flash` leaves the written value in the session at the end of the request; the next request takes it and deletes it. `stay` runs the rule: in request 1 client A writes the value and reads it, in requests 2 and 3 A only reads, and in request 4 client B only reads. If B sees it, the value is persistent; if A's third request sees it, it is session; if only the second request sees it, it is next request; if none sees it, it is this request. **FS1.** The persistent store here is in memory and goes away when the process ends. In a real application this place is a database, a cache or a queue; the rule is meaningful only within a single process.

#### Thirteen Observations

The second file, `values.rb`, carries the framework's small models and the thirteen observations. Each observation consists of a `write` and a `read` lambda. At the end of the file it gives all of them to `stay` and prints the table.

```ruby
# values.rb — thirteen observations: how many requests does this value stay for
require_relative "stay"

# Routing: a resource-based path pattern is turned into a controller, an action and an id.
ROUTES = { %r{\A/products/(?<id>\d+)\z} => "products#show" }

def route(path)
  ROUTES.each do |pattern, target|
    match = pattern.match(path) or next
    controller, action = target.split("#")
    return { controller:, action:, id: match[:id].to_i }
  end
  nil
end

# Convention: the template name is not written, it is derived from the controller and action names.
def template_name(params) = "#{params[:controller]}/#{params[:action]}"

# Object-relational mapper: a row maps to a record, a table maps to a model.
module Product
  def self.insert(b, attributes)
    errors = []
    errors << "name cannot be blank" if attributes[:name].to_s.empty?
    errors << "price cannot be negative" if attributes[:price].to_i.negative?
    return { errors: } unless errors.empty?

    table = b.app.store[:products]
    id = table.size + 1
    table[id] = attributes.merge(id:)
    { id:, errors: }
  end

  # The same query does not go to the database twice in the same request.
  def self.lookup(b, id)
    queries = (b.request_store[:queries] ||= {})
    queries.fetch([:products, id]) do
      b.request_store[:query_count] = b.request_store.fetch(:query_count, 0) + 1
      queries[[:products, id]] = b.app.store[:products][id]
    end
  end
end

def migrate(b, table, column) = b.app.store[:schema][table] << column

# Controller: a new instance on every request; a filter runs before the action.
class ProductsController
  def self.before(*names) = (@filters ||= []).concat(names)
  def self.filters = @filters || []

  before :load_product

  def initialize(context)
    @context = context
  end

  def run(action)
    self.class.filters.each { |filter| send(filter) }
    public_send(action)
  end

  def show = @product

  private

  def load_product
    @product = Product.lookup(@context, @context.params[:id])
  end
end

def controller(b) = b.request_store[:controller] ||= ProductsController.new(b)

# View: a template is compiled once; local values are escaped on every render.
def escape(text) = text.to_s.gsub("&", "&amp;").gsub("<", "&lt;").gsub(">", "&gt;").gsub('"', "&quot;")

def compile_template(b, name, source)
  b.app.store[:templates][name] ||= lambda do |locals|
    source.gsub(/\{\{(\w+)\}\}/) { escape(locals.fetch(Regexp.last_match(1).to_sym)) }
  end
end

def enqueue(b, job_name, *args) = b.app.store[:queue] << [job_name, args]

def cached(b, key) = b.app.store[:cache][key] ||= yield

def csrf_token(b) = b.session[:csrf] ||= "token-#{b.app.store[:counter] += 1}"

pen = ->(b) { Product.insert(b, name: "pen", price: 12) }
observations = {
  "template name"  => [->(b) { b.params.merge!(route("/products/7")) },
                       ->(b) { b.params[:action] && template_name(b.params) }],
  "path parameter" => [->(b) { b.params.merge!(route("/products/7")) },
                       ->(b) { b.params[:id] }],
  "flash"          => [->(b) { b.flash[:notice] = "product saved" },
                       ->(b) { b.flash[:notice] }],
  "compiled template" => [->(b) { compile_template(b, "products/show", "<h1>{{name}}</h1>") },
                          ->(b) { b.app.store[:templates]["products/show"]&.call(name: "<b>pen</b>") }],
  "record"         => [pen, ->(b) { b.app.store[:products][1]&.fetch(:name) }],
  "schema column"  => [->(b) { migrate(b, :products, :stock) },
                       ->(b) { b.app.store[:schema][:products].find { |c| c == :stock } }],
  "validation error" => [->(b) { b.request_store[:record] = Product.insert(b, name: "", price: 5) },
                         ->(b) { b.request_store.dig(:record, :errors, 0) }],
  "query cache"    => [->(b) { pen.call(b); Product.lookup(b, 1) },
                       ->(b) { b.request_store.dig(:queries, [:products, 1], :name) }],
  "user id"        => [->(b) { b.session[:user_id] = 42 },
                       ->(b) { b.session[:user_id] }],
  "queued job"     => [->(b) { enqueue(b, "send_invoice", 1) },
                       ->(b) { b.app.store[:queue].first }],
  "fragment cache" => [->(b) { cached(b, "product/1") { "<li>pen</li>" } },
                       ->(b) { b.app.store[:cache]["product/1"] }],
  "controller variable" => [->(b) { pen.call(b); b.params[:id] = 1; controller(b).run(:show) },
                            ->(b) { controller(b).instance_variable_get(:@product)&.fetch(:name) }],
  "csrf token"     => [->(b) { csrf_token(b) }, ->(b) { b.session[:csrf] }],
}

tally = { "this_request" => 0, "next" => 0, "session" => 0, "persistent" => 0 }

printf("%-22s%-14s%-13s%s\n", "observation", "stay", "evidence", "value")
observations.each do |name, (write, read)|
  kind, evidence, value = stay(write, read)
  tally[kind] += 1
  printf("%-22s%-14s%-13s%s\n", name, kind, evidence, value.inspect)
end

puts
tally.each { |kind, n| printf("%-14s%d\n", kind, n) }
puts "total observations: #{tally.values.sum}"
```
```
observation           stay          evidence     value
template name         this_request  A1 -- -- --  "products/show"
path parameter        this_request  A1 -- -- --  7
flash                 next          A1 A2 -- --  "product saved"
compiled template     persistent    A1 A2 A3 B4  "<h1>&lt;b&gt;pen&lt;/b&gt;</h1>"
record                persistent    A1 A2 A3 B4  "pen"
schema column         persistent    A1 A2 A3 B4  :stock
validation error      this_request  A1 -- -- --  "name cannot be blank"
query cache           this_request  A1 -- -- --  "pen"
user id               session       A1 A2 A3 --  42
queued job            persistent    A1 A2 A3 B4  ["send_invoice", [1]]
fragment cache        persistent    A1 A2 A3 B4  "<li>pen</li>"
controller variable   this_request  A1 -- -- --  "pen"
csrf token            session       A1 A2 A3 --  "token-1"

this_request  5
next          1
session       2
persistent    5
total observations: 13
```

Of the thirteen observations, 5 stayed for this request, 1 for the next request and 2 for the session; 5 are persistent. The evidence column is a record of four requests: the request that sees the value is written by name, and one that does not is written `--`. Everything that stays for this request is born in the request's context: the template name, the path parameter, the validation error, the query cache and the controller's instance variable. Because the context is renewed on every request, none of them reaches A2. The persistent ones are in `App#store`: the record, the schema column, the compiled template, the queued job and the fragment cache. In reality these are five separate places, but the rule does not separate them, because their behavior is the same: another client reads them too. **FS2.** The rule reads how the value behaves, not where it is stored. Landing in the same class does not say they are the same thing; in the coming lessons both faces of this sentence are measured.

This lesson examines the first row of the table.

#### The Template Name Is Not Written

The `template name` row fell into the this-request class and its value was `"products/show"`. This name was written nowhere. `route("/products/7")` gave the controller and action names, and `template_name` turned the two into a path. This is a **convention**: the template is expected to be the combination of the controller name and the action name, and there is no configuration line that says so. This is the principle of convention over configuration: the answer to the common case is the default, and only the deviation is written.

Calling the same helper with different names shows that the derivation works. `values.rb` prints the table as it runs, so this measurement runs in a separate file with its own definition; the `template_name` line is an exact copy of the one in `values.rb`.

```ruby
# convention.rb — deriving the template name
def template_name(params) = "#{params[:controller]}/#{params[:action]}"

p template_name(controller: "products", action: "show")
p template_name(controller: "products", action: "edit")
p template_name(controller: "orders", action: "show")
```
```
"products/show"
"products/edit"
"orders/show"
```

When the action name changed, the template name changed with it. The third line shows the limit of the convention: there is no controller named `orders` in the core, but `template_name` produced a name. A convention produces a name; it does not check whether such a template exists. **FS3.** This was measured only for the core's one-line `template_name`. How a real framework's file lookup behaves, and what it does when a template is missing, was not tested in this lesson.

This question is separate from the question of M08/K06. There the cause of a behavior in the source was measured: a call, an annotation, or a default. In that language the template name is a default. Here the cause is not asked, the stay is asked: even if the name derives from a default, its value is read only in this request.

#### What the Convention Produces from Missing Input

A convention is a rule, and a rule does not validate input. Looking at what the single line that produces the name gives for a missing parameter shows where the convention ends.

```ruby
# missing.rb — the name produced from a missing parameter
require_relative "stay"

# Convention: the template name is not written, it is derived from the controller and action names.
def template_name(params) = "#{params[:controller]}/#{params[:action]}"

p template_name(controller: "products", action: "show")
p template_name(controller: "products")
p template_name({})
p template_name(controller: "products", action: "")
```
```
"products/show"
"products/"
"/"
"products/"
```

When the action was missing no error was raised; the name became `products/`. When no parameter was given at all, the name became only `/`. An empty string and a missing key gave the same result, because `nil` also enters a string as an empty string. The convention builds the name; it does not check whether the name is meaningful. For this reason a missing name does not surface as an error but as a key that cannot be found in the compiled template store: the lookup silently returns `nil`. Code that wants to catch the error must itself check that the controller and action are filled in before producing the name; the convention does not take on this check.

#### The Bounding Measurement

A convention removes configuration; it does not store the value. There are two ways to measure this: whether the same name is absent in a next request, and what happens in a request where routing never ran.

```ruby
# limit.rb — the stay of the derived name
require_relative "stay"

def template_name(params) = "#{params[:controller]}/#{params[:action]}"

write = ->(b) { b.params.merge!(controller: "products", action: "show") }
read = ->(b) { b.params[:action] && template_name(b.params) }
p stay(write, read)

app = App.new
p app.request("A") { |b| read.call(b) }
p app.request("A") { |b| b.params.merge!(controller: "products", action: "show"); read.call(b) }
p app.store[:templates]
```
```
["this_request", "A1 -- -- --", "products/show"]
nil
"products/show"
{}
```

The first line is the same as the result in the table: the writing request sees the name, and the next three requests do not. The second line shows a request where routing did not run. Because `params[:action]` was empty, `read` gave `nil`; that is, the name was not computed, and there was no source from which it could have been computed. In the third line routing was written again into the same request and the name came out the same. The last line says the name was left nowhere: the application's template store is still empty. The name is recomputed from the parameters on every request. **FS4.** This measurement was made with a single pair of parameters; the stay of producing a name must be measured separately for a different convention.

#### Summary

- The **stay** of a value is how many requests it can be read for: this request, next request, session, or persistent.
- In the core, of the thirteen observations 5 stayed for this request, 1 for the next request and 2 for the session; 5 are persistent.
- The stay rule reads how the value behaves, not where it is stored.
- The template name is not written, it derives from the controller and action names; a convention produces a name but does not check that the template exists.
- The stay of this lesson is **this request**: the derived name is recomputed on every request and is not left in the store.

#### Next Step

The template name derived from the controller and action names that routing gave. So where did routing get those names? The next lesson measures the pattern that splits a path into its parts, and looks at whether the number `7` in the path passes to a next request.

### Routing

Source: https://academia.sh/en/courses/mvc-framework/routing

Routing turns a path into a controller, an action and an identifier. The lesson measures that a path parameter becomes an integer, that a path that does not match the pattern is rejected, that the pattern order changes the result, and that the value stays only for this request.

The previous lesson showed that the template name derives from the controller and action names, but a routing step had supplied those names themselves. **Routing** turns the incoming path into the action of a particular controller. The question of this lesson is narrower: when the path is `/products/7`, where does the number `7` go, and where does it stand after that request ends?

The stay axis continues from the previous lesson: **how many requests does this value stay for?** The observation of this lesson is the `path parameter` row. The core's `stay.rb` file is given again below; the block is source, it has no output, and it is pulled in with `require_relative`.

```ruby
# stay.rb — a small request loop and a helper that classifies how many requests a value stays for

# Flash: a value written in this request is carried to the next request only.
class Flash
  def initialize(incoming)
    @incoming = incoming
    @fresh = {}
  end

  def [](key) = @fresh.fetch(key) { @incoming[key] }

  def []=(key, value)
    @fresh[key] = value
  end

  def passed_on = @fresh
end

# Every request gets a new context: params, flash and the request store start empty each time.
class Context
  attr_reader :app, :params, :session, :flash, :request_store

  def initialize(app, session)
    @app = app
    @params = {}
    @session = session
    @flash = Flash.new(session.delete(:flash) || {})
    @request_store = {}
  end
end

# The application store is the same across all requests and all clients;
# the session is chosen per client.
class App
  attr_reader :store

  def initialize
    @sessions = Hash.new { |h, client| h[client] = {} }
    @store = { products: {}, schema: { products: %i[name price] }, templates: {},
               queue: [], cache: {}, counter: 0 }
  end

  def request(client)
    context = Context.new(self, @sessions[client])
    result = yield context
    context.session[:flash] = context.flash.passed_on unless context.flash.passed_on.empty?
    result
  end
end

# Rule: request 1 (client A) writes the value; requests 2 and 3 (A) and request 4 (B) read it.
# If B sees it, persistent; if A's third request sees it, session; if only the second
# request sees it, next; if none sees it, this_request.
def stay(write, read)
  app = App.new
  first = app.request("A") { |b| write.call(b); read.call(b) }
  raise ArgumentError, "value could not be read in the request that wrote it" if first.nil?

  seen = [["A", 2], ["A", 3], ["B", 4]].map do |client, order|
    app.request(client) { |b| read.call(b) }.nil? ? "--" : "#{client}#{order}"
  end
  kind = if seen[2] != "--" then "persistent"
         elsif seen[1] != "--" then "session"
         elsif seen[0] != "--" then "next"
         else "this_request"
         end
  [kind, ["A1", *seen].join(" "), first]
end
```

#### Pattern, Action and Identifier

The core's routing carries a single pattern. The pattern is a **resource-based route**: it starts with `/products/`, an identifier follows, and the target is written as `products#show`. The target is split into two parts: before `#` is the controller, after it is the action. The identifier is a named part of the pattern (`id`) and is read from the match.

The file below takes only the routing definitions from `values.rb`, exactly as they are, and calls `stay` with its own two lambdas.

```ruby
# routing.rb — the stay of a path parameter
require_relative "stay"

# Routing: a resource-based path pattern is turned into a controller, an action and an id.
ROUTES = { %r{\A/products/(?<id>\d+)\z} => "products#show" }

def route(path)
  ROUTES.each do |pattern, target|
    match = pattern.match(path) or next
    controller, action = target.split("#")
    return { controller:, action:, id: match[:id].to_i }
  end
  nil
end

write = ->(b) { b.params.merge!(route("/products/7")) }
read = ->(b) { b.params[:id] }
p stay(write, read)
p route("/products/7")
p route("/products/7")[:id].class
```
```
["this_request", "A1 -- -- --", 7]
{controller: "products", action: "show", id: 7}
Integer
```

The first line is the same as the result in the table: the path parameter was read only in `A1` and not in `A2`. The second line shows the hash that routing produced: controller, action and identifier. The third line says the identifier is an `Integer`. The `7` in the path is a string; `route` converted it with `to_i`. So the controller receives the identifier as a number and does not do the comparison on a string. **FS5.** The conversion is limited to the digits that the `\d+` pattern matches; other spellings of a number format did not enter this measurement.

#### A Path That Does Not Match

Routing's second job is to reject a path. The core's pattern requires `\d+` and ends with `\z`. Three paths test these two rules.

```ruby
# reject.rb — paths that do not match the pattern
require_relative "stay"

# Routing: a resource-based path pattern is turned into a controller, an action and an id.
ROUTES = { %r{\A/products/(?<id>\d+)\z} => "products#show" }

def route(path)
  ROUTES.each do |pattern, target|
    match = pattern.match(path) or next
    controller, action = target.split("#")
    return { controller:, action:, id: match[:id].to_i }
  end
  nil
end

p route("/products/abc")
p route("/products/7/")
p route("/products/7?order=2")
p route("/products/7")
```
```
nil
nil
nil
{controller: "products", action: "show", id: 7}
```

`/products/abc` did not match because it is not digits, `/products/7/` because it has an extra slash at the end, and `/products/7?order=2` because it carries a **query string** at the end. The third is a limit of the core: the path and the query string must be separated before routing, because the pattern wants the whole path. In all three `route` gave `nil`; that is, the core reports a rejection not as an exception but as the absence of an answer. The `\z` anchor ties the pattern to the end of the path; without it `7/` would match too. How a `nil` is interpreted is the business of whoever calls it, not of routing. **FS6.** The form of the rejection is specific to this core; another routing step may give a different answer in the same situation, and this lesson did not measure it.

#### Pattern Order

When there is a single pattern, the order question does not arise. When the table grows, the first matching pattern wins; the core's `route` also walks the table from start to end and returns at the first match. To measure this behavior with two patterns we build our own small table. There is a new helper here called `first_match`; it has no counterpart in the core.

```ruby
# order.rb — order in two patterns
def first_match(table, path)
  table.each do |pattern, target|
    match = pattern.match(path) or next
    return [target, match[:id]]
  end
  nil
end

broad = { %r{\A/products/(?<id>[^/]+)\z} => "products#show_by_name",
          %r{\A/products/(?<id>\d+)\z} => "products#show" }
narrow = broad.to_a.reverse.to_h

p first_match(broad, "/products/7")
p first_match(narrow, "/products/7")
p first_match(narrow, "/products/abc")
```
```
["products#show_by_name", "7"]
["products#show", "7"]
["products#show_by_name", "abc"]
```

The two tables carry the same two patterns, only their order differs. When the broad pattern was first, `/products/7` went to the name route, not the number route. When the narrow pattern was first, the same path went to the number route, and the non-numeric `/products/abc` fell to the broad pattern. **FS7.** This measurement is only for two patterns and three paths; with more patterns the overlap behavior must be tested separately. In a pattern table the order is a configuration, but it is not silent like a convention: when the position changes, the result changes.

#### A Nested Path

A resource can have another resource under it: the comments of a product, for example. The path carries two identifiers and the pattern needs two named parts. The pattern below is this lesson's own definition and is not added to the core table; the first question is whether the core's `route` recognizes this path.

```ruby
# nested.rb — a path with two identifiers
require_relative "stay"

# Routing: a resource-based path pattern is turned into a controller, an action and an id.
ROUTES = { %r{\A/products/(?<id>\d+)\z} => "products#show" }

def route(path)
  ROUTES.each do |pattern, target|
    match = pattern.match(path) or next
    controller, action = target.split("#")
    return { controller:, action:, id: match[:id].to_i }
  end
  nil
end
NESTED = %r{\A/products/(?<id>\d+)/comments/(?<comment_id>\d+)\z}

path = "/products/7/comments/3"
p route(path)
match = NESTED.match(path)
p match.names
p match.named_captures.transform_values(&:to_i)
```
```
nil
["id", "comment_id"]
{"id" => 7, "comment_id" => 3}
```

The core's single pattern rejected the path, because `\z` wants the end right after the identifier. The nested pattern gave the two parts by name. Named parts are read by name, not by order; so whether `comment_id` comes after or before `id` does not change the result. The two numbers in the path are also strings, and they were converted to integers with `to_i`. **FS8.** This pattern stays only in this lesson's file; if it were added to the core's `ROUTES` table, the order question would arise again.

#### Building the Path Back

Routing goes from path to target. Another place in the application wants to go from target to path: when writing a link, to give the target and the identifier instead of writing `/products/7` by hand. This direction also depends on the definition of routing. If the two directions are written separately, they can drift apart.

```ruby
# reverse.rb — building a path from a target
require_relative "stay"

# Routing: a resource-based path pattern is turned into a controller, an action and an id.
ROUTES = { %r{\A/products/(?<id>\d+)\z} => "products#show" }

def route(path)
  ROUTES.each do |pattern, target|
    match = pattern.match(path) or next
    controller, action = target.split("#")
    return { controller:, action:, id: match[:id].to_i }
  end
  nil
end
def build_path(target, id) = "/#{target.split('#').first}/#{id}"

path = build_path("products#show", 7)
p path
p route(path)
p route(build_path("products#show", "abc"))
```
```
"/products/7"
{controller: "products", action: "show", id: 7}
nil
```

The built path was decoded back with the core's pattern and gave the same identifier: the round trip is consistent. But `build_path` did not look at what the identifier is. When given `"abc"`, it built a path that the pattern would reject, and routing turned it into `nil`. The building does not know the pattern, the pattern does not know the building; tying the two to the same definition is the job of the routing layer. **FS9.** This measurement is for a single pattern and a single target; in a table with many patterns, which pattern the reverse building will choose must be defined separately.

#### The Limits of the Identifier

The pattern wants only digits, but says nothing about how many digits or what value. Passing extreme values through the path and looking at what the identifier is shows what the `Integer` conversion guarantees.

```ruby
# identifier.rb — extreme values arriving as the identifier
require_relative "stay"

# Routing: a resource-based path pattern is turned into a controller, an action and an id.
ROUTES = { %r{\A/products/(?<id>\d+)\z} => "products#show" }

def route(path)
  ROUTES.each do |pattern, target|
    match = pattern.match(path) or next
    controller, action = target.split("#")
    return { controller:, action:, id: match[:id].to_i }
  end
  nil
end

["/products/007", "/products/0", "/products/99999999999999999999"].each do |path|
  result = route(path)
  p [path, result && result[:id], result && result[:id].class]
end
```
```
["/products/007", 7, Integer]
["/products/0", 0, Integer]
["/products/99999999999999999999", 99999999999999999999, Integer]
```

The leading zeros were dropped: the path `007` gave the identifier `7`, so two different paths go to the same resource. `0` was accepted as a valid identifier; the pattern does not reject zero, and whether a record exists with that identifier is learned only afterward. A very long digit sequence was not rejected either and still became an `Integer`, because in Ruby an integer has no fixed upper limit. **FS10.** Routing validates the form of the identifier, not its existence; several paths going to the same resource is a consequence that must be considered separately for addressing, caching and link comparison. For this reason code that compares the identifier of a link should compare the integer that routing gave, not the path; even if the two path texts differ, the resolved identifier is the same.

#### Generating Routes from a Resource

The name of the resource-based route comes from a set of routes being derived from a resource name. The core carries only the `show` pattern, but the same idea can take a name and produce several patterns. The two helpers below are this lesson's own plain Ruby code and have no counterpart in the core. The pattern is looked up together with the **HTTP method** (GET, POST, DELETE); the same path can go to a different action with a different method. The method here is not a Ruby method but the kind of the request; it is named `request_method`.

```ruby
# resource.rb — generating routes from a resource name
def resource_routes(name)
  {
    ["GET", %r{\A/#{name}\z}] => "#{name}#list",
    ["GET", %r{\A/#{name}/(?<id>\d+)\z}] => "#{name}#show",
    ["POST", %r{\A/#{name}\z}] => "#{name}#add",
    ["DELETE", %r{\A/#{name}/(?<id>\d+)\z}] => "#{name}#remove",
  }
end

def find_route(table, request_method, path)
  table.each do |(verb, pattern), target|
    next unless verb == request_method
    match = pattern.match(path) or next
    return [target, match.names.empty? ? nil : match[:id].to_i]
  end
  nil
end

table = resource_routes("products").merge(resource_routes("orders"))
p table.size
p find_route(table, "GET", "/products")
p find_route(table, "GET", "/orders/3")
p find_route(table, "POST", "/products")
p find_route(table, "DELETE", "/products/7")
p find_route(table, "POST", "/products/7")
```
```
8
["products#list", nil]
["orders#show", 3]
["products#add", nil]
["products#remove", 7]
nil
```

Two resource names gave eight routes: each name produced four patterns. `/products` went to `list` with GET and to `add` with POST; the same path, a different action. `/orders/3` went to the second resource's `show` and the identifier was `3`. In the last line there is the path `/products/7`, but there is no pattern defined with POST; the result is `nil`. If the path is recognized and the method is not, it is rejected too. Producing this set from a name instead of writing it by hand is the convention in routing: the resource name is given and the patterns derive. **FS11.** These four patterns are a choice; how many actions a resource will have does not follow from this measurement.

#### The Bounding Measurement

A path parameter is carried in the URL, but its value does not pass to the next request. The core's result already said this: `A2` is `--` in the evidence column. The measurement here separates how the same value reaches two separate requests.

```ruby
# limit.rb — the parameter comes from the URL, it does not pass from request to request
require_relative "stay"

# Routing: a resource-based path pattern is turned into a controller, an action and an id.
ROUTES = { %r{\A/products/(?<id>\d+)\z} => "products#show" }

def route(path)
  ROUTES.each do |pattern, target|
    match = pattern.match(path) or next
    controller, action = target.split("#")
    return { controller:, action:, id: match[:id].to_i }
  end
  nil
end

app = App.new
p app.request("A") { |b| b.params.merge!(route("/products/7")); b.params[:id] }
p app.request("A") { |b| b.params[:id] }
p app.request("A") { |b| b.params.merge!(route("/products/7")); b.params[:id] }

same_path = ->(b) { b.params.merge!(route("/products/7")); b.params[:id] }
p stay(same_path, same_path)
```
```
7
nil
7
["persistent", "A1 A2 A3 B4", 7]
```

The first request read `7`. The second request did not ask for the path again and `params[:id]` came empty: `nil`. The third request gave the path again and received `7` again. The last line shows a trap in this. When the `read` lambda reran routing itself, every request and every client saw the value; the rule counts this as `persistent`. The value was not stored, it was derived again from the path each time. The rule reads the behavior, so the same mechanism falls into a different stay depending on how it is used. Carrying the value to the next request is not the job of routing but of the client: a link writes the identifier into the path again; a form carries it with the query string or the request body. **FS12.** This measurement was made with the in-memory core; whether a real client carries the identifier by another route (a header, a cookie) was not tested.

#### Summary

- Routing turns a path into a controller, an action and an identifier; the named part in the pattern gives the identifier, and the identifier becomes an `Integer`.
- A path that does not match the pattern is rejected: `/products/abc` and `/products/7/`, which carries a trailing slash, give `nil` in the core.
- In a pattern table the first match wins; when the order of two patterns changes, the same path goes to a different action.
- The stay of this lesson is **this request**: a path parameter is read only in the request that carries the path.
- Carrying the value to the next request is the client's job; giving the path again each time can make the value look persistent, but it does not store it.

#### Next Step

Routing gave the name of the action, but the class that runs the action is not yet on stage. The next lesson measures the controller and the filter that runs before the action, and looks at why a notification message appears once after saving and redirecting to another page.

### Controllers and Filters

Source: https://academia.sh/en/courses/mvc-framework/controllers-and-filters

The controller is built anew on every request and the filter runs before the action. The lesson measures that the filter list is bound to the class, that the controller instance is bound to the request, and that the flash message is carried only to the next request.

The previous lesson showed that routing gives the `products#show` target and that the `7` in the path is read only in that request. The class that answers the target is the **controller**. The question of this lesson has two parts: does the controller object live from one request to another, and what does a **filter** that runs at the start of an action remember? A third question is tied to both: after a product is saved and the request redirects to another page, why does the flash message that is shown appear once?

M10/K02 asked in which scope the language ends the lifetime of a value; that was the compiler's decision. Here the lifetime is decided not by the language but by the request cycle: `@product` lives as long as the object does in Ruby, but because the controller is rebuilt on every request, the value stays for this request.

The core's `stay.rb` file is given below once more. The block is source, it has no output, and it is pulled in with `require_relative`.

```ruby
# stay.rb — a small request loop and a helper that classifies how many requests a value stays for

# Flash: a value written in this request is carried to the next request only.
class Flash
  def initialize(incoming)
    @incoming = incoming
    @fresh = {}
  end

  def [](key) = @fresh.fetch(key) { @incoming[key] }

  def []=(key, value)
    @fresh[key] = value
  end

  def passed_on = @fresh
end

# Every request gets a new context: params, flash and the request store start empty each time.
class Context
  attr_reader :app, :params, :session, :flash, :request_store

  def initialize(app, session)
    @app = app
    @params = {}
    @session = session
    @flash = Flash.new(session.delete(:flash) || {})
    @request_store = {}
  end
end

# The application store is the same across all requests and all clients;
# the session is chosen per client.
class App
  attr_reader :store

  def initialize
    @sessions = Hash.new { |h, client| h[client] = {} }
    @store = { products: {}, schema: { products: %i[name price] }, templates: {},
               queue: [], cache: {}, counter: 0 }
  end

  def request(client)
    context = Context.new(self, @sessions[client])
    result = yield context
    context.session[:flash] = context.flash.passed_on unless context.flash.passed_on.empty?
    result
  end
end

# Rule: request 1 (client A) writes the value; requests 2 and 3 (A) and request 4 (B) read it.
# If B sees it, persistent; if A's third request sees it, session; if only the second
# request sees it, next; if none sees it, this_request.
def stay(write, read)
  app = App.new
  first = app.request("A") { |b| write.call(b); read.call(b) }
  raise ArgumentError, "value could not be read in the request that wrote it" if first.nil?

  seen = [["A", 2], ["A", 3], ["B", 4]].map do |client, order|
    app.request(client) { |b| read.call(b) }.nil? ? "--" : "#{client}#{order}"
  end
  kind = if seen[2] != "--" then "persistent"
         elsif seen[1] != "--" then "session"
         elsif seen[0] != "--" then "next"
         else "this_request"
         end
  [kind, ["A1", *seen].join(" "), first]
end
```

#### The Filter and a Class-Bound List

The core's controller declares the filter with the line `before :load_product`. This is a call, and it adds a name to a list of the class; before calling the action, `run` runs every name in that list in order. The block below first looks at the list itself, then at an empty class derived from the same class, and then runs the action when there is no product and when there is one.

```ruby
# filter.rb — the filter list and the product the filter found
require_relative "stay"

# Object-relational mapper: a row maps to a record, a table maps to a model.
module Product
  def self.insert(b, attributes)
    errors = []
    errors << "name cannot be blank" if attributes[:name].to_s.empty?
    errors << "price cannot be negative" if attributes[:price].to_i.negative?
    return { errors: } unless errors.empty?

    table = b.app.store[:products]
    id = table.size + 1
    table[id] = attributes.merge(id:)
    { id:, errors: }
  end

  # The same query does not go to the database twice in the same request.
  def self.lookup(b, id)
    queries = (b.request_store[:queries] ||= {})
    queries.fetch([:products, id]) do
      b.request_store[:query_count] = b.request_store.fetch(:query_count, 0) + 1
      queries[[:products, id]] = b.app.store[:products][id]
    end
  end
end

# Controller: a new instance on every request; a filter runs before the action.
class ProductsController
  def self.before(*names) = (@filters ||= []).concat(names)
  def self.filters = @filters || []

  before :load_product

  def initialize(context)
    @context = context
  end

  def run(action)
    self.class.filters.each { |filter| send(filter) }
    public_send(action)
  end

  def show = @product

  private

  def load_product
    @product = Product.lookup(@context, @context.params[:id])
  end
end

pen = ->(b) { Product.insert(b, name: "pen", price: 12) }

p ProductsController.filters

class Sub < ProductsController; end
p Sub.filters

app = App.new
p app.request("A") { |b| b.params[:id] = 99; ProductsController.new(b).run(:show) }
p app.request("A") { |b| pen.call(b); b.params[:id] = 1; ProductsController.new(b).run(:show) }
```
```
[:load_product]
[]
nil
{name: "pen", price: 12, id: 1}
```

The first line is the list, carrying the name `load_product`. The second line is surprising: although `Sub` derives from the same class, its list is empty. The filter list lives in the class's own instance variable and does not pass to a subclass; a subclass must declare these filters itself. **FS13.** This measurement is for a single declaration form; a framework that keeps the filter list in a class-level constant may behave differently toward a subclass. The third line shows that when there is no product with the identifier `99`, the action gave `nil`: the filter could not find the product, the action returned an empty instance variable, and no error was raised. In the fourth line the product was inserted first and the filter found it. The action never touched the identifier; it is the filter that prepares the product. This division of labor is the actual reason for the filter: the line that finds the product is not repeated in every action, it stands in one place, and the action only uses the prepared value. The price is a dependency; code that calls the action without the filter running notices that `show` gave `nil` only when it looks at the result. Because the previous lesson's routing wrote the identifier into `params`, the filter can rely on it, but if routing did not run, the filter looks up with a `nil` identifier and finds no product.

The core's model in this lesson makes a convenience, and it should be said once: `Product.lookup` gives the stored object itself. Many object–relational mappers build a new object from the table on every read. This difference does not change the measured stay, because the stay looks at how many requests the value can be read for, not at whether the object is the same.

#### The Controller Is New on Every Request

In the previous courses we saw that an object holds its values as long as it lives: in Ruby an object lives as long as code that can reach it remains. On the web this needs an addition. The controller object may live, but because the framework rebuilds it on every request, an instance variable stays in that request. The block below builds the controller in two consecutive requests and, in the second, looks at whether it is the same as the first one's instance.

```ruby
# new_instance.rb — the controller instance is not carried from request to request
require_relative "stay"

# Object-relational mapper: a row maps to a record, a table maps to a model.
module Product
  def self.insert(b, attributes)
    errors = []
    errors << "name cannot be blank" if attributes[:name].to_s.empty?
    errors << "price cannot be negative" if attributes[:price].to_i.negative?
    return { errors: } unless errors.empty?

    table = b.app.store[:products]
    id = table.size + 1
    table[id] = attributes.merge(id:)
    { id:, errors: }
  end

  # The same query does not go to the database twice in the same request.
  def self.lookup(b, id)
    queries = (b.request_store[:queries] ||= {})
    queries.fetch([:products, id]) do
      b.request_store[:query_count] = b.request_store.fetch(:query_count, 0) + 1
      queries[[:products, id]] = b.app.store[:products][id]
    end
  end
end

# Controller: a new instance on every request; a filter runs before the action.
class ProductsController
  def self.before(*names) = (@filters ||= []).concat(names)
  def self.filters = @filters || []

  before :load_product

  def initialize(context)
    @context = context
  end

  def run(action)
    self.class.filters.each { |filter| send(filter) }
    public_send(action)
  end

  def show = @product

  private

  def load_product
    @product = Product.lookup(@context, @context.params[:id])
  end
end

pen = ->(b) { Product.insert(b, name: "pen", price: 12) }

app = App.new
app.request("A") { |b| pen.call(b) }

first_instance = nil
result = app.request("A") do |b|
  b.params[:id] = 1
  first_instance = ProductsController.new(b)
  first_instance.run(:show)
end
p result
result = app.request("A") do |b|
  second_instance = ProductsController.new(b)
  [second_instance.equal?(first_instance), second_instance.show]
end
p result
```
```
{name: "pen", price: 12, id: 1}
[false, nil]
```

In the first request the filter found the product and placed it in `@product`, and the action gave it. The second request built a new instance: the `equal?` answer is `false`, and `show` gave `nil`, because the filter did not run in this instance and `@product` was never assigned. The instance from the first request was still in memory, but the second request did not reach it. For this reason a controller's instance variables are in the **this request** stay. If a value must be carried to the next request, it must be written somewhere outside the controller.

**FS14.** The measurement rests on the assumption that the core builds a new `Context` and a new instance on every request. A server that reuses the instance may run the same class with a different stay.

#### Filter Order and an Interrupted Chain

When more than one filter is declared, the order is the declaration order. What happens if a filter raises an error also follows from the structure of the core's `run`. The class below is this lesson's own example and is not added to the core's `ProductsController`; it rebuilds the same two-line mechanism in a class that keeps an event log.

```ruby
# order.rb — filter order and a filter that raises an error
class Trial
  def self.before(*names) = (@filters ||= []).concat(names)
  def self.filters = @filters || []

  before :first_step, :second_step

  attr_reader :log

  def initialize = @log = []

  def run(action)
    self.class.filters.each { |filter| send(filter) }
    public_send(action)
  end

  def action = @log << :action

  private

  def first_step = @log << :first_step
  def second_step = raise(ArgumentError, "second filter stopped")
end

t = Trial.new
begin
  t.run(:action)
rescue ArgumentError => e
  p e.class
end
p t.log
```
```
ArgumentError
[:first_step]
```

The first filter ran, the second raised an error, and the action was never called: there is no `:action` in the log. The chain is cut at the point where the error arose. In this core it is a design choice, because `each` stops when an error is raised and the `public_send` line is never reached. In a real framework it is also possible for a filter to give the response itself and stop the chain; the core does not model this. **FS15.** The measurement rests on the assumption that the core runs in order with `each` and does not catch the error.

#### Save, Redirect, Show the Flash Message

In this lesson's example, the submission of a form is followed by three requests. In the first, the product is saved and the flash message is written. The action then redirects to another page. In the second request the new page reads the message and shows it. In the third request the page is refreshed and there is no message.

```ruby
# notice.rb — save, redirect, show the message
require_relative "stay"

# Object-relational mapper: a row maps to a record, a table maps to a model.
module Product
  def self.insert(b, attributes)
    errors = []
    errors << "name cannot be blank" if attributes[:name].to_s.empty?
    errors << "price cannot be negative" if attributes[:price].to_i.negative?
    return { errors: } unless errors.empty?

    table = b.app.store[:products]
    id = table.size + 1
    table[id] = attributes.merge(id:)
    { id:, errors: }
  end

  # The same query does not go to the database twice in the same request.
  def self.lookup(b, id)
    queries = (b.request_store[:queries] ||= {})
    queries.fetch([:products, id]) do
      b.request_store[:query_count] = b.request_store.fetch(:query_count, 0) + 1
      queries[[:products, id]] = b.app.store[:products][id]
    end
  end
end

app = App.new
p app.request("A") { |b| saved = Product.insert(b, name: "pen", price: 12); b.flash[:notice] = "product saved"; saved }
p app.request("A") { |b| b.flash[:notice] }
p app.request("A") { |b| b.flash[:notice] }
```
```
{id: 1, errors: []}
"product saved"
nil
```

The first request inserted the record and gave the identifier `1`; `errors` is empty. The second request read the message: `"product saved"`. The third request read the same key and got `nil`. The message was carried from the request it was written in to the next one and was consumed there. The core's `Context` class reads by deleting the `:flash` key in the session on every request; so a message can be read once. The record itself, though, stayed in the store in all three requests, because the product had been written to the application store.

#### A Flash Message That Is Not Read or Is Overwritten

Two more cases can be asked: if the message is not read in the next request, is it still there in the third request, and if the same key is written twice in the same request, which value is carried?

```ruby
# unread.rb — a flash message that is not read and one that is overwritten
require_relative "stay"

app = App.new
p app.request("A") { |b| b.flash[:notice] = "first"; b.flash[:notice] = "last"; b.flash[:notice] }
p app.request("A") { |b| b.params[:page] = 1 }
p app.request("A") { |b| b.flash[:notice] }
```
```
"last"
1
nil
```

In the first request there were two writes, and the read gave the last value. The second request did not read the message at all, it only did another job. The third request got `nil`. The life of a message depends not on its being read but on a request starting: every new `Context` takes the message in the session and deletes it. A message that is not read goes at the same moment from the place where it would have gone had it been read. **FS16.** This measurement is for three requests and a single key.

Measuring the same mechanism with the core's classifier names the stay of the message with one word.

```ruby
# controllers.rb — the stay of the flash message
require_relative "stay"

write = ->(b) { b.flash[:notice] = "product saved" }
read = ->(b) { b.flash[:notice] }
p stay(write, read)
```
```
["next", "A1 A2 -- --", "product saved"]
```

The sequence `A1 A2 -- --` says that the message was seen in the request it was written in and in the next one, and not seen in the third request or by another client. The stay is **next request**: it is read only in the same client's next request. This is the result of only one of the core's thirteen observations; none of the other observations fell into this stay.

#### The Bounding Measurement

That the flash message is carried to the next request is a result not of the mechanism but of how it is used. If the reading code rewrites the value every time it reads it, the same mechanism carries the value through the session.

```ruby
# limit.rb — the same mechanism can fall into another stay
require_relative "stay"

write = ->(b) { b.flash[:counter] = 1 }
read = ->(b) { v = b.flash[:counter]; b.flash[:counter] = v if v; v }
p stay(write, read)
```
```
["session", "A1 A2 A3 --", 1]
```

The message was this time classified as `["session", "A1 A2 A3 --", 1]`: it was read in the same client's third request too, and not read by another client. What was changed is not the mechanism but the behavior of the reading code. Every request that rewrites while reading passes the value to the next one; the chain breaks only when a request reads without writing. **FS17.** This measurement is only for a single key and three requests; how long the chain lasted on another server was not measured.

#### Summary

- The filter runs before the action and prepares the value the action will use; the filter list is bound to the class and does not pass to a subclass.
- The controller is built as a new instance on every request; an instance variable stays only in that request.
- In the save, redirect, show flow, the flash message is carried from the request it is written in to the next one and is consumed there.
- The stay of this lesson is **next request**: the message is read only in the next request.
- The same mechanism falls into another stay depending on how it is used: code that rewrites while reading turns the message into a session value.

#### Next Step

The controller prepared the product and carried the message, but did not produce the output. The next lesson measures the view and the template: how many times a template is compiled, how local values are escaped, and in which stay the compiled template stands. The flash message was carried to the next request in this lesson; the compiled form of a template will stand not in the request but in a place bound to the application, and will fall into another stay.

### Views and Templates

Source: https://academia.sh/en/courses/mvc-framework/views-and-templates

A template is compiled once and local values are escaped on every render. The lesson measures what escaping does, that a missing local value raises an error, that double escaping arises in a partial, and that the compiled template is persistent.

The previous lesson showed that the controller prepares the product and carries the flash message to the next request. The prepared value finally enters a **view**, and the view turns it into the text the user will see. The source of a view is a **template**: a string made of fixed text and places to fill, such as `{{name}}`. This lesson has two questions. What happens when a value enters a template: does `<b>pen</b>`, given as a local value, become a tag or text in the output? And where does the template itself stand after it is rendered, and for how many requests does it stay?

The core's `stay.rb` file is given below once more. The block is source, it has no output, and it is pulled in with `require_relative`.

```ruby
# stay.rb — a small request loop and a helper that classifies how many requests a value stays for

# Flash: a value written in this request is carried to the next request only.
class Flash
  def initialize(incoming)
    @incoming = incoming
    @fresh = {}
  end

  def [](key) = @fresh.fetch(key) { @incoming[key] }

  def []=(key, value)
    @fresh[key] = value
  end

  def passed_on = @fresh
end

# Every request gets a new context: params, flash and the request store start empty each time.
class Context
  attr_reader :app, :params, :session, :flash, :request_store

  def initialize(app, session)
    @app = app
    @params = {}
    @session = session
    @flash = Flash.new(session.delete(:flash) || {})
    @request_store = {}
  end
end

# The application store is the same across all requests and all clients;
# the session is chosen per client.
class App
  attr_reader :store

  def initialize
    @sessions = Hash.new { |h, client| h[client] = {} }
    @store = { products: {}, schema: { products: %i[name price] }, templates: {},
               queue: [], cache: {}, counter: 0 }
  end

  def request(client)
    context = Context.new(self, @sessions[client])
    result = yield context
    context.session[:flash] = context.flash.passed_on unless context.flash.passed_on.empty?
    result
  end
end

# Rule: request 1 (client A) writes the value; requests 2 and 3 (A) and request 4 (B) read it.
# If B sees it, persistent; if A's third request sees it, session; if only the second
# request sees it, next; if none sees it, this_request.
def stay(write, read)
  app = App.new
  first = app.request("A") { |b| write.call(b); read.call(b) }
  raise ArgumentError, "value could not be read in the request that wrote it" if first.nil?

  seen = [["A", 2], ["A", 3], ["B", 4]].map do |client, order|
    app.request(client) { |b| read.call(b) }.nil? ? "--" : "#{client}#{order}"
  end
  kind = if seen[2] != "--" then "persistent"
         elsif seen[1] != "--" then "session"
         elsif seen[0] != "--" then "next"
         else "this_request"
         end
  [kind, ["A1", *seen].join(" "), first]
end
```

#### Escaping

The core's view layer consists of two helpers. `escape` replaces four characters in a text with their counterparts. `compile_template` turns a template source into a lambda once; as the lambda is called, it passes the local values through `escape` and puts them in their places. **Escaping** makes the text that comes from a user appear on the page as text and not as markup.

```ruby
# escape.rb — escaping four characters and escaping twice
require_relative "stay"

# View: a template is compiled once; local values are escaped on every render.
def escape(text) = text.to_s.gsub("&", "&amp;").gsub("<", "&lt;").gsub(">", "&gt;").gsub('"', "&quot;")

def compile_template(b, name, source)
  b.app.store[:templates][name] ||= lambda do |locals|
    source.gsub(/\{\{(\w+)\}\}/) { escape(locals.fetch(Regexp.last_match(1).to_sym)) }
  end
end

p escape('<script>alert("x")</script>')
p escape("Tom & Jerry")
p escape(escape("<b>"))
p escape(nil)
p escape(12)
```
```
"&lt;script&gt;alert(&quot;x&quot;)&lt;/script&gt;"
"Tom &amp; Jerry"
"&amp;lt;b&amp;gt;"
""
"12"
```

The first line shows all four characters: `<` and `>` protect the tag boundaries, `"` protects the attribute boundary, and `&` protects the beginning of escaping itself. A text that could be read as a tag is now only text. In the second line `&` was escaped first and became `&amp;`. The third line is the result of double escaping: `<b>` became `&lt;b&gt;` in the first pass, the `&` characters were changed again in the second pass, and the text came to the form `&amp;lt;b&amp;gt;`. On the page this shows the user the text `&lt;b&gt;`. The last two lines show that `escape` first turns every input into a string: `nil` became an empty string and `12` became `"12"`.

That escaping happens in the template engine is a choice. If the controller or the model escaped the value and stored it, the same data would appear with the text `&lt;` in a console, in an export, or in another view. Escaping belongs to the format of the output, so it is done at render time and only where the value is rendered. The value itself is stored as it is, and each view escapes it according to its own format; transforming the data twice is the first cause of double escaping.

**FS18.** This measurement is only for four characters. The single quote is not escaped in the core; putting user text into an attribute value delimited by single quotes is not safe with this escaping.

#### A Missing Local Value

Every `{{name}}` in a template needs a local value. The core's lambda reads the local value with `fetch`. This makes a missing key raise an error instead of silently giving an empty string.

```ruby
# missing.rb — a place where no local value is given
require_relative "stay"

# View: a template is compiled once; local values are escaped on every render.
def escape(text) = text.to_s.gsub("&", "&amp;").gsub("<", "&lt;").gsub(">", "&gt;").gsub('"', "&quot;")

def compile_template(b, name, source)
  b.app.store[:templates][name] ||= lambda do |locals|
    source.gsub(/\{\{(\w+)\}\}/) { escape(locals.fetch(Regexp.last_match(1).to_sym)) }
  end
end

app = App.new
template = compile_template(Context.new(app, {}), "products/show", "<h1>{{name}}</h1>")

p template.call(name: "pen")
begin
  template.call({})
rescue KeyError => e
  p e.class
end
p template.call(name: "pen", price: 12)
```
```
"<h1>pen</h1>"
KeyError
"<h1>pen</h1>"
```

With the right local value the template gave `<h1>pen</h1>`. When the local value was not given, a `KeyError` came out; the error message was not printed because the format of the message depends on the version. The third call ran with an extra local value that has no counterpart in the template and raised no error: the template looks only for its own places and ignores the excess. That a missing value is an error and an extra value is not is a design balance. Whoever writes a template wants the local values to arrive complete; a controller, however, may pass more values to a template than needed, and this is not counted as an error.

**FS19.** The measured behavior belongs to the core that uses `fetch`. A template engine that treats a missing value as an empty string produces an empty heading from the same template without an error.

#### A Partial and Double Escaping

A large page is divided into small templates. The part of a template that is rendered inside another template is called a **partial**. The core has no separate helper for a partial; a compiled template can be given as the local value of another template. The helper below is this lesson's own plain Ruby helper and is not added to the core.

```ruby
# partial.rb — giving the output of a partial to the outer template
require_relative "stay"

# View: a template is compiled once; local values are escaped on every render.
def escape(text) = text.to_s.gsub("&", "&amp;").gsub("<", "&lt;").gsub(">", "&gt;").gsub('"', "&quot;")

def compile_template(b, name, source)
  b.app.store[:templates][name] ||= lambda do |locals|
    source.gsub(/\{\{(\w+)\}\}/) { escape(locals.fetch(Regexp.last_match(1).to_sym)) }
  end
end

def part(b, name, locals) = b.app.store[:templates].fetch(name).call(locals)

app = App.new
b = Context.new(app, {})
compile_template(b, "products/row", "<li>{{name}}</li>")
compile_template(b, "products/list", "<ul>{{body}}</ul>")

row = part(b, "products/row", name: "pen")
p row
p part(b, "products/list", body: row)
p part(b, "products/list", body: escape("a&b"))
```
```
"<li>pen</li>"
"<ul>&lt;li&gt;pen&lt;/li&gt;</ul>"
"<ul>a&amp;amp;b</ul>"
```

The partial gave `<li>pen</li>`. When this output was given to the outer template as a local value, the outer template escaped it too and the list became `<ul>&lt;li&gt;pen&lt;/li&gt;</ul>`: on the page the user sees not a tag but the text of the tag. In the core's model there is no way to mark a string as already rendered and safe; every local value, whatever its source, is escaped. The third line shows the other face of the same problem: if the value is escaped by hand beforehand on the calling side, the outer template escapes it once more and the `&` character changes again. The way out of double escaping is not to give the value without escaping; the core escapes it anyway in that case. The output of a partial therefore cannot be carried to the outer template as a local value; in this model the output has to be embedded in the source of the outer template.

**FS20.** Real template engines offer a way to mark rendered output as safe and solve the double escaping problem with it. The core does not model this, so the measured behavior is not seen in an engine that carries such a mark.

#### The Type of a Local Value

A template expects a string as a local value, but `escape` first turns every input into a string with `to_s`. Looking at how values that are not strings enter the page shows what a controller can give to a template.

```ruby
# kind.rb — local values that are not strings
require_relative "stay"

# View: a template is compiled once; local values are escaped on every render.
def escape(text) = text.to_s.gsub("&", "&amp;").gsub("<", "&lt;").gsub(">", "&gt;").gsub('"', "&quot;")

def compile_template(b, name, source)
  b.app.store[:templates][name] ||= lambda do |locals|
    source.gsub(/\{\{(\w+)\}\}/) { escape(locals.fetch(Regexp.last_match(1).to_sym)) }
  end
end

p escape(:name)
p escape(1.5)
p escape([1, "<"])
p escape({ a: 1 })
```
```
"name"
"1.5"
"[1, &quot;&lt;&quot;]"
"{a: 1}"
```

The output for the symbol `:name` and for the float is in the expected form. The array and the hash were turned into strings with Ruby's own representation and then escaped: the `"` characters in the array became `&quot;` and the `<` character became `&lt;`. In this case the page gives the user Ruby's internal representation, which is mostly not the wanted output. A controller should give the template the text to be shown ready; the template only escapes the string and does no formatting.

#### One Template, Many Renders

The compiled template stands as a single entry in the application store. Calling the same template with different local values does not grow the store; the rendered output is not stored either.

```ruby
# drawing.rb — one template, three renders
require_relative "stay"

# View: a template is compiled once; local values are escaped on every render.
def escape(text) = text.to_s.gsub("&", "&amp;").gsub("<", "&lt;").gsub(">", "&gt;").gsub('"', "&quot;")

def compile_template(b, name, source)
  b.app.store[:templates][name] ||= lambda do |locals|
    source.gsub(/\{\{(\w+)\}\}/) { escape(locals.fetch(Regexp.last_match(1).to_sym)) }
  end
end

app = App.new
b = Context.new(app, {})
compile_template(b, "products/show", "<h1>{{name}}</h1>")
store = app.store[:templates]

p store["products/show"].call(name: "pen")
p store["products/show"].call(name: "notebook")
p store["products/show"].call(name: "eraser")
p store.size
p store.keys
```
```
"<h1>pen</h1>"
"<h1>notebook</h1>"
"<h1>eraser</h1>"
1
["products/show"]
```

Three calls gave three separate outputs and none affected the others: the local values pass to the lambda as input on every call and do not stay inside the lambda. There was again a single entry in the store, and its key was the name of the template. A render is therefore in the **this request** stay: the output returns only to the code that made the call and is left nowhere. What is persistent is only the compiled form of the template.

#### The Template Name and Lookup

In the first lesson we saw that the template name is derived from the controller and action names. This name is not without meaning: the compiled templates in the store are looked up with this name. There is a link between the convention that produces the name and the store that looks it up, but this link is built only by the match of the name.

```ruby
# lookup.rb — looking up a template with a derived name
require_relative "stay"

# Convention: the template name is not written, it is derived from the controller and action names.
def template_name(params) = "#{params[:controller]}/#{params[:action]}"

# View: a template is compiled once; local values are escaped on every render.
def escape(text) = text.to_s.gsub("&", "&amp;").gsub("<", "&lt;").gsub(">", "&gt;").gsub('"', "&quot;")

def compile_template(b, name, source)
  b.app.store[:templates][name] ||= lambda do |locals|
    source.gsub(/\{\{(\w+)\}\}/) { escape(locals.fetch(Regexp.last_match(1).to_sym)) }
  end
end

app = App.new
b = Context.new(app, {})
store = app.store[:templates]
name = template_name(controller: "products", action: "show")

p store[name]
begin
  store.fetch(name)
rescue KeyError => e
  p e.class
end

compile_template(b, name, "<h1>{{name}}</h1>")
p store.fetch(name).call(name: "pen")
```
```
nil
KeyError
"<h1>pen</h1>"
```

Before the template was compiled, `store[name]` gave `nil`, while `fetch` raised a `KeyError`. Which one is chosen decides whether a missing template is reported silently or with an error. After compiling, the same name found the template and the render was done. The convention produces the name and the store looks it up; that one produced the name does not mean that the other compiled a template with that name. **FS21.** The measurement is for a single name and assumes that the name is consistent at the moment it is produced and the moment it is compiled.

#### The Stay of the Compiled Template

A template is compiled once and the result is written to the application store. We can ask the core's classifier for how many requests the compiled template is read.

```ruby
# views.rb — the stay of the compiled template
require_relative "stay"

# View: a template is compiled once; local values are escaped on every render.
def escape(text) = text.to_s.gsub("&", "&amp;").gsub("<", "&lt;").gsub(">", "&gt;").gsub('"', "&quot;")

def compile_template(b, name, source)
  b.app.store[:templates][name] ||= lambda do |locals|
    source.gsub(/\{\{(\w+)\}\}/) { escape(locals.fetch(Regexp.last_match(1).to_sym)) }
  end
end

write = ->(b) { compile_template(b, "products/show", "<h1>{{name}}</h1>") }
read = ->(b) { b.app.store[:templates]["products/show"]&.call(name: "<b>pen</b>") }
p stay(write, read)
```
```
["persistent", "A1 A2 A3 B4", "<h1>&lt;b&gt;pen&lt;/b&gt;</h1>"]
```

The result was `["persistent", "A1 A2 A3 B4", "<h1>&lt;b&gt;pen&lt;/b&gt;</h1>"]`. The template was there in all four reads, and client B read it too. The stay is **persistent**, because the template stands not in the request but in the application store. The third column also shows the effect of escaping: `<b>pen</b>`, given as a local value, became text in the output and not a tag.

What is measured here is not the rendered output; the render is done again on every call. What is persistent is the compiled form of the template. This difference matters: compiling is an expensive job and is done once, while rendering is cheap and is repeated on every request.

#### The Bounding Measurement

That compiling is done once also means that when the template source changes, the new source will not be seen. For the same name, the core keeps the existing template on a second compile.

```ruby
# limit.rb — a second compile with the same name keeps the old template
require_relative "stay"

# View: a template is compiled once; local values are escaped on every render.
def escape(text) = text.to_s.gsub("&", "&amp;").gsub("<", "&lt;").gsub(">", "&gt;").gsub('"', "&quot;")

def compile_template(b, name, source)
  b.app.store[:templates][name] ||= lambda do |locals|
    source.gsub(/\{\{(\w+)\}\}/) { escape(locals.fetch(Regexp.last_match(1).to_sym)) }
  end
end

app = App.new
b = Context.new(app, {})
store = app.store[:templates]

compile_template(b, "products/show", "<h1>{{name}}</h1>")
p store["products/show"].call(name: "a")

compile_template(b, "products/show", "<h2>{{name}}</h2>")
p store["products/show"].call(name: "a")

store.delete("products/show")
compile_template(b, "products/show", "<h2>{{name}}</h2>")
p store["products/show"].call(name: "a")
```
```
"<h1>a</h1>"
"<h1>a</h1>"
"<h2>a</h2>"
```

The second compile did not see the new source: the template still gave `<h1>a</h1>`. The core does not recompile a template whose name was compiled before; the new source is silently ignored. After the template was deleted from the store, the same call gave `<h2>a</h2>`. So the only way to refresh a compiled template is to remove it from the store. **FS22.** This measurement is for a store kept in memory; in a framework that reads from a source file, detecting a change is designed separately.

#### Summary

- The template engine escapes every local value; escaping changes four characters and, applied twice, gives an output such as `&amp;lt;`.
- A missing local value raises a `KeyError`; an extra local value is ignored.
- If the output of a partial is given to the outer template as a local value, it is escaped again; the core has no way to mark rendered output as safe.
- The stay of this lesson is **persistent**: the compiled template stands in the application store and is read in all requests and by all clients.
- A second compile with the same name keeps the old template; to refresh it, the template has to be deleted from the store.

#### Next Step

The controller carried the flash message and the view kept the template in the application store; the saved product also stands in the same store. The next topic opens this store: it measures where the data has to stand so that a saved product is visible in another browser too, and how the model layer builds this stay.

## The Data Layer

The model layer that maps table rows to objects, migrations that change the schema version by version, relationships between records and validations that stop a record before it is written, chained queries, the per-request query cache and the N+1 query problem.

### Object–Relational Mapping

Source: https://academia.sh/en/courses/mvc-framework/object-relational-mapping

Measures the layer that maps a table row to an object and a table to a model; shows that a saved value is read by another client too and where the model of this core reaches its limit.

The last lesson of the previous topic measured that a compiled template is shared by everyone and can go stale once compiled: a template was compiled for one client and another client ran the same compiled template. While it is rendered, a template shows a product. Where that product came from has not been asked yet. This topic is devoted to it: a user saved a product from a form, and a different user in a different browser is reading it. Is that so?

The axis of the course is the same: **how many requests does this value stay for?** There are four stays. A value that lasts only in the request that wrote it has the stay **this request**. A value that is still there only in the next request of the same client has the stay **next request**. A value present in all later requests of the same client but absent for another client has the stay **session**. A value that another client reads as well is **persistent**. The observation of this lesson is a product record, and the expected stay is persistent; but a label does not tell the stay, a measurement does.

The concept of the lesson is **object–relational mapping** (ORM) and the piece of software that implements it, the **mapper**. In a database, data lives in tables and tables hold rows; in a program, data lives in objects. The mapper connects the two worlds: a table corresponds to a **model**, and a row of the table corresponds to a **record** of that model. The same concept was built with PHP in M12/K04. There the measurement was "did this piece run in this request"; here the concept is the same and the measure differs: the question is not the record itself but how many requests the record stays for.

#### The Measurement Core

The first file is the request loop and the stay classifier, identical in every lesson of the course. This block is the source; the file is written to disk and pulled in with `require_relative`.

```ruby
# stay.rb — a small request loop and a helper that classifies how many requests a value stays for

# Flash: a value written in this request is carried to the next request only.
class Flash
  def initialize(incoming)
    @incoming = incoming
    @fresh = {}
  end

  def [](key) = @fresh.fetch(key) { @incoming[key] }

  def []=(key, value)
    @fresh[key] = value
  end

  def passed_on = @fresh
end

# Every request gets a new context: params, flash and the request store start empty each time.
class Context
  attr_reader :app, :params, :session, :flash, :request_store

  def initialize(app, session)
    @app = app
    @params = {}
    @session = session
    @flash = Flash.new(session.delete(:flash) || {})
    @request_store = {}
  end
end

# The application store is the same across all requests and all clients;
# the session is chosen per client.
class App
  attr_reader :store

  def initialize
    @sessions = Hash.new { |h, client| h[client] = {} }
    @store = { products: {}, schema: { products: %i[name price] }, templates: {},
               queue: [], cache: {}, counter: 0 }
  end

  def request(client)
    context = Context.new(self, @sessions[client])
    result = yield context
    context.session[:flash] = context.flash.passed_on unless context.flash.passed_on.empty?
    result
  end
end

# Rule: request 1 (client A) writes the value; requests 2 and 3 (A) and request 4 (B) read it.
# If B sees it, persistent; if A's third request sees it, session; if only the second
# request sees it, next; if none sees it, this_request.
def stay(write, read)
  app = App.new
  first = app.request("A") { |b| write.call(b); read.call(b) }
  raise ArgumentError, "value could not be read in the request that wrote it" if first.nil?

  seen = [["A", 2], ["A", 3], ["B", 4]].map do |client, order|
    app.request(client) { |b| read.call(b) }.nil? ? "--" : "#{client}#{order}"
  end
  kind = if seen[2] != "--" then "persistent"
         elsif seen[1] != "--" then "session"
         elsif seen[0] != "--" then "next"
         else "this_request"
         end
  [kind, ["A1", *seen].join(" "), first]
end
```

`App` carries the same `store` for all requests and all clients, while the session is chosen per client. Every request gets a new `Context`, and `request_store` is empty in each request. `stay` takes two lambdas, one that writes and one that reads: in the first request client A runs both, then two requests of A and one request of B only read. If B sees the value, the stay is persistent; if B does not but the third request of A does, session; if only the second request sees it, next; if none sees it, this request. The rule reads the value's behavior, not where it is stored.

The file of this lesson supplies the model. The excerpt is as much of the core as the lesson needs: the `Product` model.

```ruby
# mapping.rb — product model: a table maps to a model, a row maps to a record
require_relative "stay"

module Product
  def self.insert(b, attributes)
    errors = []
    errors << "name cannot be blank" if attributes[:name].to_s.empty?
    errors << "price cannot be negative" if attributes[:price].to_i.negative?
    return { errors: } unless errors.empty?

    table = b.app.store[:products]
    id = table.size + 1
    table[id] = attributes.merge(id:)
    { id:, errors: }
  end

  # The same query does not go to the database twice in the same request.
  def self.lookup(b, id)
    queries = (b.request_store[:queries] ||= {})
    queries.fetch([:products, id]) do
      b.request_store[:query_count] = b.request_store.fetch(:query_count, 0) + 1
      queries[[:products, id]] = b.app.store[:products][id]
    end
  end
end
```

This block is source as well. `insert` writes a record: the table is the hash `store[:products]`, whose keys are ids and whose values are rows. The id derives from the size of the table, so the first record gets `1` and the second `2`, and the row is the given attributes with the id added. For now the validation lines only fill the error list; the third lesson returns to them. `lookup` reads a row by id; the query cache belongs to the fourth lesson, and here only the read `store[:products][id]` matters.

#### The Stay of a Record

Does another browser read the saved product? The writing lambda saves a `pen`, and the reading lambda takes the name of the first row from the table.

```ruby
# measure.rb — stay of the record observation
require_relative "mapping"

pen = ->(b) { Product.insert(b, name: "pen", price: 12) }
read = ->(b) { b.app.store[:products][1]&.fetch(:name) }
p stay(pen, read)
```
```
["persistent", "A1 A2 A3 B4", "pen"]
```

Three fields are read. The stay is **persistent**: the rule saw the value in the fourth request of B as well. The evidence column `A1 A2 A3 B4` says that all four requests read the `pen`; The last field is the value read in the first request. This answers the question left at the end of the previous topic: another browser does read the saved product. **DL1.** The measurement is within a single process. The store of the core lives in memory, and it does not cover whether the data survives a process restart.

The reason for this is where `store` lives. Flash and the request store are born again in every request together with the `Context`; the session is tied to the client; `store`, however, belongs to the `App` object, and the app serves all requests. The model writes the value to the place that lasts the longest, so the stay came out persistent.

There is one more consequence. The table is shared between clients, so the records of two clients go into the same sequence of ids. A creates a product, B creates another one, and A reads the table.

```ruby
# two_clients.rb — two clients, one table
require_relative "mapping"

app = App.new
app.request("A") { |b| Product.insert(b, name: "pen", price: 12) }
app.request("B") { |b| Product.insert(b, name: "notebook", price: 30) }
app.request("A") do |b|
  table = b.app.store[:products]
  p table.keys
  p table.values.map { |row| row[:name] }
end
```
```
[1, 2]
["pen", "notebook"]
```

The third request of A saw both records. The record of B received the id `2`, because the id derives from the size of the table and the table is one piece together with the record of A. A persistent value does not belong only to the client that wrote it: what one client writes determines the id of another. **DL5.** The two requests here run one after the other; whether ids would collide when two requests arrive at the same time has not been measured, and this lesson does not claim it.

#### What the Model Takes On

Without a mapper, every controller would have to find its own way to the table: it would know under which key a row is written, how the id is produced, and which fields a row has. The model gathers this knowledge in one place. `insert` produces the id and writes the row; `lookup` reads a row by id; the caller uses two methods without knowing how the table's hash is built. This is the model layer of MVC (model–view–controller): the controller interprets the request, the view draws the page, and the model is the only layer that knows where and in what form the data lives.

The gain of this separation is that a change is made in one place. If the shape of a row changes, only `insert` and `lookup` are updated, and the code that calls them stays the same. The stay measured in this lesson is also a result of the model: because it is known which method writes the record, it is also known where the record is written.

#### Table, Row and Id

The same model is examined during one request: two records are created, the keys of the table and the second row are printed, and a missing id is asked for.

```ruby
# row.rb — table, row and id
require_relative "mapping"

app = App.new
app.request("A") do |b|
  one = Product.insert(b, name: "pen", price: 12)
  two = Product.insert(b, name: "notebook", price: 30)
  p [one[:id], two[:id]]
  p b.app.store[:products].keys
  p b.app.store[:products][2]
  p Product.lookup(b, 99)
end
```
```
[1, 2]
[1, 2]
{name: "notebook", price: 30, id: 2}
nil
```

The two `insert` calls gave the ids `[1, 2]`, and the keys of the table are the same list: the id is the address of the row in the table. The second row is the union of the two given attributes and the id. The missing id did not raise an error but gave `nil`; the "not found" answer of the model is a value, not an exception. For this reason the code that calls `lookup` handles the `nil` case itself: in the language of the model, "absent" is said with `nil`, not with an exception. **DL2.** Because ids derive from the size of the table, this measurement holds on the assumption that nothing is ever deleted; deletion does not exist in this core and has not been measured.

#### Input and Row

`insert` builds the row not from the hash the caller gave but from a new hash made with `merge`. So the caller's attributes and the row in the table do not share a stay. The measurement creates the record from a local variable and then changes the variable.

```ruby
# input.rb — the caller's hash and the row in the table
require_relative "mapping"

app = App.new
app.request("A") do |b|
  attributes = { name: "pen", price: 12 }
  Product.insert(b, attributes)
  attributes[:name] = "changed"
  p attributes.key?(:id)
  p b.app.store[:products][1][:name]
  p attributes.equal?(b.app.store[:products][1])
end

write = lambda do |b|
  attributes = { name: "pen", price: 12 }
  Product.insert(b, attributes)
  attributes[:name] = "changed"
end
read = ->(b) { b.app.store[:products][1]&.fetch(:name) }
p stay(write, read)
```
```
false
"pen"
false
["persistent", "A1 A2 A3 B4", "pen"]
```

The caller's hash never received the `id` key, the row stayed `"pen"`, and the two are not the same object. The changed local variable goes away with the request; the row in the table is persistent: although the writing lambda changes the local hash after the insert, the fourth request of B reads `"pen"`. In the core model the copy is taken when writing, not when reading: in the next section `lookup` hands back the very object in the table. This asymmetry belongs to the model; what a real mapper does in the two directions has not been measured here.

#### The Bounding Measurement

The model is not the whole of a real mapper, and its limit shows in one place. Does `lookup` return the row in the table or a copy of it? Whether two objects are the same object is asked with `equal?`. Then the record that was read is changed without saving.

```ruby
# boundary.rb — the model returns the stored object itself
require_relative "mapping"

app = App.new
app.request("A") do |b|
  Product.insert(b, name: "pen", price: 12)
  fetched = Product.lookup(b, 1)
  p fetched.equal?(b.app.store[:products][1])
  fetched[:name] = "changed"
  p b.app.store[:products][1][:name]
end
```
```
true
"changed"
```

`lookup` returned the very object in the table: `equal?` gave `true`. When the record that was read was changed, the table changed too, and there is no call that saves anything. In many mappers this is not so: every read builds a new object from the row, and a change to the object reaches the row only through an explicit save. This should be read not as a claim but as the limit of the core model. A small helper makes the difference visible; the helper is the code of this lesson and imitates the way such a mapper reads.

```ruby
# copy.rb — helper that imitates the read style of a real mapper
require_relative "mapping"

def fresh_read(b, id) = b.app.store[:products][id]&.dup

def write_back(b, record)
  b.app.store[:products][record[:id]] = record.dup
end

app = App.new
app.request("A") do |b|
  Product.insert(b, name: "pen", price: 12)
  duplicate = fresh_read(b, 1)
  duplicate[:name] = "changed"
  p b.app.store[:products][1][:name]
  write_back(b, duplicate)
  p b.app.store[:products][1][:name]
  p duplicate.equal?(b.app.store[:products][1])
end
```
```
"pen"
"changed"
false
```

When the copy was changed, the table stayed `"pen"`; the change reached the table only with the `write_back` call. The last line confirms that the object in the table is not the same object as the copy that was read: `false`. In this helper, what stays depends on the change being written back. **DL3.** The helper takes a copy and writes the record back; it does not fully represent what a real mapper does, and it only separates what the table sees before and after the write.

Finally, the same two behaviors are put in front of the stay rule: a writing lambda that creates the record and then changes only the object in memory.

```ruby
# same_object.rb — a record changed without saving
require_relative "mapping"

write = lambda do |b|
  Product.insert(b, name: "pen", price: 12)
  Product.lookup(b, 1)[:name] = "changed"
end
read = ->(b) { b.app.store[:products][1]&.fetch(:name) }
p stay(write, read)
```
```
["persistent", "A1 A2 A3 B4", "changed"]
```

The stay is persistent again and the value is `"changed"`. In the core model, the object that was read and the object in the table are the same, so there is no save step: every touch of the record is written directly to the place that another client reads too. For this reason, what is persistent in the core is the creation of the record as much as every change made to the record object. This is a simplification of the model and should not be read as the behavior of a real mapper.

#### Holding a Record Versus Saving It

The contrasting measurement keeps the same information in a variable of the request instead of writing it to the table. The writing lambda puts the name of the product into `request_store` and writes nothing to the table.

```ruby
# held.rb — the same information, not written to the table
require_relative "mapping"

write = ->(b) { b.request_store[:product] = { name: "pen", price: 12 } }
read = ->(b) { b.request_store[:product]&.fetch(:name) }
p stay(write, read)
```
```
["this_request", "A1 -- -- --", "pen"]
```

The value is the same, `"pen"`; the stay changed. The evidence column `A1 -- -- --` shows that it was read only in the first request, and the rule says **this request**. The difference between the two measurements lies not in the content of the data but in where it was written: `insert` wrote the value to the table, and this lambda wrote it to a variable of the request. Object–relational mapping establishes exactly this difference. What the model writes goes to a place that is not erased when the request ends. A product name in a local variable of a controller, on the other hand, is lost with the request. **DL4.** This contrasting measurement keeps the variable in `request_store`; what is measured is that a value that is not written to the table in this core never goes beyond one request, not the behavior of any other place of storage.

#### Summary

- Object–relational mapping connects a table to a model and a row of the table to a record of that model; the piece that makes this connection is called a mapper.
- The record, the observation of this lesson, is a value whose stay is persistent: the product saved in the first request was read in the next two requests of A and in the fourth request of B.
- The cause of the persistent stay is that `store` lives in the app; the request store, flash and session do not give this stay.
- Ids derive from the table and a missing id gives `nil`; the answer of the model is not an exception.
- The core model returns the stored object itself; many mappers build a new object on every read, and a change reaches the table only through a save.
- `insert` builds the row as a copy of the caller's hash; a hash changed afterwards does not affect the row.

#### Next Step

The record stays persistent, but the shape of the records depends on a table schema. When a column is added to the schema later, is it present in the old rows? The next lesson reads the migrations that change the schema with the same measure.

### Migrations

Source: https://academia.sh/en/courses/mvc-framework/migrations

Measures migrations, which change the table schema version by version; shows that an added column stays persistent, does not reach old rows, and why migrations are numbered.

The previous lesson showed that a saved product is read by another client too. In that measurement the product had two attributes: `name` and `price`. What happens when a stock count is to be added to a product as well? The shape of the table, that is, the **schema**, does not change as a hand-edited file but through recorded steps. These steps are called **migrations**: a migration is a named and ordered step that carries the schema from one state to another. Is a column added to the schema present in the old rows?

The axis has not changed: **how many requests does this value stay for?** A column lives in the schema, and the schema, like the records, is in the store of the app. For this reason the expected stay is **persistent**. The lesson measures this and then shows two limits of a migration: what happens if the same migration runs twice, and whether rows written before the migration see the new column. Where a change reaches was measured separately in M12/K02; the question there was "which target does the change reach", and here the question is how long the change that did reach stays.

#### The Measurement Core

The first file is the request loop and the stay classifier, identical in every lesson of the course. This block is the source; the file is written to disk and pulled in with `require_relative`.

```ruby
# stay.rb — a small request loop and a helper that classifies how many requests a value stays for

# Flash: a value written in this request is carried to the next request only.
class Flash
  def initialize(incoming)
    @incoming = incoming
    @fresh = {}
  end

  def [](key) = @fresh.fetch(key) { @incoming[key] }

  def []=(key, value)
    @fresh[key] = value
  end

  def passed_on = @fresh
end

# Every request gets a new context: params, flash and the request store start empty each time.
class Context
  attr_reader :app, :params, :session, :flash, :request_store

  def initialize(app, session)
    @app = app
    @params = {}
    @session = session
    @flash = Flash.new(session.delete(:flash) || {})
    @request_store = {}
  end
end

# The application store is the same across all requests and all clients;
# the session is chosen per client.
class App
  attr_reader :store

  def initialize
    @sessions = Hash.new { |h, client| h[client] = {} }
    @store = { products: {}, schema: { products: %i[name price] }, templates: {},
               queue: [], cache: {}, counter: 0 }
  end

  def request(client)
    context = Context.new(self, @sessions[client])
    result = yield context
    context.session[:flash] = context.flash.passed_on unless context.flash.passed_on.empty?
    result
  end
end

# Rule: request 1 (client A) writes the value; requests 2 and 3 (A) and request 4 (B) read it.
# If B sees it, persistent; if A's third request sees it, session; if only the second
# request sees it, next; if none sees it, this_request.
def stay(write, read)
  app = App.new
  first = app.request("A") { |b| write.call(b); read.call(b) }
  raise ArgumentError, "value could not be read in the request that wrote it" if first.nil?

  seen = [["A", 2], ["A", 3], ["B", 4]].map do |client, order|
    app.request(client) { |b| read.call(b) }.nil? ? "--" : "#{client}#{order}"
  end
  kind = if seen[2] != "--" then "persistent"
         elsif seen[1] != "--" then "session"
         elsif seen[0] != "--" then "next"
         else "this_request"
         end
  [kind, ["A1", *seen].join(" "), first]
end
```

The excerpt of this lesson is a single method. A migration, in its plainest form, adds a column name to a table in the schema. The schema is in the hash `store[:schema]`: the key is the table name and the value is the list of column names.

```ruby
# migrate.rb — schema change: a column is added to a table
require_relative "stay"

def migrate(b, table, column) = b.app.store[:schema][table] << column
```

This block is source as well. The core is not a framework but a model of a framework; this method stands for a single forward migration. Most migration mechanisms write each migration as a class or a file and can define a backward direction next to the forward one. Below, helpers of this lesson build the same distinction on a small scale.

#### The Stay of a Column

The writing lambda adds the column `stock` to the table `products`, and the reading lambda looks for this column in the schema.

```ruby
# measure.rb — stay of the schema column observation
require_relative "migrate"

write = ->(b) { migrate(b, :products, :stock) }
read = ->(b) { b.app.store[:schema][:products].find { |c| c == :stock } }
p stay(write, read)
```
```
["persistent", "A1 A2 A3 B4", :stock]
```

The stay is **persistent**: the fourth request of B found the column `:stock` in the schema too. The evidence column contains all four requests and none has `--`. The schema is in the store of the app, and the app serves all clients; a migration run by one client changes the schema that the others see as well. **DL6.** This measurement is within a single process: whether the migration remains after the process restarts cannot be measured in the in-memory store of the core.

Who runs the migration does not change the stay. If the same column is added in a request of client B, the next request of A sees it.

```ruby
# other_client.rb — the migration runs for one client, the other client reads
require_relative "migrate"

app = App.new
app.request("B") { |b| migrate(b, :products, :stock) }
app.request("A") { |b| p b.app.store[:schema][:products] }
```
```
[:name, :price, :stock]
```

The schema belongs to the app, not to a client: a schema changed in the request of one client is visible in every later request in its changed form, whichever client the request comes from. This rests on the same cause as the stay of the records in the previous lesson; both live in `store`. **DL9.** Here the migration ran inside a request because the core has no other place; where a migration runs in a real application is not the subject of this measurement.

#### Forward and Backward

Migrations have two directions. The forward migration changes the schema, and the **rollback** restores the change to its former state. The core has only the forward direction; the rollback is a helper of this lesson. First the state of the schema before and after the migration is examined, then its state after the rollback.

```ruby
# forward_back.rb — forward and backward migration
require_relative "migrate"

def revert(b, table, column) = b.app.store[:schema][table].delete(column)

app = App.new
app.request("A") do |b|
  schema = b.app.store[:schema]
  p schema[:products]
  migrate(b, :products, :stock)
  p schema[:products]
  revert(b, :products, :stock)
  p schema[:products]
end
```
```
[:name, :price]
[:name, :price, :stock]
[:name, :price]
```

The three output lines are three states: before the migration, after the forward migration, and after the rollback. The rollback returned the schema to the first state. That a migration has two directions says that the schema is a sequence of versions: each version is built on the previous one and forms a path that moves on in one direction and can go back in the other. This lesson measures only the schema side; the values inside the rows are asked separately in the bounding measurement below.

How many requests does the effect of the rollback stay for? The measurement below adds the column, writes a value into a row, rolls the column back, and reads the schema and the row separately. Then it passes the same rollback through the stay rule.

```ruby
# rollback_stay.rb — the rolled-back column and the value in the row
require_relative "migrate"

def revert(b, table, column) = b.app.store[:schema][table].delete(column)

app = App.new
app.request("A") do |b|
  migrate(b, :products, :stock)
  b.app.store[:products][1] = { name: "pen", price: 12, id: 1, stock: 5 }
  revert(b, :products, :stock)
  p b.app.store[:schema][:products].include?(:stock)
  p b.app.store[:products][1].key?(:stock)
end

write = lambda do |b|
  migrate(b, :products, :stock)
  revert(b, :products, :stock)
end
read = ->(b) { b.app.store[:schema][:products].include?(:stock) }
p stay(write, read)
```
```
false
true
["persistent", "A1 A2 A3 B4", false]
```

The schema no longer contains the column `:stock`, but the row keeps carrying the key and the value `5`. In this model the rollback touches only the schema; it does not delete the data in the rows. The stay line shows that the rolled-back schema is persistent as well: even the fourth request of B reads `false`, so the absence of the column was not limited to the request of A. In a real migration mechanism, whether a rollback also deletes the value in the rows depends on the mechanism and has not been measured here. In this model the step that restored the schema did not restore the data.

#### A New Table and a Missing Table

`migrate` only adds a column to an existing table. What happens if the same call is made for a table that is not in the schema? The measurement prints only the class of the exception that is caught.

```ruby
# missing_table.rb — adding a column to a table that does not exist
require_relative "migrate"

def create_table(b, table, *columns)
  b.app.store[:schema][table] = columns
end

app = App.new
app.request("A") do |b|
  begin
    migrate(b, :orders, :quantity)
  rescue => e
    p e.class
  end
  create_table(b, :orders, :product_id, :quantity)
  p b.app.store[:schema].keys
  p b.app.store[:schema][:orders]
end
```
```
NoMethodError
[:products, :orders]
[:product_id, :quantity]
```

Adding a column to a table that does not exist gave `NoMethodError`: that name was not in the schema, and the append method was called on `nil`. Creating a table is a different kind of migration. `create_table` wrote the table into the schema together with its column list, and the schema now contains two tables. In the next lesson, `Order` records will use this table. **DL7.** Here `orders` is only defined in the schema; no row is written to the table.

#### Why Migrations Are Numbered

What happens to the schema if the same forward migration runs twice? `migrate` appends to the end of a list and does not remember what it appended.

```ruby
# twice.rb — the same migration twice
require_relative "migrate"

app = App.new
app.request("A") do |b|
  migrate(b, :products, :stock)
  migrate(b, :products, :stock)
  p b.app.store[:schema][:products]
end
```
```
[:name, :price, :stock, :stock]
```

`:stock` is in the schema twice. The model itself has no version trace: no record remembers which migration has run, and it cannot be told whether the second call repeats the first or is a new migration. A real migration mechanism gives every migration a number for exactly this reason and keeps in a version table which numbers have been applied. The helper of this lesson builds the same idea on a small scale: a migration whose number has been applied does not run a second time.

```ruby
# numbered.rb — helper that remembers applied migrations
require_relative "migrate"

def numbered_migration(b, number, table, column)
  applied = (b.app.store[:applied] ||= [])
  return false if applied.include?(number)

  migrate(b, table, column)
  applied << number
  true
end

app = App.new
app.request("A") do |b|
  p numbered_migration(b, 1, :products, :stock)
  p numbered_migration(b, 1, :products, :stock)
  p b.app.store[:schema][:products]
  p b.app.store[:applied]
end

write = ->(b) { numbered_migration(b, 1, :products, :stock) }
read = ->(b) { b.app.store[:applied]&.first }
p stay(write, read)
```
```
true
false
[:name, :price, :stock]
[1]
["persistent", "A1 A2 A3 B4", 1]
```

The first call gave `true` and ran the migration; the second call with the same number gave `false` and did not touch the schema. `:stock` appears in the schema once. The list of applied migrations is `[1]`: the migration number lives in the store of the app like the schema, and its stay is persistent, because even the fourth request of B sees the first item of the list. The version record remembers that the migration ran, not what its result was. **DL8.** Here the number is an integer and migrations are called one by one, in order; what happens if the order is broken has not been measured.

What the record verifies can be asked separately. Adding the same column under two different numbers shows whether the record follows the number or the content of the migration.

```ruby
# two_numbers.rb — the same column, two numbers
require_relative "migrate"

def numbered_migration(b, number, table, column)
  applied = (b.app.store[:applied] ||= [])
  return false if applied.include?(number)

  migrate(b, table, column)
  applied << number
  true
end

app = App.new
app.request("A") do |b|
  p numbered_migration(b, 1, :products, :stock)
  p numbered_migration(b, 2, :products, :stock)
  p b.app.store[:schema][:products]
  p b.app.store[:applied]
end
```
```
true
true
[:name, :price, :stock, :stock]
[1, 2]
```

The second call gave `true` and `:stock` entered the schema a second time. The record follows the number and does not follow what the migration does: a migration that brings the same change under another number is not blocked. The protection that numbering gives rests on the numbers being given correctly. Whether real mechanisms keep a checksum of the content of the migration file has not been measured here.

#### The Bounding Measurement

A column added to the schema is persistent. But does a row written before the migration have this column? The stay of the schema and the stay of the data are separate questions. The measurement below writes a row directly to the table, runs the migration, and looks at both the schema and the row.

```ruby
# boundary.rb — a row written before the migration
require_relative "migrate"

app = App.new
app.request("A") do |b|
  b.app.store[:products][1] = { name: "pen", price: 12, id: 1 }
  migrate(b, :products, :stock)
  p b.app.store[:schema][:products].include?(:stock)
  p b.app.store[:products][1].key?(:stock)
  b.app.store[:products][2] = { name: "notebook", price: 30, id: 2, stock: 5 }
  p b.app.store[:products].values.map { |row| row.key?(:stock) }
end
```
```
true
false
[false, true]
```

The schema contains the column `:stock`, but the row written before the migration does not carry the key `:stock`: `key?` gave `false`. The row written after the migration has the key. The schema and the row are not one thing: the schema says what rows should look like and does not change the existing rows. In this model old rows never carry the new key; in a relational database the new column appears in old rows as empty (NULL) or with a default value. In both cases, old rows must be filled separately to get a meaningful value. For this reason two jobs are kept apart: the schema migration that adds the column and the data migration that writes a value into old rows. The helper below writes a default into every row that lacks the key.

```ruby
# backfill.rb — data migration that writes the new column into old rows
require_relative "migrate"

def backfill_column(b, table, column, value)
  b.app.store[table].each_value { |row| row[column] = value unless row.key?(column) }
end

app = App.new
app.request("A") do |b|
  b.app.store[:products][1] = { name: "pen", price: 12, id: 1 }
  migrate(b, :products, :stock)
  p b.app.store[:products].values.map { |row| row.key?(:stock) }
  backfill_column(b, :products, :stock, 0)
  p b.app.store[:products][1]
end

write = lambda do |b|
  b.app.store[:products][1] = { name: "pen", price: 12, id: 1 }
  migrate(b, :products, :stock)
end
read = ->(b) { b.app.store[:products][1]&.key?(:stock) }
p stay(write, read)
```
```
[false]
{name: "pen", price: 12, id: 1, stock: 0}
["persistent", "A1 A2 A3 B4", false]
```

First the old row gave `false`; after the backfill the row has `stock: 0`. The last line puts two stays side by side: this reading lambda looks for the column key in the row and finds `false`, but because `false` is a value, the stay rule counts it as seen and reads the same value in every request. As long as no key is added to the row after the migration, the fourth request of B reads `false` too. The same migration left a persistent column in the schema and a persistent absence in the row. These two must be kept apart: if the reader asks the schema, `:stock` exists; if it asks the row, it does not. The stay rule does not show this difference, because the rule reads the behavior of a value; whether an absence is an error or a default is decided by the code that reads the row.

#### Summary

- A migration is a named step that carries the schema from one state to another; its forward direction adds a column and its rollback direction removes it.
- The schema column, the observation of this lesson, is a value whose stay is persistent: the column `:stock` was read in the schema in the next two requests of A and in the fourth request of B.
- The core model has no version trace; when the same migration ran twice, `:stock` was written twice, which is why migrations are numbered and the list of applied ones is kept.
- A migration changes the schema and does not change existing rows: a row written before the migration has no new key, and filling is a separate step.
- In this model the rollback restores the schema and does not touch the data in the rows. The record follows the number, not the content of the migration; the same content under a different number is applied again.
- The stay of the column and the stay of the data are separate things; both are persistent, but one preserves the existence of the column and the other the absence of the value in the row.

#### Next Step

The schema and the rows are now defined. But a record must obey rules before it enters the table, and a record can be linked to another record. Where does the error of a rejected record stay? The next lesson reads relationships and validations with the same measure.

### Relationships and Validations

Source: https://academia.sh/en/courses/mvc-framework/relationships-and-validations

Measures validations that stop a record before it is written and the one-to-many relationship that links records; shows that a validation error stays only in this request and differs from a notice message.

The previous lesson showed that a column added to the schema is persistent and that it does not reach rows written before the migration. The schema and the rows were defined, but must a record accept every value when it enters the table? If a product with a negative price or an empty name is written to the table, every client that reads that row reads corrupt data. The rules that check a record before it is written are called **validations**. Where does the error of a rejected record stay?

The question is asked throughout the lesson with the same measure: **how many requests does this value stay for?** A validation error is produced to be shown to the user who tried to save the product; the user will correct the form and send it again. Does this information need to be at hand in the next request as well? The lesson measures the answer. The second concept of the lesson is the **relationship**: one record pointing to another record with an id field. The lesson reads the relationship with the same measure.

#### The Measurement Core

The first file is the request loop and the stay classifier, identical in every lesson of the course. This block is the source; the file is written to disk and pulled in with `require_relative`.

```ruby
# stay.rb — a small request loop and a helper that classifies how many requests a value stays for

# Flash: a value written in this request is carried to the next request only.
class Flash
  def initialize(incoming)
    @incoming = incoming
    @fresh = {}
  end

  def [](key) = @fresh.fetch(key) { @incoming[key] }

  def []=(key, value)
    @fresh[key] = value
  end

  def passed_on = @fresh
end

# Every request gets a new context: params, flash and the request store start empty each time.
class Context
  attr_reader :app, :params, :session, :flash, :request_store

  def initialize(app, session)
    @app = app
    @params = {}
    @session = session
    @flash = Flash.new(session.delete(:flash) || {})
    @request_store = {}
  end
end

# The application store is the same across all requests and all clients;
# the session is chosen per client.
class App
  attr_reader :store

  def initialize
    @sessions = Hash.new { |h, client| h[client] = {} }
    @store = { products: {}, schema: { products: %i[name price] }, templates: {},
               queue: [], cache: {}, counter: 0 }
  end

  def request(client)
    context = Context.new(self, @sessions[client])
    result = yield context
    context.session[:flash] = context.flash.passed_on unless context.flash.passed_on.empty?
    result
  end
end

# Rule: request 1 (client A) writes the value; requests 2 and 3 (A) and request 4 (B) read it.
# If B sees it, persistent; if A's third request sees it, session; if only the second
# request sees it, next; if none sees it, this_request.
def stay(write, read)
  app = App.new
  first = app.request("A") { |b| write.call(b); read.call(b) }
  raise ArgumentError, "value could not be read in the request that wrote it" if first.nil?

  seen = [["A", 2], ["A", 3], ["B", 4]].map do |client, order|
    app.request(client) { |b| read.call(b) }.nil? ? "--" : "#{client}#{order}"
  end
  kind = if seen[2] != "--" then "persistent"
         elsif seen[1] != "--" then "session"
         elsif seen[0] != "--" then "next"
         else "this_request"
         end
  [kind, ["A1", *seen].join(" "), first]
end
```

The core is not a framework but a model of a framework. The second file of the lesson takes from the core only the `insert` method that carries the validations. Validations run before the record is written, and the failure of each rule adds a message to the error list.

```ruby
# validations.rb — product model: validations run before the record is written
require_relative "stay"

module Product
  def self.insert(b, attributes)
    errors = []
    errors << "name cannot be blank" if attributes[:name].to_s.empty?
    errors << "price cannot be negative" if attributes[:price].to_i.negative?
    return { errors: } unless errors.empty?

    table = b.app.store[:products]
    id = table.size + 1
    table[id] = attributes.merge(id:)
    { id:, errors: }
  end
end
```

This block is source as well. `insert` returns a hash: if the validation passes, an `id` and an empty `errors` list; if it does not, only the `errors` list. The lines that write the record are reached only when the error list is empty; `return` skips them.

#### The Error List of a Validation

Does a validation stop at the first error, or does it run all the rules? The measurement below creates two valid records and then tries a record with an empty name and a negative price.

```ruby
# errors.rb — validation errors and the size of the table
require_relative "validations"

app = App.new
app.request("A") do |b|
  p Product.insert(b, name: "pen", price: 12)
  p Product.insert(b, name: "notebook", price: 30)
  result = Product.insert(b, name: "", price: -1)
  p result[:errors]
  p result.key?(:id)
  p b.app.store[:products].size
end
```
```
{id: 1, errors: []}
{id: 2, errors: []}
["name cannot be blank", "price cannot be negative"]
false
2
```

The valid records returned an id and an empty error list. The failed record gave two errors at once: the validation did not stop at the first error and tried every rule. This lets the user who corrects the form see all the problems in a single pass. The result has no `id` key and the size of the table stayed `2`: the failed record never entered the table. Because the rejected record does not exist, looking for its error in the table is meaningless; the error exists only in the value that `insert` returned.

What a validation rejects is exactly what the written rules say. Boundary values show this: a zero price, a space-only name and a missing price.

```ruby
# limits.rb — exactly what the rule says
require_relative "validations"

app = App.new
app.request("A") do |b|
  p Product.insert(b, name: "pen", price: 0)[:errors]
  p Product.insert(b, name: " ", price: 5)[:errors]
  p Product.insert(b, name: "notebook", price: nil)[:errors]
  p b.app.store[:products].size
end
```
```
[]
[]
[]
3
```

All three records were accepted and the size of the table is `3`. The price rule rejects only a negative number; zero and `nil` pass, because `nil.to_i` is zero. The name rule rejects only an empty string; a single space character does not count as empty. A validation applies the written rule, not the intention: whoever wants to stop a space-only name or a missing price must write a separate rule.

#### The Stay of an Error

The observation of the lesson is the error of a product with an empty name. The writing lambda puts the result into `request_store`; the reading lambda takes the first error message.

```ruby
# measure.rb — stay of the validation error observation
require_relative "validations"

write = ->(b) { b.request_store[:record] = Product.insert(b, name: "", price: 5) }
read = ->(b) { b.request_store.dig(:record, :errors, 0) }
p stay(write, read)
```
```
["this_request", "A1 -- -- --", "name cannot be blank"]
```

The stay is **this request**. The evidence column is `A1 -- -- --`: the error was read only in the request that wrote it; it is absent in the second and third requests of A and in the fourth request of B. The reason lies in the nature of `request_store`: every request gets a new `Context` and the request store starts empty. The error is not in the table because it was not saved; it is in the request store, so it goes away when the request ends. **DL10.** In this measurement the error was saved in `request_store`. An app that keeps the error somewhere else could give another stay; the claim of this lesson is only that the stay is this request when the error is in the request store.

#### A One-to-Many Relationship

A product can have more than one order. This is a **one-to-many relationship**; the same relationship seen from the order to the product is called the **belongs-to relationship**. The core has no mechanism for relationships; records are plain hashes. For this reason the relationship is built with a helper of the lesson in plain Ruby: the order record carries the id of the product in a field, and the orders of a product are found by filtering on this field.

```ruby
# order.rb — one-to-many relationship: the order carries the id of the product
require_relative "validations"

module Order
  def self.place(b, attributes)
    errors = []
    errors << "product not found" unless b.app.store[:products].key?(attributes[:product_id])
    errors << "quantity must be positive" unless attributes[:quantity].to_i.positive?
    return { errors: } unless errors.empty?

    table = (b.app.store[:orders] ||= {})
    id = table.size + 1
    table[id] = attributes.merge(id:)
    { id:, errors: }
  end

  def self.orders_for_product(b, product_id)
    (b.app.store[:orders] || {}).values.select { |o| o[:product_id] == product_id }
  end
end
```

This block is source as well. `Order.place` carries a validation too: if the product that the order is placed for is not in the table, the record is rejected. A validation protects the relationship; no order is written without a product to link to. `orders_for_product` is the other direction of the relationship: it takes the id of a product and filters the order table by that id.

```ruby
# relationship.rb — two products, three orders
require_relative "order"

app = App.new
app.request("A") do |b|
  Product.insert(b, name: "pen", price: 12)
  Product.insert(b, name: "notebook", price: 30)
  Order.place(b, product_id: 1, quantity: 3)
  Order.place(b, product_id: 1, quantity: 1)
  Order.place(b, product_id: 2, quantity: 5)
  p Order.orders_for_product(b, 1).map { |o| o[:quantity] }
  p Order.orders_for_product(b, 2).map { |o| o[:quantity] }
  p Order.orders_for_product(b, 99)
  p Order.place(b, product_id: 99, quantity: 1)
  p b.app.store[:orders].size
end
```
```
[3, 1]
[5]
[]
{errors: ["product not found"]}
3
```

The orders of the first product are `[3, 1]` and those of the second `[5]`. A product that does not exist has no orders: an empty list, not an error. An order placed for a product that does not exist was rejected and the error message was returned; the size of the order table stayed `3`. The relationship is not separate data: it is the `product_id` field in the order record. For this reason the stay of the relationship is the stay of the record that carries it. The measurement below tests this against the rule.

```ruby
# relationship_stay.rb — stay of the relationship
require_relative "order"

write = lambda do |b|
  Product.insert(b, name: "pen", price: 12)
  Order.place(b, product_id: 1, quantity: 3)
  Order.place(b, product_id: 1, quantity: 1)
end
read = ->(b) { Order.orders_for_product(b, 1).size }
p stay(write, read)
```
```
["persistent", "A1 A2 A3 B4", 2]
```

The relationship is persistent: the two orders of the product were read in all four requests. The order records are in the table and the table lives in the app; because the relationship is an id in the record rather than extra data, it stays together with the record. **DL11.** Filtering walks the whole table; the cost of this approach when the number of orders grows has not been measured, and the claim of the lesson concerns only the stay.

#### Belongs-to Direction and the Orphan Record

The other direction of the relationship goes from the order to the product. Because the order carries the id of the product and not the product itself, reaching the product means looking that id up in the table. This means that when the product changes, the order will appear changed too. The measurement changes the name of the product after the order is placed and reads the name through the order.

```ruby
# belongs_to.rb — reading from the order to the product
require_relative "order"

lookup = ->(b, id) { b.app.store[:products][b.app.store[:orders][id][:product_id]] }

app = App.new
app.request("A") do |b|
  Product.insert(b, name: "pen", price: 12)
  Order.place(b, product_id: 1, quantity: 3)
  p lookup.call(b, 1)[:name]
  b.app.store[:products][1][:name] = "pencil"
  p lookup.call(b, 1)[:name]
  p b.app.store[:orders][1].key?(:name)
end

write = lambda do |b|
  Product.insert(b, name: "pen", price: 12)
  Order.place(b, product_id: 1, quantity: 3)
  b.app.store[:products][1][:name] = "pencil"
end
read = ->(b) { lookup.call(b, 1)[:name] }
p stay(write, read)
```
```
"pen"
"pencil"
false
["persistent", "A1 A2 A3 B4", "pencil"]
```

The order does not carry the name of the product inside itself: `key?(:name)` gave `false`. When the name changed, the name read through the order changed too, and the change is persistent: the fourth request of B reads the new name as well. In this model the relationship is an id, so the current state of the related record is what is seen; whether a loaded object in a real mapper can go stale has not been measured here. To keep a past price or name in the order, the value must be copied into the order separately; this model does not do that.

What happens if the product is deleted? The order record keeps carrying the id of the product. Because the validation runs only while an order is being written, it does not protect the orders that already exist.

```ruby
# orphan.rb — an order whose product was deleted
require_relative "order"

app = App.new
app.request("A") do |b|
  Product.insert(b, name: "pen", price: 12)
  Order.place(b, product_id: 1, quantity: 3)
  b.app.store[:products].delete(1)
  p Order.orders_for_product(b, 1).size
  p b.app.store[:products][1]
  p Order.place(b, product_id: 1, quantity: 1)
end
```
```
1
nil
{errors: ["product not found"]}
```

Even though the product was deleted, the product with the id `1` still has `1` order, and in the product table that id is `nil`. A new order, on the other hand, was rejected with the `product not found` error. The validation protects the relationship for new records and does not protect it for existing ones; an order without a product, that is an orphan record, is prevented by handling the related records at the moment of deletion (refusing the deletion, deleting the related records, or emptying them); this model has no such step. What a real mapper does to related records on deletion depends on its configuration and has not been measured here.

#### The Bounding Measurement

A validation error is not stored with the record: a valid record is persistent, and the error stays in this request. But what if the error has to be carried to the next page? That is the job of the notice message (flash), and its stay is **next request**. Three mechanisms are put side by side with the same measure: keeping the error in the request store, putting the error into the notice message, and saving a valid record.

```ruby
# contrast.rb — error, notice message and valid record
require_relative "validations"

failure = ->(b) { Product.insert(b, name: "", price: 5) }

in_store = ->(b) { b.request_store[:record] = failure.call(b) }
store_read = ->(b) { b.request_store.dig(:record, :errors, 0) }
p stay(in_store, store_read)

in_flash = ->(b) { b.flash[:error] = failure.call(b)[:errors].first }
flash_read = ->(b) { b.flash[:error] }
p stay(in_flash, flash_read)

in_table = ->(b) { Product.insert(b, name: "pen", price: 12) }
table_read = ->(b) { b.app.store[:products][1]&.fetch(:name) }
p stay(in_table, table_read)
```
```
["this_request", "A1 -- -- --", "name cannot be blank"]
["next", "A1 A2 -- --", "name cannot be blank"]
["persistent", "A1 A2 A3 B4", "pen"]
```

The same error message gave two stays. In the request store it was read only in the request that wrote it; when it was put into the notice message it was read in the second request of A as well and disappeared in the third. The valid record is persistent. The validation error itself is always the same value; its stay is determined by where you put it. The stay rule looks at the behavior of the value, not at its content.

This has a consequence for showing the form again. If the request that submits the product form is rejected, the form with errors must be drawn for the user in the same request: the error exists in that request and may not exist in the next one. If the user will be redirected to a later page, the error must be put into the notice message. The measurement below shows the first way.

```ruby
# form.rb — the error is drawn in the same request, absent in the next
require_relative "validations"

app = App.new
rendered = app.request("A") do |b|
  result = Product.insert(b, name: "", price: -1)
  b.request_store[:record] = result
  result[:errors].empty? ? "saved" : "form: #{result[:errors].join(', ')}"
end
p rendered
p app.request("A") { |b| b.request_store[:record] }
```
```
"form: name cannot be blank, price cannot be negative"
nil
```

The first request drew the form with errors together with both errors. The very next request of the same client read `nil` for `request_store[:record]`: the error is no longer there. **DL12.** This drawing only produces text; it does not run a real view template. What is measured is in which request the error can be read, not the appearance of the form.

#### Summary

- A validation runs before the record is written, tries every rule and returns an error list; a failed record is not written to the table.
- A one-to-many relationship is an id field in a record; the stay of the relationship is persistent because it is the stay of the record that carries it.
- The validation error, the observation of this lesson, is a value whose stay is this request: the error of a product with an empty name was read only in the request that wrote it.
- The error is not stored with the record: a valid record is persistent and the error stays only in this request.
- In the belongs-to direction an order carries the id of the product, not its name; when the product name changes, the new name is read through the order. A validation runs only while writing; if the product is deleted the order is left an orphan, and the validation does not protect existing records.
- Carrying the error to the next page is the job of the notice message (flash), and its stay is next request; to show the form again, the error must be drawn in the same request.

#### Next Step

Records, relationships and validations can now be written. When a request reads the orders of a product, how many times does it go to the database? The next lesson measures how many times the same query goes to the database in one request.

### The Query Interface and the N+1 Problem

Source: https://academia.sh/en/courses/mvc-framework/query-interface-and-the-n-plus-one-problem

Measures how many queries go to the database: the query cache stays in this request, the N+1 problem and its fix with eager loading, and the way the cache stores an empty result too.

The previous lesson showed that a validation error is read only in the request that wrote it, and that a relationship is nothing more than an id field in an order record. The order table was filtered to read the orders of a product. How many times does a request go to the database to fill a page? Reading one product is one query; reading the product of each of a hundred orders separately is a hundred queries. The number of queries is one of the numbers that decide how long it takes to draw a page.

The **query interface** is the sum of the methods that read records by condition, by id or by relationship. This lesson measures not the interface itself but how many queries run behind it. The measure is the same question as always in the course: **how many requests does this value stay for?** The value measured here is the **query cache**, which prevents the same query from being asked a second time. Because its name says cache, it may be taken for persistent. The lesson measures this.

#### The Measurement Core

The first file is the request loop and the stay classifier, identical in every lesson of the course. This block is the source; the file is written to disk and pulled in with `require_relative`.

```ruby
# stay.rb — a small request loop and a helper that classifies how many requests a value stays for

# Flash: a value written in this request is carried to the next request only.
class Flash
  def initialize(incoming)
    @incoming = incoming
    @fresh = {}
  end

  def [](key) = @fresh.fetch(key) { @incoming[key] }

  def []=(key, value)
    @fresh[key] = value
  end

  def passed_on = @fresh
end

# Every request gets a new context: params, flash and the request store start empty each time.
class Context
  attr_reader :app, :params, :session, :flash, :request_store

  def initialize(app, session)
    @app = app
    @params = {}
    @session = session
    @flash = Flash.new(session.delete(:flash) || {})
    @request_store = {}
  end
end

# The application store is the same across all requests and all clients;
# the session is chosen per client.
class App
  attr_reader :store

  def initialize
    @sessions = Hash.new { |h, client| h[client] = {} }
    @store = { products: {}, schema: { products: %i[name price] }, templates: {},
               queue: [], cache: {}, counter: 0 }
  end

  def request(client)
    context = Context.new(self, @sessions[client])
    result = yield context
    context.session[:flash] = context.flash.passed_on unless context.flash.passed_on.empty?
    result
  end
end

# Rule: request 1 (client A) writes the value; requests 2 and 3 (A) and request 4 (B) read it.
# If B sees it, persistent; if A's third request sees it, session; if only the second
# request sees it, next; if none sees it, this_request.
def stay(write, read)
  app = App.new
  first = app.request("A") { |b| write.call(b); read.call(b) }
  raise ArgumentError, "value could not be read in the request that wrote it" if first.nil?

  seen = [["A", 2], ["A", 3], ["B", 4]].map do |client, order|
    app.request(client) { |b| read.call(b) }.nil? ? "--" : "#{client}#{order}"
  end
  kind = if seen[2] != "--" then "persistent"
         elsif seen[1] != "--" then "session"
         elsif seen[0] != "--" then "next"
         else "this_request"
         end
  [kind, ["A1", *seen].join(" "), first]
end
```

The core is not a framework but a model of a framework. The second file carries the product model and the `lookup` method that reads by id. `lookup` first looks at the query cache in the request store; if the id is not there, it increments the counter, reads the table and writes the result into the cache.

```ruby
# query.rb — product model: lookup runs the same query once in the same request
require_relative "stay"

module Product
  def self.insert(b, attributes)
    errors = []
    errors << "name cannot be blank" if attributes[:name].to_s.empty?
    errors << "price cannot be negative" if attributes[:price].to_i.negative?
    return { errors: } unless errors.empty?

    table = b.app.store[:products]
    id = table.size + 1
    table[id] = attributes.merge(id:)
    { id:, errors: }
  end

  def self.lookup(b, id)
    queries = (b.request_store[:queries] ||= {})
    # The same query does not go to the database twice in the same request.
    queries.fetch([:products, id]) do
      b.request_store[:query_count] = b.request_store.fetch(:query_count, 0) + 1
      queries[[:products, id]] = b.app.store[:products][id]
    end
  end
end
```

This block is source as well. The cache key is the pair `[:products, id]`: which table and which id. The cache and the counter live in `request_store`, so they start anew for every request.

#### The Stay of the Cache

The observation of the lesson is the query cache. The writing lambda creates a product and reads it by id; the reading lambda looks at the name of the record in the cache.

```ruby
# measure.rb — stay of the query cache observation
require_relative "query"

pen = ->(b) { Product.insert(b, name: "pen", price: 12) }
write = ->(b) { pen.call(b); Product.lookup(b, 1) }
read = ->(b) { b.request_store.dig(:queries, [:products, 1], :name) }
p stay(write, read)
```
```
["this_request", "A1 -- -- --", "pen"]
```

The stay is **this request**. The evidence column is `A1 -- -- --`: the record in the cache was read only in the request that wrote it; it is absent in the later requests of A and in the request of B. The name cache gives the impression that the value lives long, but the measured value is in the request store and goes away when the request ends. What is persistent is the product itself, not the copy in the cache. **DL13.** In this measurement the cache is in `request_store`. A mechanism that kept the cache in the store of the app would give another stay; the claim of this lesson concerns only the stay of a cache in the request store.

#### The Number of Queries

To see that the cache works, look at the counter. Three products are created in one request; in the next request `lookup` is called with three different ids, and then again with the same three ids.

```ruby
# counter.rb — the same id is not queried a second time
require_relative "query"

app = App.new
app.request("A") do |b|
  %w[pen notebook eraser].each_with_index { |name, i| Product.insert(b, name:, price: 10 + i) }
end

counts = app.request("A") do |b|
  [1, 2, 3].each { |id| Product.lookup(b, id) }
  first = b.request_store[:query_count]
  [1, 2, 3].each { |id| Product.lookup(b, id) }
  [first, b.request_store[:query_count]]
end
p counts
p app.request("A") { |b| b.request_store[:query_count] }
```
```
[3, 3]
nil
```

Three different ids counted three queries. The second round with the same ids did not move the counter: it stayed `3`. In the next request the counter is `nil`: because the request store started empty with the new request, both the counter and the cache start from zero. The same query runs twice in two separate requests.

An earlier lesson of the course made "how many times did it run" the measure, and a number is printed in this lesson too. The difference is this: here the number is not the thing measured by itself but the proof that the cache works. The value the lesson classifies is still the stay: the query cache stays in this request.

Whether the cache recognizes the same query depends on the key. The key is the pair `[:products, id]`, and the pair is compared together with the type of the id. An id that comes from a form or from the address line often arrives as text; the measurement below asks for the same id as an integer and as text.

```ruby
# key.rb — the same id, two rounds
require_relative "query"

app = App.new
app.request("A") { |b| Product.insert(b, name: "pen", price: 12) }

app.request("A") do |b|
  p Product.lookup(b, 1)[:name]
  p Product.lookup(b, "1")
  p Product.lookup(b, "1".to_i)[:name]
  p b.request_store[:query_count]
  p b.request_store[:queries].keys
end
```
```
"pen"
nil
"pen"
2
[[:products, 1], [:products, "1"]]
```

`1` and `"1"` became two separate keys: the counter showed `2`, so the cache did not count the second call as the same query. The query made with the text id also returned `nil`, because the keys of the table are integers. `"1".to_i`, on the other hand, coincided with the first key: the third call returned from the cache and the counter did not rise. This is a property of the core model: converting the id to an integer is the caller's job. Whether real mappers convert a text id by themselves has not been measured here; what is measured is that the cache depends on key equality and that equality covers the type. Code that wants to keep the number of queries low must convert the id to a single type before it enters the cache.

#### The N+1 Problem

Let a page show the orders and the product of each order. First the orders are read: one query. Then reading the product for each order with a separate query means as many more queries as there are orders. If the number of orders is N, the total is N+1 queries. This is called the **N+1 problem**, and the approach that fetches a related record only when it is accessed is called **lazy loading**; when each record is accessed once, this means a separate query for each record. The fix is **eager loading**, where the related records are fetched together with a single query.

This lesson builds a small query helper that does not depend on the cache and is measured only by its counter. Each method increments the counter when it runs a query.

```ruby
# order_queries.rb — helpers that count queries
require_relative "query"

module OrderQueries
  def self.tick(b)
    b.request_store[:query_count] = b.request_store.fetch(:query_count, 0) + 1
  end

  def self.orders(b)
    tick(b)
    b.app.store[:orders].values
  end

  def self.product(b, id)
    tick(b)
    b.app.store[:products][id]
  end

  def self.products(b, ids)
    tick(b)
    b.app.store[:products].values_at(*ids)
  end
end
```

This block is source as well. `orders` reads all the orders with one query, `product` reads one product with one query, and `products` reads several products, again with one query. The measurement below reads five orders in three ways: lazy, eager, and through the `Product.lookup` cache.

```ruby
# n_plus_one.rb — the same page, three ways, three query counts
require_relative "order_queries"

app = App.new
app.request("A") do |b|
  %w[pen notebook eraser].each_with_index { |name, i| Product.insert(b, name:, price: 10 + i) }
  records = [1, 1, 2, 3, 2].each_with_index.to_h do |product_id, i|
    [i + 1, { id: i + 1, product_id:, quantity: i + 1 }]
  end
  b.app.store[:orders] = records
end

lazy = app.request("A") do |b|
  names = OrderQueries.orders(b).map { |o| OrderQueries.product(b, o[:product_id])[:name] }
  [names, b.request_store[:query_count]]
end

eager = app.request("A") do |b|
  records = OrderQueries.orders(b)
  products = OrderQueries.products(b, records.map { |o| o[:product_id] }.uniq).to_h { |pr| [pr[:id], pr] }
  names = records.map { |o| products[o[:product_id]][:name] }
  [names, b.request_store[:query_count]]
end

cached = app.request("A") do |b|
  names = OrderQueries.orders(b).map { |o| Product.lookup(b, o[:product_id])[:name] }
  [names, b.request_store[:query_count]]
end

p lazy
p eager
p cached
p lazy[0] == eager[0] && eager[0] == cached[0]
```
```
[["pen", "pen", "notebook", "eraser", "notebook"], 6]
[["pen", "pen", "notebook", "eraser", "notebook"], 2]
[["pen", "pen", "notebook", "eraser", "notebook"], 4]
true
```

The three ways produced the same five names, but the query counts differ. The lazy way ran `6` queries: one for the orders and one more for each order, that is 1+5. The eager way ran `2` queries: one for the orders and one for the distinct products. Even if the number of orders grows, the count of the eager way stays two; the count of the lazy way grows with the orders. The third way is between the two: the cache queries repeated ids once, so `4` queries ran, that is 1 plus three different products. The cache does not remove N+1; it lowers it to the number of different ids, and if the different ids are as many as the orders, the count grows again.

**DL14.** This measurement counts queries and does not measure time. The cost per query is close to zero in this in-memory model; that every query in a real database has a round-trip cost has not been measured in this lesson, and the claim of the lesson is only the difference between the counts.

#### An Id That Does Not Exist in Eager Loading

Eager loading lowers the query for related records to one, but it also changes the shape of the result. In this model, if one of the ids read together is not in the table, its place in the result list stays `nil`; what real mappers do with a missing record has not been measured here. The measurement looks at what the eager way does when the product of an order has been deleted.

```ruby
# missing_product.rb — an id that does not exist in eager loading
require_relative "order_queries"

app = App.new
app.request("A") do |b|
  Product.insert(b, name: "pen", price: 12)
  b.app.store[:orders] = {
    1 => { id: 1, product_id: 1, quantity: 2 },
    2 => { id: 2, product_id: 99, quantity: 1 }
  }
end

app.request("A") do |b|
  records = OrderQueries.orders(b)
  products = OrderQueries.products(b, records.map { |o| o[:product_id] }.uniq)
  p products.map { |pr| pr&.fetch(:name) }
  p b.request_store[:query_count]
  begin
    products.to_h { |pr| [pr[:id], pr] }
  rescue => e
    p e.class
  end
end
```
```
["pen", nil]
2
NoMethodError
```

One query for the orders and one for the products ran, and the counter showed `2`; but in the result list the place of the id that does not exist is `nil`. Calling `pr[:id]` on this `nil` raised `NoMethodError`. The hash built in bulk could not be built at all because of a single missing record. Eager loading keeps the number of queries constant, but it leaves the responsibility of handling a missing related record to the calling code. The orphan order of the previous lesson appears here as `NoMethodError`.

#### The Bounding Measurement

Does the cache store only the records that are found? The first query with an id that does not exist returns `nil`. If the query cache stores this `nil` too, the second query for the missing id does not go to the database. This has a side effect: if the record is added later in the same request, the cache still returns `nil`.

```ruby
# boundary.rb — the cache stores nil too
require_relative "query"

app = App.new
app.request("A") do |b|
  %w[pen notebook eraser].each_with_index { |name, i| Product.insert(b, name:, price: 10 + i) }
end

result = app.request("A") do |b|
  [1, 2, 3].each { |id| Product.lookup(b, id) }
  Product.lookup(b, 99)
  Product.lookup(b, 99)
  count = b.request_store[:query_count]
  before = Product.lookup(b, 4)
  Product.insert(b, name: "ruler", price: 8)
  after = Product.lookup(b, 4)
  [count, before, after, b.app.store[:products][4][:name]]
end
p result
p app.request("A") { |b| Product.lookup(b, 4)[:name] }
p app.request("A") { |b| b.request_store[:query_count] }
```
```
[4, nil, nil, "ruler"]
"ruler"
nil
```

The three different ids and the missing id `99` counted four queries; the second call for `99` did not move the counter. Because the cache stores a `nil` result too, it remembered it as an empty record. The same happened with the fourth id: `nil` entered the cache while the product did not yet exist; after the product was created, `lookup(4)` returned `nil` again in the same request, although the product was in the table as `"ruler"`. The cache stayed stale in this request. Because the next request started the cache from zero, it read the same product correctly, and the counter was again `nil` in the new request.

This is the flip side of the stay of the cache. Because the cache stays only in this request, its staleness is limited to this request as well; the next request starts clean. Code that reads and then writes within one request can read the cache as it was before the write. **DL15.** Here the cache is not invalidated on a write. A mechanism that emptied the cache on a write would have prevented this staleness; no such mechanism was built or measured in this lesson.

#### Summary

- The query interface reads records by id, by condition or by relationship; how many queries run in a request is measured with a counter.
- The query cache, the observation of this lesson, is a value whose stay is this request: the record in the cache was read only in the request that wrote it.
- The same id is not queried a second time in the same request; in a new request the counter and the cache start from zero.
- The N+1 problem is fetching the related record with a separate query for each record; eager loading lowers the queries to two, and the cache only removes repeated ids.
- The cache key includes the type of the id; a `"1"` that arrives without being converted to a type counts as a separate query. In eager loading an id that is not in the table leaves `nil` in the list in this model, and the code that builds the result must handle this separately.
- The cache stores a `nil` result too; a record added later in the same request is not seen, but this staleness is limited to that request.

#### Next Step

The data layer is complete: record, schema, relationship, validation and query count have been measured. How is a user who signs in recognized on the next page? The next topic measures the answer to this question in authentication.

## Application Services

Identity carried in the session and per-action authorization checks, background jobs moved out of the request, fragment caching and stale content, testing with isolated applications, and the security protections the framework turns on by default.

### Authentication and Authorization

Source: https://academia.sh/en/courses/mvc-framework/authentication-and-authorization

Measures how many requests a signed-in user's identity stays for, and why the authorization decision is recomputed on every request.

The earlier lessons showed a value that is born inside a request and leaves with it: the path parameter, the validation error, the query cache and the controller's instance variable. The query cache, despite its name, was in the **this request** stay; the second request started the counter from zero. The flash message (`flash`) passed to the next request only. This lesson measures, for the first time, a value that stays in the session, for every later request of the same client. The question is this: after a user signs in, how does every later page know who that user is?

**Authentication** is the work of establishing who sent the request; its result is an identity. **Authorization** is the work of establishing whether that identity may perform a given action; its result is a yes or a no. The two are often mentioned together, but they behave differently in time, and this lesson measures that difference. The measuring axis is the course's axis: **how many requests does this value stay for?** There are four stays: **this request**, **next request**, **session** and **persistent**. M11/K03 also set persistent against session, but the question there was who can see a value. Here nobody is asked who sees what; the question is time. After the request that writes a value, how many more requests see it? M12/K03 measured who can change a result; here the one who changes it does not matter, only the stay is measured.

#### Core

The measurements in this lesson rest on a small request loop. The file below is the course core and is given as it is. `App` holds the store shared by all requests and all clients; every request receives a new `Context`; and `session` is a hash chosen per client. `stay(write, read)` writes a value in the first request, reads it in two more requests of the same client and in one request of another client, looks at where the value was seen and classifies the stay.

```ruby
# stay.rb — a small request loop and a helper that classifies how many requests a value stays for

# Flash: a value written in this request is carried to the next request only.
class Flash
  def initialize(incoming)
    @incoming = incoming
    @fresh = {}
  end

  def [](key) = @fresh.fetch(key) { @incoming[key] }

  def []=(key, value)
    @fresh[key] = value
  end

  def passed_on = @fresh
end

# Every request gets a new context: params, flash and the request store start empty each time.
class Context
  attr_reader :app, :params, :session, :flash, :request_store

  def initialize(app, session)
    @app = app
    @params = {}
    @session = session
    @flash = Flash.new(session.delete(:flash) || {})
    @request_store = {}
  end
end

# The application store is the same across all requests and all clients;
# the session is chosen per client.
class App
  attr_reader :store

  def initialize
    @sessions = Hash.new { |h, client| h[client] = {} }
    @store = { products: {}, schema: { products: %i[name price] }, templates: {},
               queue: [], cache: {}, counter: 0 }
  end

  def request(client)
    context = Context.new(self, @sessions[client])
    result = yield context
    context.session[:flash] = context.flash.passed_on unless context.flash.passed_on.empty?
    result
  end
end

# Rule: request 1 (client A) writes the value; requests 2 and 3 (A) and request 4 (B) read it.
# If B sees it, persistent; if A's third request sees it, session; if only the second
# request sees it, next; if none sees it, this_request.
def stay(write, read)
  app = App.new
  first = app.request("A") { |b| write.call(b); read.call(b) }
  raise ArgumentError, "value could not be read in the request that wrote it" if first.nil?

  seen = [["A", 2], ["A", 3], ["B", 4]].map do |client, order|
    app.request(client) { |b| read.call(b) }.nil? ? "--" : "#{client}#{order}"
  end
  kind = if seen[2] != "--" then "persistent"
         elsif seen[1] != "--" then "session"
         elsif seen[0] != "--" then "next"
         else "this_request"
         end
  [kind, ["A1", *seen].join(" "), first]
end
```

This block is source; the file is written to disk and pulled in with `require_relative`. The core is not a framework but a model of one: `session` is a hash, and the persistent store lives in process memory. In a real application the session works through a cookie and a store, and the persistent store works through a database or a cache store; the rule reads how the value behaves, not where it is stored. The measurements are meaningful within a single application process.

#### The Stay of Identity

Signing in means writing a user id to the session. Signing out means deleting it. This is why an identity is verified once and kept in the session instead of being asked for on every request: later requests do not receive the password again, they look at the identity in the session. The excerpt below defines the sign-in, sign-out and identity lookup helpers, as well as a permission table.

```ruby
# identity.rb — sign-in, sign-out, identity lookup and the permission derived from identity
require_relative "stay"

PERMISSIONS = { 42 => %i[show edit], 7 => %i[show] }

def sign_in(b, user_id)
  b.session[:user_id] = user_id
end

def sign_out(b) = b.session.delete(:user_id)

def identity(b) = b.session[:user_id]

def allowed?(b, action) = (PERMISSIONS[identity(b)] || []).include?(action)

def report(name, kind, evidence, value)
  printf("%-14s%-14s%-13s%s\n", name, kind, evidence, value.inspect)
end
```

This block is source as well. `PERMISSIONS` is a table from an identity to a list of permitted actions; `allowed?` reads the identity from the session, looks it up in the table and returns a boolean. For someone whose identity is not known at all, `PERMISSIONS[nil]` returns `nil`, `|| []` turns it into an empty list and the result is `false`. Now the first observation: the user id written to the session.

```ruby
# user.rb — how many requests does the user id stay for
require_relative "identity"

write = ->(b) { b.session[:user_id] = 42 }
read = ->(b) { b.session[:user_id] }
report("identity", *stay(write, read))
```
```
identity      session       A1 A2 A3 --  42
```

The identity appeared in all three requests of A and did not appear in the request of B. This is the **session** stay: the value stays for every request of the client that wrote it and does not pass to other clients. That it is not persistent is a deliberate outcome. Had the identity been written to the shared store of the application, B would also have appeared with A's identity. **AS1.** The session staying for A's requests was measured only across three requests; when the session ends, through a timeout or the deletion of the cookie, is not modeled in this core.

#### Every Client Has Its Own Identity

Because the session is chosen per client, two clients can work in the same application with different identities and neither sees the other's. In the measurement below A signs in as user 42 and B as user 7; then each client reads its identity. In the last step B signs out and A's identity is looked at again.

```ruby
# clients.rb — two clients, two identities
require_relative "identity"

app = App.new
app.request("A") { |b| sign_in(b, 42) }
app.request("B") { |b| sign_in(b, 7) }
["A", "B", "A"].each do |client|
  puts "#{client} #{app.request(client) { |b| identity(b) }.inspect}"
end
app.request("B") { |b| sign_out(b) }
puts "after sign-out A #{app.request("A") { |b| identity(b) }.inspect}"
puts "after sign-out B #{app.request("B") { |b| identity(b) }.inspect}"
```
```
A 42
B 7
A 42
after sign-out A 42
after sign-out B nil
```

Each client read only its own identity, and B's sign-out did not touch A's session. This is the practical meaning of the identity being in the session stay: the value is bound to the client, not to the shared store of the application. This separation is one of the foundations that keep the many users connected to the same server process from getting mixed up; the measurement was made with two clients.

#### The Permission Decision Is Recomputed on Every Request

Since the identity stays in the session, does the authorization decision stay as well? The decision is derived from the identity and the derivation is cheap; the lesson treats writing it to the session or not as a choice. First, look at the derivation itself: two users and a client that has not signed in are asked the same two actions.

```ruby
# permissions.rb — one identity, two actions, three clients
require_relative "identity"

app = App.new
app.request("A") { |b| sign_in(b, 42) }
app.request("B") { |b| sign_in(b, 7) }

["A", "B", "C"].each do |client|
  result = app.request(client) { |b| [allowed?(b, :show), allowed?(b, :edit)] }
  puts "#{client} #{result.inspect}"
end
```
```
A [true, true]
B [true, false]
C [false, false]
```

The same two questions gave three different answers, and the difference came only from the identity in the session. C never signed in, so it was refused for both actions. Authorization is a separate step that reads the value authentication produced: without an identity there is no decision, but the decision is not contained in the identity either.

Now measure the stay of the decision. The decision is computed in the request that signs in and is put only in the request store of that request.

```ruby
# decision.rb — how many requests does the permission decision stay for
require_relative "identity"

write = lambda do |b|
  sign_in(b, 42)
  b.request_store[:decision] = allowed?(b, :edit)
end
read = ->(b) { b.request_store[:decision] }
report("access check", *stay(write, read))
```
```
access check  this_request  A1 -- -- --  true
```

The decision is in the **this request** stay. In the next request the request store starts empty and the decision is computed again. The practical consequence is this: if a user's permission changes, the next request runs with the new permission; there is no copy that carries the old decision. Had the decision been written to the session, a change in the permission table would stay invisible until that session ended. **AS2.** Here the decision is a cheap table lookup. Storing a decision whose computation is expensive is a separate design question, and the next two lessons, on jobs and on caching, treat it from the point of view of the stay.

#### If We Write the Decision to the Session

Above, recomputing the authorization decision on every request was treated as a choice. To measure the result of that choice, try the other way: the decision is computed in the request that signs in and is written to the session together with the identity.

```ruby
# session_decision.rb — how many requests does a permission decision written to the session stay for
require_relative "identity"

write = lambda do |b|
  sign_in(b, 42)
  b.session[:decision] = allowed?(b, :edit)
end
read = ->(b) { b.session[:decision] }
report("stored check", *stay(write, read))
```
```
stored check  session       A1 A2 A3 --  true
```

The decision moved to the **session** stay like the identity: it appears in all later requests of the same client. A longer stay does not make a decision correct. To see this, change the permission table after the decision is written and put the two ways side by side: the decision stored in the session and the decision recomputed on every request.

```ruby
# stale_decision.rb — the stored decision and the recomputed decision after the permission table changes
require_relative "identity"

app = App.new
app.request("A") do |b|
  sign_in(b, 42)
  b.session[:decision] = allowed?(b, :edit)
end
PERMISSIONS[42] = %i[show]
stored = app.request("A") { |b| b.session[:decision] }
recomputed = app.request("A") { |b| allowed?(b, :edit) }
puts "stored #{stored}, recomputed #{recomputed}"
```
```
stored true, recomputed false
```

Although the permission was taken back, the decision in the session kept saying the old value; the recomputed decision showed the new situation. Staleness is the loss of the link between a value whose stay has been lengthened and its source. We will meet the same problem for a fragment in the lesson on caching. This is also why the identity is kept in the session and the decision is not: the identity is a fact that does not change until the user signs out (or until the session ends), whereas the decision depends on something else besides the identity, namely the permission table.

#### Not Signed In and an Unrecognized Identity

Because the authorization decision is derived from the identity, there are two cases in which no identity is found: the client has never signed in, or the identity in the session does not appear in the permission table. In both cases `allowed?` has to decide instead of raising an error. The measurement below puts three clients side by side: a client that has not signed in, user 99, who is not in the table, and user 42, who is.

```ruby
# unknown.rb — the permission decision for clients with no identity and for an unrecognized one
require_relative "identity"

app = App.new
app.request("B") { |b| sign_in(b, 99) }
app.request("C") { |b| sign_in(b, 42) }
%w[A B C].each do |client|
  who, verdict = app.request(client) { |b| [identity(b), allowed?(b, :show)] }
  puts "#{client} #{who.inspect} #{verdict}"
end
```
```
A nil false
B 99 false
C 42 true
```

The client that has not signed in has an empty identity and the decision is negative; user 99 has an identity, but since the table has no entry for it the decision is again negative. Identity and permission are two separate questions: the first is "who is this client?", the second "can this person do this?". An identity being present in the session does not mean a permission is present. Because the decision is read from the table, every identity that is not in the table remains unauthorized by default.

#### Signing Out

Signing out means deleting the identity from the session. Our rule writes only in the first request; to measure the sign-out, the `read` lambda keeps its own call count and, in the second request, reads the identity and then deletes it. The first and second requests see the identity; the third and fourth do not.

```ruby
# signout.rb — what happens to the stay if the sign-out comes in the second request
require_relative "identity"

write = ->(b) { sign_in(b, 42) }
calls = 0
read = lambda do |b|
  calls += 1
  value = identity(b)
  sign_out(b) if calls == 2
  value
end
report("sign out", *stay(write, read))
```
```
sign out      next          A1 A2 -- --  42
```

The rule classified this value as **next request**, because only the second request saw it. The same identity, without the sign-out, was in the **session** stay. This is not a contradiction but evidence that the rule reads behavior: the value is stored in the session, yet the sign-out deletes it early and the stay becomes shorter. That is the purpose of a sign-out. **AS3.** Having the sign-out occur in the second request is the arrangement of the measurement; in a real application nobody knows in advance in which request the sign-out will arrive.

#### The Bounding Measurement

The rule of this lesson carries a limit as well. If `stay` cannot read the value even in the request that wrote it, it does not classify; it raises an exception. Measure this with an arrangement in which the writing side writes nothing.

```ruby
# boundary.rb — what does the rule do if no value is written
require_relative "identity"

begin
  stay(->(b) { }, ->(b) { identity(b) })
rescue => e
  puts e.class
end
```
```
ArgumentError
```

The rule raised `ArgumentError`. That is, **a value that cannot be read in the request that wrote it cannot be classified into any stay**; the stay takes at least one appearance of the value as a precondition. The identity of a client that has not signed in therefore carries no stay: it does not exist. All four measurements of the lesson stayed inside this limit. The identity stayed in the session, the decision in this request, the decision written to the session in the session, and after a sign-out the identity in the next request. The limit is also a warning: before measuring the stay of a value, make sure the value really exists in the first request. If a `read` lambda that reads the identity looks at the wrong key, the answer of the rule is not a stay but an error.

#### Summary

- Authentication produces an identity, authorization derives a yes or a no from that identity; the two are separate steps.
- A user id written to the session appears in all requests of the client that wrote it and does not appear for another client; two clients work in the same application with different identities. The stay of this lesson is **session**.
- If the permission decision is recomputed from the identity on every request, it is in the **this request** stay and a permission change appears in the next request; if it is written to the session, its stay extends to the session and it goes stale when the permission changes.
- A client that has not signed in, or an identity that is not in the table, remains unauthorized by default; an identity being present does not show that a permission is present.
- The request that signs out deletes the identity from the session; after the deleting request the identity is gone and the rule sees it as **next request**.
- `stay` does not classify a value that cannot be read in the request that wrote it, and raises `ArgumentError`.

#### Next Step

The identity stays in the session and the decision is produced again on every request; but where does a piece of work that must run after the request has finished wait? The next lesson measures background jobs: how many requests a job placed in a queue stays for, and why that stay differs from the session.

### Background Jobs

Source: https://academia.sh/en/courses/mvc-framework/background-jobs

Measures how many requests the record of a queued job stays for, what happens when the worker drains it, and why the result of the job is not in the same stay.

The previous lesson measured that the identity stays in the session and that the permission decision is produced again on every request. Both values were computed inside a request, while that request waited for its response. But not every piece of work an application does has to wait for the response. Sending an invoice, shrinking an image or preparing a report can be done after the user has received the response. The question of this lesson is this: where does a job that will run after the request is over wait until then, and for how many requests does that wait last?

A **background job** is the name given to work that is left for later so that the response of a request is not delayed. Leaving work for later means turning it into a record to be done: a record made of a job name and arguments is put into a **queue**. A **worker** then drains the queue later and runs the job in the record. M12/K04 also showed a part that did not run in this request but was deferred; what was measured there was whether code ran. Here we leave the code and measure the value: even if it does not run in this request, for how many requests does the record in the queue live? The measuring axis is the course's axis: **how many requests does this value stay for?** Recall the four stays: **this request**, **next request**, **session** and **persistent**.

#### Core

The measurements again rest on the same small request loop. The file is the course core and is given as it is. In this core the queue is an array inside `App#store`; in a real application the queue lives in an external store, and the core does not model it, it only carries its behavior.

```ruby
# stay.rb — a small request loop and a helper that classifies how many requests a value stays for

# Flash: a value written in this request is carried to the next request only.
class Flash
  def initialize(incoming)
    @incoming = incoming
    @fresh = {}
  end

  def [](key) = @fresh.fetch(key) { @incoming[key] }

  def []=(key, value)
    @fresh[key] = value
  end

  def passed_on = @fresh
end

# Every request gets a new context: params, flash and the request store start empty each time.
class Context
  attr_reader :app, :params, :session, :flash, :request_store

  def initialize(app, session)
    @app = app
    @params = {}
    @session = session
    @flash = Flash.new(session.delete(:flash) || {})
    @request_store = {}
  end
end

# The application store is the same across all requests and all clients;
# the session is chosen per client.
class App
  attr_reader :store

  def initialize
    @sessions = Hash.new { |h, client| h[client] = {} }
    @store = { products: {}, schema: { products: %i[name price] }, templates: {},
               queue: [], cache: {}, counter: 0 }
  end

  def request(client)
    context = Context.new(self, @sessions[client])
    result = yield context
    context.session[:flash] = context.flash.passed_on unless context.flash.passed_on.empty?
    result
  end
end

# Rule: request 1 (client A) writes the value; requests 2 and 3 (A) and request 4 (B) read it.
# If B sees it, persistent; if A's third request sees it, session; if only the second
# request sees it, next; if none sees it, this_request.
def stay(write, read)
  app = App.new
  first = app.request("A") { |b| write.call(b); read.call(b) }
  raise ArgumentError, "value could not be read in the request that wrote it" if first.nil?

  seen = [["A", 2], ["A", 3], ["B", 4]].map do |client, order|
    app.request(client) { |b| read.call(b) }.nil? ? "--" : "#{client}#{order}"
  end
  kind = if seen[2] != "--" then "persistent"
         elsif seen[1] != "--" then "session"
         elsif seen[0] != "--" then "next"
         else "this_request"
         end
  [kind, ["A1", *seen].join(" "), first]
end
```

This block is source; the file is written to disk and pulled in with `require_relative`. The core is not a framework but a model of one. The rule reads how the value behaves, not where it is stored; the measurements are meaningful within a single application process.

#### Queue, Job and Worker

A job is a name and a list of arguments. Enqueuing appends this pair to the end of the queue array; the worker reads from the front, finds the code to run from the name of the job and calls it with the arguments. The excerpt below defines these three parts. The job table `JOBS` is a hash that turns a name into code.

```ruby
# jobs.rb — enqueue, job table and worker
require_relative "stay"

def enqueue(b, job_name, *args) = b.app.store[:queue] << [job_name, args]

JOBS = { "send_invoice" => ->(invoice_id) { "invoice #{invoice_id} sent" } }

def run_worker(b)
  queue = b.app.store[:queue]
  results = []
  until queue.empty?
    job_name, args = queue.shift
    results << JOBS.fetch(job_name).call(*args)
  end
  results
end

def report(name, kind, evidence, value)
  printf("%-14s%-14s%-13s%s\n", name, kind, evidence, value.inspect)
end
```

This block is source as well. `run_worker` takes the record at the front until the queue is empty, looks it up in the `JOBS` table and collects the results in an array. First, see that we run the job outside the request, not inside it. A request only leaves the record and gives its response; the record waits in the queue until the worker, which runs in a separate call.

```ruby
# deferral.rb — the request only leaves the record, the worker runs the job
require_relative "jobs"

app = App.new
response = app.request("A") { |b| enqueue(b, "send_invoice", 1); "response ready" }
p response
p app.store[:queue].size
p app.request("worker") { |b| run_worker(b) }
p app.store[:queue].size
```
```
"response ready"
1
["invoice 1 sent"]
0
```

The request gave its response and one record remained in the queue. The worker took the record out in a separate call and ran the job; the queue was empty again. This gap between the response and the job is the basic reason for a background job. **AS4.** In the core the worker is a method called by hand after the request. A real worker is often a separate process, and when it will run is not decided by the measurement; this timing is not modeled here.

#### The Stay of a Record in the Queue

The first observation is the record of the enqueued job. The `write` lambda leaves the record, and the `read` lambda returns the first record in the queue.

```ruby
# queue.rb — how many requests does the queued job stay for
require_relative "jobs"

write = ->(b) { enqueue(b, "send_invoice", 1) }
read = ->(b) { b.app.store[:queue].first }
report("queued job", *stay(write, read))
```
```
queued job    persistent    A1 A2 A3 B4  ["send_invoice", [1]]
```

The record appeared in all four requests, including the request of B. This is the **persistent** stay. The difference from the session is clear: the record is not bound to a client, it is in the shared store of the application. The invoice job that one client's request enqueued is also seen by the process that handles the request of another client; that is the very task of a worker. In M12/K04 the record was in the "deferred" class, that is, it did not run in this request. Here that deferral is the same thing as the record going beyond the request: a record that does not run waits in the shared store of the application until the worker arrives.

The order of the queue is also part of this stay. Take two jobs in turn and look at the queue.

```ruby
# order.rb — two jobs stay in the order they entered the queue
require_relative "jobs"

app = App.new
app.request("A") do |b|
  enqueue(b, "x")
  enqueue(b, "y", 2)
end
p app.store[:queue]
```
```
[["x", []], ["y", [2]]]
```

A job without arguments carries an empty list, and the job `y` carries the value `2`. The order was preserved: the worker takes `x` first and `y` after it. **AS5.** It was measured with only two jobs; anything that could break the order with many jobs, for example several workers reading the queue at the same time, is not in this core.

#### A Record Runs Once

The queue holds a record; the worker takes it out. Taking it out deletes the record from the queue. This has a consequence: calling the same worker a second time does not run the same job again, because no record is left to run. To measure it, a job is enqueued and the worker is called in two requests in a row.

```ruby
# repeat.rb — how many times does the job run if the worker is called twice
require_relative "jobs"

app = App.new
app.request("A") { |b| enqueue(b, "send_invoice", 1) }
p app.request("worker1") { |b| run_worker(b) }
p app.request("worker2") { |b| run_worker(b) }
p app.store[:queue].size
```
```
["invoice 1 sent"]
[]
0
```

The first call ran the job, the second returned an empty list. The job running once is the direct consequence of the record leaving the queue; no separate "this job was done" mark was needed for it. This ties the stay of the job to its presence in the queue: the moment the record leaves the queue, the job no longer exists for this core. Code that wants to learn later that a job ran cannot find this information in the queue. This is the result of this single-worker core; the delivery guarantees of real queue systems are examined separately.

#### The Stay of an Argument Placed in the Queue

The argument of an enqueued job is persistent together with the record. But does the record hold a copy of the argument, or the argument itself? The question matters, because the code that makes the request can change the argument after putting it in the queue. The measurement goes like this: a list is given to the queue as an argument, then the same list is changed and the record in the queue is looked at.

```ruby
# argument.rb — what does the record see if the queued argument is changed afterwards
require_relative "jobs"

app = App.new
list = [1]
app.request("A") { |b| enqueue(b, "send_invoice", list) }
list << 2
p app.store[:queue]
p app.store[:queue].first[1].first.equal?(list)
```
```
[["send_invoice", [[1, 2]]]]
true
```

The record held the list itself, not a copy of it. A change made after the request ended reached the job in the queue as well. Therefore the argument a job sees when it runs may not be the value it had when it was enqueued. In terms of the stay, this means: the record is persistent, but the object inside the record is not a persistent copy but a shared object, and every place that holds it sees its changes. In this core, code that wants to fix the argument of a job copies the value before enqueuing it.

#### An Unknown Job Name

The worker looks up the code to run from the name of the job in the `JOBS` table. A name that is not in the table can enter the queue; the queue does not check names. The error appears when the worker reaches that record. The measurement uses two records: the first a name that is not in the table, the second a valid job. Only the class of the exception is printed.

```ruby
# unknown_job.rb — how a job name that is not in the table stops the worker
require_relative "jobs"

app = App.new
app.request("A") do |b|
  enqueue(b, "unknown")
  enqueue(b, "send_invoice", 1)
end
begin
  app.request("worker") { |b| run_worker(b) }
rescue => e
  puts e.class
end
p app.store[:queue]
```
```
KeyError
[["send_invoice", [1]]]
```

The worker stopped at the first record. The record had been taken out of the queue before the error, so the faulty record did not stay in the queue; the second record is still in place because it was never read. In this core a faulty job loses its own record and delays the jobs after it. This core has no retry mechanism; nothing was written to bring the lost record back. In terms of the stay, this is a limit of the persistence of the record: the record is persistent while it is in the queue, but its presence in the queue ends the moment it passes into the worker's hands. A job leaving the queue does not mean the job has finished; the queue knows only the moment of removal, not whether the job finished successfully. A core that wants to retry on error has to build a separate record scheme that waits for the end of the job instead of deleting the record from the queue before running it; in real queue systems this mechanism can be configured.

#### When the Worker Drains the Queue

A persistent stay is not endless. A worker that drains the record makes the stay shorter. To measure this, the `read` lambda keeps the call count: in the second request, after looking at the record, it runs the worker and drains the queue.

```ruby
# drain.rb — what happens to the stay if the worker drains the queue in the second request
require_relative "jobs"

write = ->(b) { enqueue(b, "send_invoice", 1) }
calls = 0
read = lambda do |b|
  calls += 1
  record = b.app.store[:queue].first
  run_worker(b) if calls == 2
  record
end
report("drained", *stay(write, read))
```
```
drained       next          A1 A2 -- --  ["send_invoice", [1]]
```

The rule classified this as **next request**: the record appeared in the first and second requests, and after the worker ran it was gone in the third and fourth. The same record without a worker was **persistent**. What determines the stay is not the place where it is stored but the behavior: if a process drains the queue, the record stays until the process that handles it arrives. **AS6.** The draining was arranged in the second request. In reality, the request in which the worker runs decides how many requests can see the record, and that number depends on the length of the queue and the speed of the worker.

#### Storing the Result in the Worker's Request

In the bounding measurement below the result stays in the request store. But does the stay change if we write the result to the shared store of the application inside the worker's request? The measurement uses the same enqueuing and worker call as before; the only difference is where the result is written.

```ruby
# result_store.rb — how writing the result to the shared store changes the stay
require_relative "jobs"

write = lambda do |b|
  enqueue(b, "send_invoice", 1)
  b.app.store[:result] = run_worker(b).first
end
read = ->(b) { b.app.store[:result] }
report("stored result", *stay(write, read))
```
```
stored result persistent    A1 A2 A3 B4  "invoice 1 sent"
```

This time the result is in the **persistent** stay: it appeared in all four requests. Only the place it is written changed. How many requests the result stays for is decided not by the worker but by the code that writes the result. This is the direct counterpart of the result in the next section: the queue does not carry the "what happened" information, and code that wants to carry it writes it explicitly to the shared store of the application.

#### The Bounding Measurement

So far, what was persistent was the **record** of the job, that is, its name and arguments. But where is the **result** of the job? The lambda in the `JOBS` table returns a string, but `run_worker` only collects that string in an array. Measure the result with the same stay rule: the job is enqueued and run by the worker in the same request, and the result is put into the request store of that request.

```ruby
# result.rb — how many requests does the result of the job stay for
require_relative "jobs"

write = lambda do |b|
  enqueue(b, "send_invoice", 1)
  b.request_store[:result] = run_worker(b).first
end
read = ->(b) { b.request_store[:result] }
report("job result", *stay(write, read))
```
```
job result    this_request  A1 -- -- --  "invoice 1 sent"
```

The result is in the **this request** stay. The argument of the job was persistent, its result is not. The reason is simple: the worker returns the result, but writing the returned value somewhere is the job's own task. If the result is to be visible in later requests, the job has to store it itself. In this core the queue carries only the "what is to be done" information, not the "what happened" information. This distinction can lead to a mistaken assumption: throwing a job into the queue and expecting to read its result from the queue later means looking for a value the queue does not carry.

#### Summary

- A background job is a record made of the name and arguments of work left for later so that the response is not delayed; the queue holds the record, the worker drains it and runs it.
- The record of an enqueued job is in the shared store of the application, independent of the client; the stay of this lesson is **persistent**.
- The queue holds jobs in the order they entered. When the worker runs, the record leaves the queue and runs once; a second worker call returns empty. If the worker drains the record in the second request, the rule sees it as **next request**.
- The queue holds the argument itself, not a copy; a later change reaches the job. An unknown job name stops the worker with `KeyError`, and the faulty record has already left the queue.
- The argument of a job is persistent, its result is not; storing the result is the job's own work.
- The result depends on where it is written: if written to the shared store it stays **persistent**, if written to the request store it stays **this request**.

#### Next Step

The queue makes a record persistent in order to defer a job. What happens if we want to make a result persistent instead, so that the same computation is not repeated on every request? The next lesson measures caching: a fragment being produced once and served to later requests, and how the resulting staleness is managed.

### Caching

Source: https://academia.sh/en/courses/mvc-framework/caching

Measures a fragment being produced once and served to later requests, how it differs from the query cache, and how content becomes stale when the key does not change.

The previous lesson measured that the record of an enqueued job is persistent; the record waited until the worker that processes the job arrived. In this lesson what is made persistent is not a record but the result of a computation. Instead of redrawing one row of a product list on every request, it is possible to draw it once, store the result and serve it to later requests. The question is this: until which request is the stored result valid, and when does it stop being valid?

A **cache** is a store that keeps the result of an expensive computation so that the computation is not repeated when it is requested again. A **fragment cache** stores the drawn output of one part of a view, for example a single list row. The **cache key** decides which result belongs to which computation: a second request that arrives with the same key does not perform the computation, it takes the stored result. M11/K03 showed that a cache is a shared store; here we do not ask who sees it, we ask how long it stays. The measuring axis is the course's axis: **how many requests does this value stay for?** The four stays: **this request**, **next request**, **session** and **persistent**.

#### Core

The measurements rest on the same small request loop. The file is the course core and is given as it is. The cache in this core is a hash inside `App#store`; in a real application the cache is a separate store, and the core carries its behavior, not the cache itself.

```ruby
# stay.rb — a small request loop and a helper that classifies how many requests a value stays for

# Flash: a value written in this request is carried to the next request only.
class Flash
  def initialize(incoming)
    @incoming = incoming
    @fresh = {}
  end

  def [](key) = @fresh.fetch(key) { @incoming[key] }

  def []=(key, value)
    @fresh[key] = value
  end

  def passed_on = @fresh
end

# Every request gets a new context: params, flash and the request store start empty each time.
class Context
  attr_reader :app, :params, :session, :flash, :request_store

  def initialize(app, session)
    @app = app
    @params = {}
    @session = session
    @flash = Flash.new(session.delete(:flash) || {})
    @request_store = {}
  end
end

# The application store is the same across all requests and all clients;
# the session is chosen per client.
class App
  attr_reader :store

  def initialize
    @sessions = Hash.new { |h, client| h[client] = {} }
    @store = { products: {}, schema: { products: %i[name price] }, templates: {},
               queue: [], cache: {}, counter: 0 }
  end

  def request(client)
    context = Context.new(self, @sessions[client])
    result = yield context
    context.session[:flash] = context.flash.passed_on unless context.flash.passed_on.empty?
    result
  end
end

# Rule: request 1 (client A) writes the value; requests 2 and 3 (A) and request 4 (B) read it.
# If B sees it, persistent; if A's third request sees it, session; if only the second
# request sees it, next; if none sees it, this_request.
def stay(write, read)
  app = App.new
  first = app.request("A") { |b| write.call(b); read.call(b) }
  raise ArgumentError, "value could not be read in the request that wrote it" if first.nil?

  seen = [["A", 2], ["A", 3], ["B", 4]].map do |client, order|
    app.request(client) { |b| read.call(b) }.nil? ? "--" : "#{client}#{order}"
  end
  kind = if seen[2] != "--" then "persistent"
         elsif seen[1] != "--" then "session"
         elsif seen[0] != "--" then "next"
         else "this_request"
         end
  [kind, ["A1", *seen].join(" "), first]
end
```

This block is source; the file is written to disk and pulled in with `require_relative`. The core is not a framework but a model of one. The rule reads how the value behaves, not where it is stored; the measurements are meaningful within a single application process.

#### The Fragment Cache

The caching helper is a single line: if the key is in the cache it returns the stored value; if not, it runs the block, puts the result under the key and returns it. The excerpt of the lesson gives it together with two small helpers to be used in the measurements. The `Product` module is a short copy of the one from an earlier lesson and is needed for the comparison with the query cache.

```ruby
# cache.rb — the cached helper, a fragment-producing helper and Product for comparison
require_relative "stay"

def cached(b, key) = b.app.store[:cache][key] ||= yield

def escape(text) = text.to_s.gsub("&", "&amp;").gsub("<", "&lt;").gsub(">", "&gt;").gsub('"', "&quot;")

def fragment(b, product)
  cached(b, "product/#{product[:id]}/#{product[:version]}") { "<li>#{escape(product[:name])}</li>" }
end

module Product
  def self.insert(b, attributes)
    errors = []
    errors << "name cannot be blank" if attributes[:name].to_s.empty?
    errors << "price cannot be negative" if attributes[:price].to_i.negative?
    return { errors: } unless errors.empty?

    table = b.app.store[:products]
    id = table.size + 1
    table[id] = attributes.merge(id:)
    { id:, errors: }
  end

  def self.lookup(b, id)
    queries = (b.request_store[:queries] ||= {})
    queries.fetch([:products, id]) do
      b.request_store[:query_count] = b.request_store.fetch(:query_count, 0) + 1
      queries[[:products, id]] = b.app.store[:products][id]
    end
  end
end

def report(name, kind, evidence, value)
  printf("%-14s%-14s%-13s%s\n", name, kind, evidence, value.inspect)
end
```

This block is source as well. `cached` writes the key with `||=`: if the value is `nil` or `false` it is computed again, but since the fragment outputs are strings this causes no problem in these measurements. The first observation is the fragment written to the cache itself.

```ruby
# fragment.rb — how many requests does the fragment cache stay for
require_relative "cache"

write = ->(b) { cached(b, "product/1") { "<li>pen</li>" } }
read = ->(b) { b.app.store[:cache]["product/1"] }
report("fragment", *stay(write, read))
```
```
fragment      persistent    A1 A2 A3 B4  "<li>pen</li>"
```

The fragment appeared in all four of the four requests, including the request of B. This is the **persistent** stay, and it is the purpose of a cache: the fragment that one client produced is also used by the request of another client. The model in the core shows this plainly. **AS7.** In the core the fragment is in process memory and has no lifetime at all. A real cache can give entries a lifetime or discard the old ones when it fills up; this measurement does not model them.

#### The Computation Runs Once

The gain of a cache is that the computation is not repeated. This can be seen by counting: three calls with the same key in one request, and one call in another client; how many times does the block run?

```ruby
# count.rb — how many times is the block entered with the same key
require_relative "cache"

runs = 0
app = App.new
app.request("A") do |b|
  3.times { cached(b, "product/1") { runs += 1; "<li>pen</li>" } }
end
app.request("B") { |b| cached(b, "product/1") { runs += 1; "<li>pen</li>" } }
puts "same key: #{runs}"
app.request("A") { |b| cached(b, "product/2") { runs += 1; "<li>eraser</li>" } }
puts "second key: #{runs}"
```
```
same key: 1
second key: 2
```

In spite of four calls the block ran once; a different key ran the block once more. The key is the identity of the computation: if two calls use the same key, the same computation is considered to have been done for them.

#### Comparison with the Query Cache

An earlier lesson measured the query cache. The same query in the same request did not go to the database a second time, but the next request started the counter from zero. The same rule applies to the two caches together.

```ruby
# comparison.rb — the query cache and the fragment cache side by side
require_relative "cache"

pen = ->(b) { Product.insert(b, name: "pen", price: 12) }
write = ->(b) { pen.call(b); Product.lookup(b, 1) }
read = ->(b) { b.request_store.dig(:queries, [:products, 1], :name) }
report("query", *stay(write, read))
write = ->(b) { cached(b, "product/1") { "<li>pen</li>" } }
read = ->(b) { b.app.store[:cache]["product/1"] }
report("fragment", *stay(write, read))
```
```
query         this_request  A1 -- -- --  "pen"
fragment      persistent    A1 A2 A3 B4  "<li>pen</li>"
```

Both carry the name of a cache, but their stays are opposite. The query cache is in the **this request** stay: it lives in the request store and goes at the end of the request. The fragment cache is in the **persistent** stay: it lives in the store of the application. The difference is that the repetition of the query is a repetition within the same request, while the repetition of the fragment is a repetition across requests. The limit of the query cache is one request; therefore its staleness (the state of lagging behind the source) is also limited to that request. The limit of the fragment cache is not one request; therefore going stale is its real problem. We have come to that problem. Because these two caches were measured with the same rule, both are part of the same core, but one follows the request and the other follows the application.

#### Invalidation

If the fragment is persistent, what happens when the product changes? The cache stores the fragment independently of its source; it does not know that the source changed. Managing a cache therefore includes replacing an entry that has gone out of date with a new one. Treating an entry as outdated and discarding it is called **invalidation**. One way is to delete the entry. The other way is to change the key: the old entry is not invalidated, but the new key never reaches it. Our `fragment` helper uses the second way; it adds the version of the product to the key. When the product changes the version increases and the key is different.

```ruby
# invalidation.rb — invalidation with a versioned key
require_relative "cache"

app = App.new
product = { id: 1, name: "pen", version: 1 }
p app.request("A") { |b| fragment(b, product) }
product = product.merge(name: "eraser", version: 2)
p app.request("A") { |b| fragment(b, product) }
p app.store[:cache].keys
```
```
"<li>pen</li>"
"<li>eraser</li>"
["product/1/1", "product/1/2"]
```

When the product changed the output changed too. But when we look at the keys of the cache, the old entry is still there: a versioned key does not invalidate the old entry, it only never visits it again. The old entry fills the cache although it is never used; this cost is paid by a lifetime or a space-reclaiming policy. **AS8.** Deleting the old entry was not shown here; at which moment and by whom the deletion is done depends on the configuration of the cache.

#### Invalidating by Deleting the Entry

The old entry left behind by a versioned key shows why the first way needs a separate measurement: deleting the entry directly. In the measurement the same key is used twice; in between the entry is deleted from the cache. The block increments a counter every time it runs.

```ruby
# delete.rb — does deleting the entry make the block run again
require_relative "cache"

runs = 0
app = App.new
2.times { app.request("A") { |b| cached(b, "k") { runs += 1; "old" } } }
puts "before delete: #{runs}"
app.request("A") { |b| b.app.store[:cache].delete("k") }
app.request("A") { |b| cached(b, "k") { runs += 1; "new" } }
puts "after delete: #{runs}"
p app.store[:cache]
```
```
before delete: 1
after delete: 2
{"k" => "new"}
```

Before the delete the block ran once. After the entry was deleted, the next call ran the block again and the cache now holds the new value. Deleting does the opposite of a versioned key: the old entry goes, the key stays the same. On this way the cache does not swell, but code that does the deleting has to run when the source changes; if it does not, the entry stays stale.

#### `cached` Does Not Store a `nil` Result

The `cached` helper writes the value with `||=`. In Ruby `nil` and `false` count as false, so these two values do not stay in the cache. If the result of a computation is `nil` in the sense of "not found", the computation is redone on every call. Make two calls with the same key and look at how many times the block runs.

```ruby
# empty.rb — does a computation that returns nil enter the cache
require_relative "cache"

runs = 0
app = App.new
2.times { app.request("A") { |b| cached(b, "missing") { runs += 1; nil } } }
puts "nil: #{runs}"
2.times { app.request("A") { |b| cached(b, "present") { runs += 1; "x" } } }
puts "filled: #{runs}"
```
```
nil: 2
filled: 3
```

The block that returns `nil` ran in both of the two calls; the block that returns a filled value ran only once. The gain of the helper exists only for the values that enter the cache. An expensive computation can also arrive at a "none" result, and that result is recomputed every time. Code that wants to prevent this writes the empty result to the cache as a separate value, for example an empty string.

#### The Key Must Say Whose Value It Is

The fragment cache is persistent and shared across clients. This causes a problem if the fragment is specific to one client. If nothing in the key separates the clients, the second client receives the fragment of the first. In the measurement two clients write different user numbers into their sessions and ask for a greeting fragment with the same key.

```ruby
# shared.rb — a client-specific fragment stored under a common key
require_relative "cache"

app = App.new
app.request("A") { |b| b.session[:user_id] = 42 }
app.request("B") { |b| b.session[:user_id] = 7 }
%w[A B].each do |client|
  fragment = app.request(client) do |b|
    cached(b, "greeting") { "<p>user #{b.session[:user_id]}</p>" }
  end
  puts "#{client} #{fragment}"
end
```
```
A <p>user 42</p>
B <p>user 42</p>
```

B saw the fragment of user 42 although B is user 7. The request that produced the fragment was A's; when B's request found the same key it did not enter the block. This is the cost of the persistent stay: the value written to the cache belongs not to the request that wrote it but to the application. Adding the user number to the key solves the problem, but then the sharing gain of the fragment is divided into as many fragments as there are users. Deciding which fragment is common to everyone and which is specific to a user is the real work of designing the key. A key written without this decision can, as seen in the measurement, carry the data of one user into the response of another, and the error shows up not as an exception but only as a wrong output.

#### The Bounding Measurement

If the key does not change, that is, if no version is added, what happens when the source changes? Make two calls with the same key; in the second the block produces a different value.

```ruby
# stale.rb — content goes stale if the key stays the same
require_relative "cache"

app = App.new
p app.request("A") { |b| cached(b, "k") { "old" } }
p app.request("A") { |b| cached(b, "k") { "new" } }
```
```
"old"
"old"
```

The block of the second call never ran. The cache saw the key, returned the stored value and remained unaware that the source is now `new`. This is called **stale content**. The rule measures only the stay; whether the value is still correct is not its question, and this measurement shows why the two questions are separate. The persistent stay of the fragment cache is an advantage: the computation is shared across requests. The same stay is a cost when invalidation is not performed: the stale content is served to all clients, in the same form for everyone. For every fragment that is cached, two questions must be asked together: how expensive is this computation, and how will the cache learn that its source has changed?

#### Summary

- A cache stores the result of an expensive computation; a fragment cache stores a single part of a view, and its key is the identity of the computation.
- The fragment cache is in the shared store of the application; the stay of this lesson is **persistent**, and the request of another client uses the same fragment.
- In four calls with the same key the block runs once; a different key runs the block again.
- The query cache is in the **this request** stay and the fragment cache in the **persistent** stay; the same name hides two separate stays.
- As long as the key does not change, the content goes stale; adding a version to the key stops visits to the stale entry but leaves the old entry in the cache, while deleting the entry does not swell the cache but requires the deleting code to run.
- `cached` does not store a `nil` or `false` result; a user-specific fragment stored under a common key becomes visible to another user.

#### Next Step

To use a persistent fragment without it going stale, one first has to know that the fragment works correctly. The next lesson measures testing: reading the instance variable of a controller from outside, and an isolation setup in which every test builds its own application.

### Testing

Source: https://academia.sh/en/courses/mvc-framework/testing

Measures reading a controller's instance variable from outside, the difference between a unit test and a feature test, and the effect on isolation of each test building its own application.

The previous lesson measured the advantage of the persistent stay of a fragment and the cost of its going stale. To trust a persistent value, one has to be sure that the code producing that value works correctly, and the way to check this is a test. A test also depends on the stay of a value: how far does the state a test sets up reach by the end of the test? If a record left by one test mixes into the starting state of the next, the tests are not independent of each other and their results change with their order.

In this lesson the measuring axis stays the same: **how many requests does this value stay for?** The four stays: **this request**, **next request**, **session**, **persistent**. The instance variable of a controller belongs to one request; the store of an application belongs to all requests. The isolation of a test depends on whether these two stays mix between tests.

#### Core

The measurements rest on the same small request loop. The file is the course core and is given as it is.

```ruby
# stay.rb — a small request loop and a helper that classifies how many requests a value stays for

# Flash: a value written in this request is carried to the next request only.
class Flash
  def initialize(incoming)
    @incoming = incoming
    @fresh = {}
  end

  def [](key) = @fresh.fetch(key) { @incoming[key] }

  def []=(key, value)
    @fresh[key] = value
  end

  def passed_on = @fresh
end

# Every request gets a new context: params, flash and the request store start empty each time.
class Context
  attr_reader :app, :params, :session, :flash, :request_store

  def initialize(app, session)
    @app = app
    @params = {}
    @session = session
    @flash = Flash.new(session.delete(:flash) || {})
    @request_store = {}
  end
end

# The application store is the same across all requests and all clients;
# the session is chosen per client.
class App
  attr_reader :store

  def initialize
    @sessions = Hash.new { |h, client| h[client] = {} }
    @store = { products: {}, schema: { products: %i[name price] }, templates: {},
               queue: [], cache: {}, counter: 0 }
  end

  def request(client)
    context = Context.new(self, @sessions[client])
    result = yield context
    context.session[:flash] = context.flash.passed_on unless context.flash.passed_on.empty?
    result
  end
end

# Rule: request 1 (client A) writes the value; requests 2 and 3 (A) and request 4 (B) read it.
# If B sees it, persistent; if A's third request sees it, session; if only the second
# request sees it, next; if none sees it, this_request.
def stay(write, read)
  app = App.new
  first = app.request("A") { |b| write.call(b); read.call(b) }
  raise ArgumentError, "value could not be read in the request that wrote it" if first.nil?

  seen = [["A", 2], ["A", 3], ["B", 4]].map do |client, order|
    app.request(client) { |b| read.call(b) }.nil? ? "--" : "#{client}#{order}"
  end
  kind = if seen[2] != "--" then "persistent"
         elsif seen[1] != "--" then "session"
         elsif seen[0] != "--" then "next"
         else "this_request"
         end
  [kind, ["A1", *seen].join(" "), first]
end
```

This block is source; the file is written to disk and pulled in with `require_relative`. The core is not a framework but a model of one. It carries one difference: `Product.lookup` in the core returns the stored object itself, while in many mappers every read builds a new object from the row. The measurements in this lesson do not depend on the observable results of this difference.

#### Reading the Controller from Outside

When a controller runs an action it leaves what it did in two places: in the value it returns and in its instance variables. The second is the state carried to the view, and a **unit test** can read it directly. A unit test examines a single part, here a single controller, separated from the other parts. In Ruby an instance variable is read from outside with `instance_variable_get`. The excerpt gathers `Product` and `ProductsController`, familiar from earlier lessons, and the `controller` helper in a single file.

```ruby
# testing.rb — Product, ProductsController and the controller helper
require_relative "stay"

module Product
  def self.insert(b, attributes)
    errors = []
    errors << "name cannot be blank" if attributes[:name].to_s.empty?
    errors << "price cannot be negative" if attributes[:price].to_i.negative?
    return { errors: } unless errors.empty?

    table = b.app.store[:products]
    id = table.size + 1
    table[id] = attributes.merge(id:)
    { id:, errors: }
  end

  def self.lookup(b, id)
    queries = (b.request_store[:queries] ||= {})
    queries.fetch([:products, id]) do
      b.request_store[:query_count] = b.request_store.fetch(:query_count, 0) + 1
      queries[[:products, id]] = b.app.store[:products][id]
    end
  end
end

class ProductsController
  def self.before(*names) = (@filters ||= []).concat(names)
  def self.filters = @filters || []

  before :load_product

  def initialize(context)
    @context = context
  end

  def run(action)
    self.class.filters.each { |filter| send(filter) }
    public_send(action)
  end

  def show = @product

  private

  def load_product
    @product = Product.lookup(@context, @context.params[:id])
  end
end

def controller(b) = b.request_store[:controller] ||= ProductsController.new(b)

def report(name, kind, evidence, value)
  printf("%-14s%-14s%-13s%s\n", name, kind, evidence, value.inspect)
end
```

This block is source as well. The first measurement reads the instance variable of the controller after the action and classifies its stay.

```ruby
# variable.rb — how many requests does the controller variable stay for
require_relative "testing"

pen = ->(b) { Product.insert(b, name: "pen", price: 12) }
write = ->(b) { pen.call(b); b.params[:id] = 1; controller(b).run(:show) }
read = ->(b) { controller(b).instance_variable_get(:@product)&.fetch(:name) }
report("controller", *stay(write, read))
```
```
controller    this_request  A1 -- -- --  "pen"
```

The instance variable was seen only in the request in which the action ran. In the next request `controller(b)` builds a new controller and its `@product` is empty. This is the **this request** stay: the controller is built from the request store and goes away with the request. This is also a convenience for a test, because the state a test reads cannot have come from the previous test. The record is different: `Product.insert` writes to the store of the application, and that store outlives requests. Separating the two stays is the first condition of isolation. **AS9.** Here we built the controller directly and called it with `run`; no routing or middleware came into play, so the measurement is a unit test.

#### A Small Verification Helper

It is possible to write a test without needing a test tool: it is enough to compare the expected and the actual value and raise an exception on a mismatch. The lesson does not introduce a test tool; it only shows the smallest form of a verification and measures what happens when it fails.

```ruby
# verification.rb — the smallest verification helper
require_relative "testing"

class AssertionError < StandardError; end

def verify(expected, actual)
  raise AssertionError, "expected #{expected.inspect}, got #{actual.inspect}" unless expected == actual

  :passed
end
```

This block is source as well. The helper is called twice in a request that creates a product; in the second call the expected value is given wrongly.

```ruby
# failure.rb — a passing and a failing verification
require_relative "verification"

app = App.new
result = app.request("A") do |b|
  Product.insert(b, name: "pen", price: 12)
  b.params[:id] = 1
  controller(b).run(:show)
end
p verify("pen", result[:name])
begin
  verify("eraser", result[:name])
rescue AssertionError => e
  puts "#{e.class}: #{e.message}"
end
```
```
:passed
AssertionError: expected "eraser", got "pen"
```

The first verification passed. The second raised an exception because the expected value differed from the actual value, and the message of the exception wrote both values. This is basically what a test tool does: report the difference as an error and stop the rest of the test with that error. The rest, that is, finding the tests, running them in order and summarizing the results, is added on top of this core.

#### Unit Test and Feature Test

A unit test examines a single part. A **feature test** examines the path a user would follow from beginning to end: a request arrives, is processed, and a response returns. The difference between the two is scope, and as the scope grows, finding the cause of a failure becomes harder. The same behavior is measured at two scopes: a product is created and found. In the unit, `Product` and the controller are called directly. In the feature, there is a single call that carries the path passing through a request.

```ruby
# scope.rb — a unit test and a feature test examine the same behavior at different scopes
require_relative "verification"

# unit: only Product
app = App.new
app.request("A") do |b|
  Product.insert(b, name: "pen", price: 12)
  p verify("pen", Product.lookup(b, 1)[:name])
end

# feature: the path through a request, controller included
app = App.new
app.request("A") { |b| Product.insert(b, name: "pen", price: 12) }
response = app.request("B") do |b|
  b.params[:id] = 1
  controller(b).run(:show)
end
p verify("pen", response[:name])
```
```
:passed
:passed
```

Both tests passed, but one only called `Product`; the other followed the path up to the request of another client. A feature test verifies more things because it examines the combination of one part with another, and when it breaks it does not show what broke. A unit test verifies fewer things and when it breaks its place is clear. The two do not replace each other.

#### A Failing Test Does Not Undo Its Writes

When a verification fails, the exception stops the rest of the test. But is the write the test has made up to that moment stopped, or undone? Measure it in a test that shares the application: the test first creates a product, then fails with a wrong expectation. After that the store of the application is looked at.

```ruby
# residue.rb — does a failing test undo the product it wrote
require_relative "verification"

shared = App.new
begin
  shared.request("A") do |b|
    Product.insert(b, name: "pen", price: 12)
    verify("eraser", Product.lookup(b, 1)[:name])
  end
rescue AssertionError => e
  puts e.class
end
p shared.store[:products].size
```
```
AssertionError
1
```

The verification raised an exception, but the product stayed in the store. The exception cuts the flow of the test; it does not delete a write already made to the store of the application. This means that an error has two results: the test itself fails, and the residue it leaves enters the starting state of later tests because of the persistent stay of the store. This is one of the reasons the tests that come after a failed test may break as well. Leaving the cleanup to the test itself is not a solution, because the cleanup code can also stay behind the flow cut by the failing test; the more reliable way is to rebuild its own application for each test. In this core the cost of building is one `App.new` call; when the cost is this low, the gain of sharing does not exist in this core.

#### If the Order Changes the Result

The bounding measurement below will show the broken isolation with two identifiers; here we measure the same breakage through whether a test passes or not. Two tests share the same application: one expects the identifier of the first product to be `1`, the other only creates a product. The tests are run in two different orders, and a new application is built in each run.

```ruby
# order_test.rb — the same two tests, in two orders, in a shared application
require_relative "verification"

expecting = ->(b) { verify(1, Product.insert(b, name: "pen", price: 12)[:id]) }
writing = ->(b) { Product.insert(b, name: "eraser", price: 5) }

{ "expecting, writing" => [expecting, writing], "writing, expecting" => [writing, expecting] }.each do |name, ordering|
  app = App.new
  results = ordering.map do |test|
    app.request("A") { |b| test.call(b) }
    :passed
  rescue AssertionError
    :failed
  end
  puts "#{name}: #{results.inspect}"
end
```
```
expecting, writing: [:passed, :passed]
writing, expecting: [:passed, :failed]
```

The same two tests passed in one order and failed in the other. Each of the tests looks correct on its own; the problem is that the two share the application. A failure that depends on order is a sign that the test does not examine a behavior but the state the tests leave to each other.

#### Tests That Share the Cache

A measurement shows that isolation holds for the cache as well. Two tests share the same application. The first test writes a value into the cache of the application. The second test assumes the cache is empty and wants to fill the same key with its own value.

```ruby
# cache_leak.rb — does the second test that shares the cache see its own value
require_relative "verification"

shared = App.new
shared.request("A") { |b| b.app.store[:cache]["k"] ||= "first test's value" }
read = shared.request("A") { |b| b.app.store[:cache]["k"] ||= "second test's value" }
p read
isolated = App.new.request("A") { |b| b.app.store[:cache]["k"] ||= "second test's value" }
p isolated
```
```
"first test's value"
"second test's value"
```

In the shared application the second test read the value of the first, not its own; in the isolated application it read its own value. Because of its persistent stay, the cache is a leakage channel between tests, and this leakage does not tell that the test failed: the second test may pass, but it has not examined what it expected to.

#### Relation to the Production Environment

M12/K05 measured whether the test environment goes out to production: the configuration of an environment is part of a process and does not leave a process. The same principle holds one step further here. A test builds its own application; the store and sessions of that application live only in the process of that test, inside the objects that test built. This gives a test safety: a test does not read or change the records of the production application.

#### The Bounding Measurement

The situation in which isolation breaks is the one in which two tests share the same application. The first measurement is the situation in which each test builds its own application: in two separate `App.new` instances, what is the identifier of the first product?

```ruby
# isolation.rb — if each test builds its own application
require_relative "testing"

first = App.new.request("A") { |b| Product.insert(b, name: "pen", price: 12)[:id] }
second = App.new.request("A") { |b| Product.insert(b, name: "eraser", price: 5)[:id] }
puts "separate apps: #{first} and #{second}"
```
```
separate apps: 1 and 1
```

Both tests received the identifier `1` as their first record: one can run without the other, and the order does not affect the result. The second measurement sets up two tests that share the same application.

```ruby
# sharing.rb — if two tests share the same application
require_relative "testing"

shared = App.new
first = shared.request("A") { |b| Product.insert(b, name: "pen", price: 12)[:id] }
second = shared.request("A") { |b| Product.insert(b, name: "eraser", price: 5)[:id] }
puts "shared app: #{first} and #{second}"
```
```
shared app: 1 and 2
```

The second test, which would have received `1` when run independently, received `2` after the first test. If a test expects `1`, it passes only when it runs first in the shared application; when run in another order or alone, its result changes. This can be called **leakage between tests**: because the store of the application is in the persistent stay, the value a test wrote changes the state the next test reads. The solution is isolation: each test builds its own application, the store of that application and its clients, from scratch. This holds for the fragment cache of the previous lesson as well. If tests share a cache, the fragment written by one test becomes the fragment the next test sees, and in the same way as stale content it changes the result of the test.

#### Summary

- The instance variable of a controller is in the **this request** stay; in the next request a new controller is built and the variable is empty.
- A unit test examines a single part by reading the instance variable from outside; the smallest verification helper raises an exception if the expected and actual values differ.
- A feature test examines a path from beginning to end; a unit test verifies less but shows the place when it breaks.
- Each test building its own application provides isolation: the first identifier is `1` and `1` in separate applications, and `1` and `2` in a shared application.
- Because the store of the application is in the persistent stay, a shared application leads to leakage between tests: a failing test leaves residue, and the result depends on the order.
- A shared cache is a leakage channel as well; the second test may pass even though it has not examined what it expected to.

#### Next Step

The last thing to be verified in an application is its common security defaults. The next lesson asks why the anti-forgery token is kept in the session and measures the stay of this token.

### Security Defaults

Source: https://academia.sh/en/courses/mvc-framework/security-defaults

Measures why the anti-forgery token is kept in the session, default escaping in the template and an allow list against mass assignment, and closes the course with a table of the stays of thirteen lessons.

The previous lesson measured the isolation of tests from one another. This lesson takes up the last application service of the course: the security defaults an application has to offer even when it does nothing on behalf of the user. Three were chosen: the anti-forgery token, escaping in the template and an allow list against mass assignment. The first question of the lesson is why the anti-forgery token is kept in the session.

The measuring axis has been the same since the start of the course: **how many requests does this value stay for?** The four stays: **this request**, **next request**, **session**, **persistent**. The question asked in M12/K03 was a different one: who can change a result, the server, the environment, the client or a third party. The token goes to the client and returns inside the form; but the answer to that question does not decide the stay of the token. Its stay is the session, because the session is where it is written.

#### Core

The measurements rest on the same small request loop. The file is the course core and is given as it is.

```ruby
# stay.rb — a small request loop and a helper that classifies how many requests a value stays for

# Flash: a value written in this request is carried to the next request only.
class Flash
  def initialize(incoming)
    @incoming = incoming
    @fresh = {}
  end

  def [](key) = @fresh.fetch(key) { @incoming[key] }

  def []=(key, value)
    @fresh[key] = value
  end

  def passed_on = @fresh
end

# Every request gets a new context: params, flash and the request store start empty each time.
class Context
  attr_reader :app, :params, :session, :flash, :request_store

  def initialize(app, session)
    @app = app
    @params = {}
    @session = session
    @flash = Flash.new(session.delete(:flash) || {})
    @request_store = {}
  end
end

# The application store is the same across all requests and all clients;
# the session is chosen per client.
class App
  attr_reader :store

  def initialize
    @sessions = Hash.new { |h, client| h[client] = {} }
    @store = { products: {}, schema: { products: %i[name price] }, templates: {},
               queue: [], cache: {}, counter: 0 }
  end

  def request(client)
    context = Context.new(self, @sessions[client])
    result = yield context
    context.session[:flash] = context.flash.passed_on unless context.flash.passed_on.empty?
    result
  end
end

# Rule: request 1 (client A) writes the value; requests 2 and 3 (A) and request 4 (B) read it.
# If B sees it, persistent; if A's third request sees it, session; if only the second
# request sees it, next; if none sees it, this_request.
def stay(write, read)
  app = App.new
  first = app.request("A") { |b| write.call(b); read.call(b) }
  raise ArgumentError, "value could not be read in the request that wrote it" if first.nil?

  seen = [["A", 2], ["A", 3], ["B", 4]].map do |client, order|
    app.request(client) { |b| read.call(b) }.nil? ? "--" : "#{client}#{order}"
  end
  kind = if seen[2] != "--" then "persistent"
         elsif seen[1] != "--" then "session"
         elsif seen[0] != "--" then "next"
         else "this_request"
         end
  [kind, ["A1", *seen].join(" "), first]
end
```

This block is source; the file is written to disk and pulled in with `require_relative`. The core is not a framework but a model of one. It carries one difference: `Product.lookup` in the core returns the stored object itself, while in many mappers every read builds a new object from the row. The measurements in this lesson do not depend on this difference.

#### The Anti-Forgery Token

**Cross-site request forgery (CSRF)** is another site sending a request from the user's browser on behalf of the user. The defense is simple: a value produced by the server is placed in the form, and the submitted value is compared with the value the server stored. This is the **anti-forgery token**. In the pattern measured in this lesson (the synchronizer token), the server has to store the token somewhere for the comparison to work; there are also stateless patterns that do not store the token, and this lesson does not measure them. Where? The place it is written decides the stay of the token. The excerpt gives two helpers and the form of a result line: `csrf_token`, which produces the token, `escape`, which escapes text, and `report`, which prints the measurement line.

```ruby
# security.rb — token, escaping and the result line
require_relative "stay"

def escape(text) = text.to_s.gsub("&", "&amp;").gsub("<", "&lt;").gsub(">", "&gt;").gsub('"', "&quot;")
def csrf_token(b) = b.session[:csrf] ||= "token-#{b.app.store[:counter] += 1}"

def report(name, kind, evidence, value)
  printf("%-16s%-14s%-13s%s\n", name, kind, evidence, value.inspect)
end
```

This block is source as well. `csrf_token` returns the value in the session if there is one; if not, it produces one and writes it to the session. The measurement produces the token in the first request and reads it from the session in later requests.

```ruby
# token.rb — how many requests does the csrf token stay for
require_relative "security"

write = ->(b) { csrf_token(b) }
read = ->(b) { b.session[:csrf] }
report("csrf token", *stay(write, read))
```
```
csrf token      session       A1 A2 A3 --  "token-1"
```

The token appeared in all three requests of A and did not appear in the request of B. This is the **session** stay. The token goes to a form, but on the server it stays in the session of one client. If it were in the shared store of the application, it would be persistent and every client would see the same value; then the token of one client would be valid for everyone and the comparison would prove nothing. **AS10.** In the core the token derives from a counter; a real token must be unpredictable. This measurement measures only the stay, not the security of the production.

The token goes away with the session. If the token in the session is deleted, the next request produces a new token; the old one no longer exists on the server. In the measurement A gets a token, the token in its session is deleted and A asks for a token again; B also asks for its own token.

```ruby
# renewal.rb — what happens when the token in the session is deleted
require_relative "security"

app = App.new
first = app.request("A") { |b| csrf_token(b) }
app.request("A") { |b| b.session.delete(:csrf) }
fresh = app.request("A") { |b| csrf_token(b) }
other = app.request("B") { |b| csrf_token(b) }
p [first, fresh, other]
```
```
["token-1", "token-2", "token-3"]
```

Each new token got a separate value, and because the counter is shared across clients the token of B came out different from those of A. The token exists as long as it stays in the session; a token deleted from the session is not on the server, so a request arriving with the old value does not match in the comparison (this was not tried in this measurement).

#### Does One Client's Token Pass for Another?

The consequence of the stay is that it separates by client. In the same application A, B and A again make requests; each asks for the token in its own session.

```ruby
# clients.rb — the token in the order A, B, A
require_relative "security"

app = App.new
a1 = app.request("A") { |b| csrf_token(b) }
b1 = app.request("B") { |b| csrf_token(b) }
a2 = app.request("A") { |b| csrf_token(b) }
puts [a1, b1, a2].inspect
```
```
["token-1", "token-2", "token-1"]
```

The second request of A received the first token; B received a separate one. The comparison is added next: if the submitted value matches the value in the session, the request is accepted.

```ruby
# compare.rb — does the submitted token match the one in the session
require_relative "security"

def csrf_valid?(b, submitted) = !submitted.nil? && submitted == b.session[:csrf]

app = App.new
in_form = app.request("A") { |b| csrf_token(b) }
app.request("B") { |b| csrf_token(b) }
p app.request("A") { |b| csrf_valid?(b, in_form) }
p app.request("B") { |b| csrf_valid?(b, in_form) }
p app.request("A") { |b| csrf_valid?(b, nil) }
```
```
true
false
false
```

The own token of A is valid in the session of A. The same value is invalid in the session of B, and a request that submits no token at all is also invalid. There is no value another site could put in the form: the token is bound only to the session of this client. **AS11.** Here the comparison is plain equality; a real application compares in constant time and can renew the token in every form. These were not measured.

#### Default Escaping in the Template

The second default was measured in the Views and Templates lesson: the template passes local values through `escape` when placing them. The text the user wrote appears on the page as text, not as markup. The safe behavior is the default; turning escaping off must be requested separately. The same helper is measured here with harmless markup.

```ruby
# escaping.rb — default escaping
require_relative "security"

p escape("Tom & Jerry")
p escape("<b>pen</b>")
```
```
"Tom &amp; Jerry"
"&lt;b&gt;pen&lt;/b&gt;"
```

Three characters turned into their counterparts. This measurement covers only these three characters. The definition of the helper also converts the double quote; this measurement did not show it. The single quote is not in the definition. **AS12.** A real view layer handles other contexts separately, for example the address of a link or the inside of a script; the core does not model them.

#### An Allow List Against Mass Assignment

The third default prevents the fields coming from a form from being written directly to a record. Even if the form has only `name` and `price`, the request can carry an extra field. If the record takes all the fields, the client also writes a field that is not in the form. This is called **mass assignment**. The defense is an **allow list**: only the fields whose names are in the list pass.

```ruby
# allow.rb — allow list
require_relative "security"

def permitted(params, *keys) = params.slice(*keys)

incoming = { name: "pen", price: 12, admin: true }
p permitted(incoming, :name, :price)
```
```
{name: "pen", price: 12}
```

The `admin` field did not enter the result. An allow list is not a deny list: everything not in the list, including a field added tomorrow, stays out automatically. **AS13.** The helper filters only a single-level hash; nested fields need a separate rule, and it was not written in this lesson.

What happens if a field named in the list is missing from the request? Call the helper with a request that has no `price` field.

```ruby
# missing.rb — a field in the list but not in the request
require_relative "security"

def permitted(params, *keys) = params.slice(*keys)

p permitted({ name: "pen" }, :name, :price)
p permitted({ name: "pen" }, :name, :price).key?(:price)
```
```
{name: "pen"}
false
```

The missing field took its place in the result not with a `nil` value but by not being there at all. An allow list does not make a field required; it only draws the boundary of the fields that may pass. A check that requires a field to be present is a separate rule, and it was measured as validation in the Data Layer topic.

#### The Bounding Measurement

A secure default must be applied only once and in the right layer. If it is applied twice, security does not increase, the output breaks. What happens if the same text is escaped twice?

```ruby
# double.rb — escaping twice
require_relative "security"

p escape("<")
p escape(escape("<"))
```
```
"&lt;"
"&amp;lt;"
```

The second escaping also changed the `&` character of the first escaping; the page shows `&lt;` instead of `<`. A security default therefore stays in a single layer: the template escapes, the controller does not escape, and the record is not stored in its escaped form. In whichever layer a value is to be escaped, it is not touched again after that layer. The same rule applies to partial templates; this breakage was measured in the Views and Templates lesson.

#### Summary

- The anti-forgery token goes to the client but its stay is the **session**: it appears in three requests of the same client and does not appear in another client.
- Clients making requests in the order A, B, A received `"token-1"`, `"token-2"`, `"token-1"`; the token of one client is invalid in the session of another.
- Default escaping in the template shows user text as text on the page.
- An allow list does not write a field that is not in the form to the record.
- Mass assignment is closed with the allow list: a missing field is not present as `nil` but not present at all; the list is not a deny list, and every field not in the list stays out.
- A secure default is applied in a single layer: `escape(escape("<"))` gives `"&amp;lt;"`.

#### Course Wrap-Up

Every lesson of the course asked the same question: how many requests does this value stay for? Thirteen measurements answered this question with an observation. The table below gathers all of them by stay.

| Lesson | Measured value | Stay | Bounding measurement |
| --- | --- | --- | --- |
| Convention over Configuration | `template name` | this request | the name is produced; the template's existence is not checked |
| Routing | `path parameter` | this request | the client carries the value to the next request |
| Controllers and Filters | `flash` | next request | a flash rewritten on every request appears in the session |
| Views and Templates | `compiled template` | persistent | a second compilation keeps the first |
| Object–Relational Mapping | `record` | persistent | the model returns the stored object itself |
| Migrations | `schema column` | persistent | the old row has no new column |
| Relationships and Validations | `validation error` | this request | the error is not stored with the record |
| The Query Interface and the N+1 Problem | `query cache` | this request | a `nil` result is stored too |
| Authentication and Authorization | `user id` | session | a value that cannot be read in the request that wrote it raises `ArgumentError` |
| Background Jobs | `queued job` | persistent | the result depends on where it is written |
| Caching | `fragment cache` | persistent | stays stale while the key does not change |
| Testing | `controller variable` | this request | a shared app leaks between tests |
| Security Defaults | `csrf token` | session | `escape(escape("<"))` gives `"&amp;lt;"` |

The distribution of stays: **5 this request · 1 next request · 2 session · 5 persistent**. The table carries one finding of the course: a stay depends not on the type of a value but on the place it is written and the behavior of the code that reads it. The same name can hide two stays: the query cache was this request, the fragment cache was persistent; the same flash became a session value when it was rewritten on every request.

Two inferences can be drawn. The first is that all five persistent values stay in the shared store of the application and do not separate clients; that is why each carried a bound. The second is that the session and next request stays are bound to the client; they are not visible outside the client that wrote them.

**AS14.** The table is only a classification of thirteen measurements; the distribution reflects the choice of the measured values, not the distribution of all the values in an application. The stay cells were taken from the statement in the Summary section of each lesson.

The core model carried a fact here as well: `Product.lookup` returns the stored object itself, while in many mappers every read builds a new object from the row. The table is therefore the behavior of the model of a framework, not the behavior of a framework.

The course closes here. The next course takes up the packages that make up a Ruby project, dependency management and development tools. The same rule remains valid there too: nothing is guessed, everything is measured.