Files and Input/Output

At the end of the previous lesson, ensure was seen as a branch that runs in every case. Files are where this construct is used most often: an opened file has to be closed, and forgetting to close it is a leak. In Ruby, File.open can be given a block; the question is who owns the job of closing. Input/output is the program’s exchange of data with the outside world, that is, with files and the standard streams.

This lesson’s responder is the block: the File.open message gives two different results for the same file with two different blocks. At the end of the lesson, all twelve observations of the course are gathered in a single table.

The File Reading Observation

The core’s twelfth observation is file reading. The classification helper reply.rb has been the same since the start of the course and is only written to disk here.

# reply.rb — helpers that classify who answers a message

# Classification asks in order: is the expression a message at all (defined?),
# then do two different blocks give two different results, then who owns the method.
def responder(definition, owner: nil, result: nil, with_blocks: nil)
  return ["syntax", "defined? #{definition}"] unless definition == "method"

  if with_blocks
    first, second = with_blocks.call
    return ["block", "#{first.inspect} / #{second.inspect}"] unless first == second
  end

  kind = owner.instance_of?(Class) ? "class" : "module"
  [kind, "#{owner}, result #{result.inspect}"]
end

# file_reading.rb — the same File.open gives two results with two blocks
require_relative "reply"

File.write("note.txt", "first line\nsecond line\n")

definition = defined?(File.open("note.txt"))
kind, evidence = responder(definition, with_blocks: -> {
  [File.open("note.txt") { |f| f.gets.chomp }, File.open("note.txt") { |f| f.readlines.size }]
})
puts definition
puts "#{kind}  #{evidence}"
puts File.method(:open).owner.instance_of?(Class)

method
block  "first line" / 2
true

File.write wrote the two-line file note.txt. defined? reported the call without a block as "method", so there is a message. The first block returned the first line and the second block returned the line count: the same message gave "first line" and 2. The responder is the block. The owner of File.open is a class (instance_of?(Class) gives true); why the owner appears as a singleton class is the subject of the second course, and here only the result is read. CB56. The measurement was made with a single file and two blocks.

Line Reading and Writing Streams

The shortcuts without a block open and close the file themselves. File.write returns the number of bytes written; File.readlines makes the lines an array and chomp: true drops the line endings; File.foreach yields the lines one at a time. A file that does not exist raises Errno::ENOENT. Because the error message carries a path, only the class is printed.

# streams.rb — writing, reading and a missing file
File.write("note.txt", "first line\nsecond line\n")

puts File.write("b.txt", "abc")
puts File.readlines("note.txt", chomp: true).inspect
puts File.foreach("note.txt").count
begin
  File.read("missing.txt")
rescue Errno::ENOENT => e
  puts e.class
end

3
["first line", "second line"]
2
Errno::ENOENT

abc is three bytes and File.write returned 3. readlines gave an array of two strings. When foreach is called without a block it returns an iterator; count consumed it and found 2. The missing file raised Errno::ENOENT and the branch caught it. CB57. File.write was measured only with string content.

When working with an open file, the line reading methods are called on the instance. The owners of these can also be measured.

# owners.rb — the owners of file and standard output methods
File.write("note.txt", "first line\nsecond line\n")

File.open("note.txt") do |f|
  puts f.class
  puts f.method(:gets).owner
  puts f.method(:each_line).owner
end
puts $stdout.method(:puts).owner

File
IO
IO
IO

The file object is a File, but gets and each_line are defined not in its own class but in the IO class. $stdout.puts is in the same IO class too. A file and standard output are members of the same class family, and for all three the responder is the class. CB58. The owners were measured on Ruby 4.0.7; which method is defined in which class can move from version to version.

A File Left Open and a File Closed

If File.open is called without a block it returns the file object and the file stays open; closing it is the caller’s job. In a call with a block, the file object is given to the block, the file closes when the block ends, and the value of the call is the last expression of the block.

# closing.rb — File.open without and with a block
File.write("note.txt", "first line\nsecond line\n")

f = File.open("note.txt")
puts f.class
puts f.closed?
f.close
puts f.closed?

taken = nil
value = File.open("note.txt") do |file|
  taken = file
  puts file.closed?
  42
end
puts value
puts taken.closed?

File
false
true
false
42
true

The call without a block returned a File, closed? printed false, and it became true only when close was called by hand. In the call with a block, the file was open inside the block. The block ended with 42: the return value was 42, while the file object carried out of the block was now closed. The result comes from the block, the closing comes from the method. CB59. The measurement was made for a local file and a single block.

Opening Modes

File.open takes a mode as its second argument. "w" resets the file as it opens it, and "a" appends to the end of the existing content. The difference is measured with the line count. The value of a File.open call with a block is the last expression of the block; write gives the number of bytes written.

# modes.rb — w resets, a appends
File.write("mode.txt", "one\ntwo\n")

File.open("mode.txt", "a") { |f| f.puts "three" }
puts File.readlines("mode.txt").size

result = File.open("mode.txt", "w") { |f| f.write("one\n") }
puts result
puts File.readlines("mode.txt").size

File.open("mode.txt", "a") { |f| puts f.method(:puts).owner }

3
4
1
IO

When a line was added to the two-line file with "a", the count became 3. Opening with "w" reset the file: the block returned the value of write, that is, the 4 bytes, and the line count dropped to 1. The puts method is defined in the IO class; the responder is the class. CB60. The measurement was made only with the "a" and "w" modes; the read and write modes were not tried together.

The Difference Between puts and print

When writing to a file, puts adds a line ending and print does not. The difference shows in the inspect output of the written content.

# write_difference.rb — puts adds a line ending, print does not
File.open("diff.txt", "w") do |f|
  f.puts "a"
  f.print "b"
  f.puts "c"
end
puts File.read("diff.txt").inspect
puts File.size("diff.txt")
File.open("diff.txt") { |f| puts f.method(:print).owner }

"a\nbc\n"
5
IO

The file has a, a line ending, b, c and a line ending again; b and c were written adjacent. Five bytes in total. print is also in the IO class; the responder is the class. CB61. The content was ASCII strings; with multibyte characters the byte count separates from the character count, and that was not measured here.

Existence and Size

File.exist? tells whether a path exists and File.size gives the size in bytes. An open file object also has its own size method, and its owner is File.

# size.rb — existence and size
File.write("size.txt", "abcd")
puts File.exist?("size.txt")
puts File.exist?("missing.txt")
puts File.size("size.txt")
File.open("size.txt") do |f|
  puts f.size
  puts f.method(:size).owner
end

true
false
4
4
File

It returned true for the existing file and false for the missing one. The size was 4 bytes and the open object gave the same value. This time the owner is not IO but File: information specific to files sits in the subclass, stream work in the parent class. CB62. An empty file and a deleted file were not measured.

Line by Line and Enumerable

If each_line is not given a block it returns an iterator, and chomp: true drops the line endings. The iterator’s map method comes from the Enumerable module. Reading, transforming, filtering and writing can be chained as one flow.

# flow.rb — read, map, select, write
File.write("input.txt", "plum\nfig\ngrape\nox\n")

lines = []
File.open("input.txt") { |f| f.each_line(chomp: true) { |s| lines << s } }
puts lines.inspect

iterator = File.foreach("input.txt", chomp: true)
puts iterator.method(:map).owner

selected = iterator.map(&:upcase).select { |s| s.size > 3 }
written = File.write("output.txt", selected.join("\n") + "\n")
puts written
puts File.read("output.txt").inspect

["plum", "fig", "grape", "ox"]
Enumerable
11
"PLUM\nGRAPE\n"

each_line gave the four lines without the trailing line ending. When File.foreach was called without a block it returned an iterator, and map for it is in the Enumerable module; the responder is the module. After upcase, select kept those longer than three letters: PLUM and GRAPE. The written PLUM, a line ending, GRAPE, a line ending is eleven bytes. CB63. The flow was single-pass and measured with a small input; what happens to memory with a large file was not measured.

Reading Standard Input from a Pipe

Standard input is the stream from which the program takes lines from outside, and it is read with $stdin. The lesson’s blocks do not wait for interactive input; so the input is given through a pipe: the output of one command becomes the input of the next.

# read.rb — reads standard input line by line
first = $stdin.gets.chomp
rest = $stdin.each_line.map(&:chomp)
puts first
puts rest.inspect
puts $stdin.method(:gets).owner

# input.sh — feeds read.rb from a pipe
printf 'one\ntwo\nthree\n' | ruby read.rb

one
["two", "three"]
IO

gets read the first line and each_line consumed the rest. $stdin.gets is also in the IO class; the same class reads a file and the keyboard. CB64. The input was a fixed three-line string; empty input and very large input were not measured.

The same pipe is also used for counting lines. When each_line is called without a block it returns an iterator; count comes to it from the Enumerable module.

# count.rb — counts the lines on standard input
puts $stdin.each_line.count
puts $stdin.method(:each_line).owner
puts $stdin.each_line.method(:count).owner

# count.sh — feeds count.rb from a pipe
printf 'a\nb\nc\nd\n' | ruby count.rb

4
IO
Enumerable

The four-line input was counted as 4. The each_line method is in the IO class, while the iterator’s count method is in the Enumerable module; these two responders work one after the other in the same expression. CB65. The input was fixed and four lines long; input whose last line does not end with a line ending was not measured.

The Bounding Measurement

This lesson’s responder is the block, but if this sentence is left half finished it misleads: the block determines the result, and it is not the block that cleans up the resource. The block form of File.open takes on the closing. To separate this, an exception is raised inside the block. The file object is kept outside the block, the exception is caught, and then the state of the file is asked. For comparison, the same exception is raised next to a file opened without a block.

# exception_closing.rb — the file closes even when the block raises
File.write("note.txt", "first line\nsecond line\n")

taken = nil
begin
  File.open("note.txt") do |file|
    taken = file
    raise "block broke"
  end
rescue => e
  puts e.class
end
puts taken.closed?

manual = File.open("note.txt")
begin
  raise "left open"
rescue => e
  puts e.class
end
puts manual.closed?
manual.close
puts manual.closed?

RuntimeError
true
RuntimeError
false
true

In the first case the block raised a RuntimeError, the branch caught it, and the file was closed all the same (true). In the second case the same exception was raised next to a file opened without a block, and the file stayed open (false) until close was called by hand. The difference is not in the content of the block but in where the responsibility for closing sits: File.open with a block runs the block inside an ensure and closes the file whatever happens. This is the resource cleanup construct of the previous lesson placed in the language’s library. The block determines the result and File.open takes on the resource cleanup. CB66. The measurement was made with RuntimeError; leaving the block with break or throw was not tried.

Summary

• File.read, File.readlines and File.foreach open and close the file themselves; File.write returns the number of bytes written; reading a missing file raises Errno::ENOENT.
• This lesson’s responder is the block: the same File.open message gave "first line" with one block and 2 with the other.
• File.open without a block leaves the file open; in a call with a block, closed? becomes true when the block ends and the return value is the block’s value.
• gets, each_line and $stdout.puts are defined in the IO class; standard input is read by feeding it from a pipe.
• "w" resets the file and "a" appends to the end; puts adds a line ending and print does not; the iterator’s map and count methods are in the Enumerable module.
• The bounding measurement: even if the block raises an exception the file closes, because the closing is done by the File.open method, not the block.

Course Wrap-Up

Every lesson of the course asked the same question: who answers this message. The table puts side by side the message each of the twelve lessons measured, that lesson’s responder, and what its bounding measurement said. The responder column is what is written in each lesson’s ## Summary section.

Lesson	Measured message	Responder	Bounding measurement
The Design Philosophy of Ruby	5.between?(1, 10)	module	between? is in Comparable, <=> in the receiver’s class
An Intuition for the Object Model	5 + 3	class	Module answers the Integer.=== message
Variables and Constants	x = 5	syntax	y += 1 is an assignment, Integer answers the addition inside it
Control Flow	(1..10) === 7	class	case asks the class of the branch object
Methods	format("%03d", 7)	module	a top-level def is written to Object and is private
Symbols and Strings	"name".to_sym	class	the magic comment does not change the owner, it applies per file
Arrays and Hashes	Hash.new { 0 }[:x]	block	the block answers only when the key is missing
Blocks	numbers.map { ... }	block	each gives [1, 2] with both blocks, its owner is Array
The Enumerable Module	(1..5).each_slice(2)	module	sum is in the module on a range, in Array on an array
Procs and Lambdas	->(n) { n * 2 }.call(3)	class	the owner is the same, the difference is in the lambda? state
Exceptions	begin ... rescue ... end	syntax	the match of a branch is made with the === message
Files and Input/Output	File.open("note.txt") { ... }	block	the method, not the block, does the closing

The distribution of responders over the twelve lessons: 4 class · 3 module · 3 block · 2 syntax. This is the same as the distribution of the core’s twelve observations, because the core carries one observation per lesson. Two results can be read from the table. First, the responder often did not work alone: most of the bounding measurements showed that another responder was working next to the lesson’s responder. Second, the answer to the responder question depends not on the message but on the message’s receiver, the block given and the form of the expression. CB67. The table gives the distribution of this course’s twelve observations; it carries no claim about the distribution over Ruby as a whole.

The course ends here. In Ruby, the code that produces the result of an operation sits in four places: in the receiver’s class, in a mixed-in module, in the block the caller gave, or in the language’s own syntax. This course told these four places apart only by measuring the result; the question of why a method is found in a particular class or module stayed open. The subject area of the next course is class definition, inheritance, modules and mixins, the method lookup chain and metaprogramming; those lessons will take up the machinery behind today’s measurements.