File and Media Storage

In the previous lesson the disk tier was a small store of preferences. An application sometimes keeps a photo, a document, or an invoice, and these files do not fit in a row or in a single string. Where the file sits matters as well: in the directory of the application itself, or in a shared place that other applications also see? The question of the lesson is the same as in the earlier lessons: where does the answer to this read come from?

// Origin.kt — classifies which tier a read's answer came from
class Tier(val origin: String, private val entries: Map<String, String>) {
    init {
        require(origin in listOf("memory", "disk", "network")) { "invalid tier: $origin" }
    }

    fun lookup(key: String): String? = entries[key]
}

class Read(val label: String, val key: String, chain: List<Tier>, fallback: String) {
    private val hit: Tier? = chain.firstOrNull { it.lookup(key) != null }
    val value: String = hit?.lookup(key) ?: fallback
    val origin: String = hit?.origin ?: "default"

    fun show() {
        println("$label: key=$key | value=$value -> $origin")
    }
}

fun countOrigins(reads: List<Read>) {
    for (name in listOf("memory", "disk", "network", "default")) {
        println("$name: ${reads.count { it.origin == name }}")
    }
}

The App-Specific Directory

Every application has a directory given by the platform that belongs only to itself. This directory is called the app-specific directory. The application asks for no special permission to write files there or to read files from there; other applications cannot see that directory. The directory is also deleted when the application is deleted.

// Platform code: PrivateFiles
class PrivateFiles(private val context: Context) {
    fun save(name: String, bytes: ByteArray) {
        File(context.filesDir, name).writeBytes(bytes)
    }

    fun read(name: String): ByteArray? {
        val file = File(context.filesDir, name)
        return if (file.exists()) file.readBytes() else null
    }
}

read returns null if the file does not exist; that is, this class, like the stores in the earlier lessons, gives no answer for something that was not written. This snippet is platform code, and here we make no claim about output, file path, or size. The model builds the rule of the platform; the model is not the platform itself.

In the model, the directory is a function that extracts a map from file names. Each file is bound to a key with the file. prefix; the value is the path of the file inside the directory. No real file is read or written.

// Files.kt — extracts a map of keys and paths from the file names in a directory
fun indexDirectory(directory: String, files: List<String>): Map<String, String> =
    files.associate { name -> "file.${name.substringBeforeLast('.')}" to "$directory/$name" }

A screen asks for three files: a profile photo, an invoice, and an image chosen from the gallery.

// Gallery.kt — the three files the file screen asks for
fun readFileScreen(chain: List<Tier>): List<Read> = listOf(
    Read("profile", "file.profile", chain, "none"),
    Read("invoice", "file.invoice", chain, "none"),
    Read("gallery", "file.gallery", chain, "none"),
)

A File in the Directory Comes from Disk

There are two files in the directory of the application: profile.jpg and invoice.pdf. The gallery image was not copied into the application.

// Main.kt — a file in the application's own directory is read from disk
fun main() {
    val privateDir = Tier("disk", indexDirectory("private", listOf("profile.jpg", "invoice.pdf")))
    val reads = readFileScreen(listOf(privateDir))
    reads.forEach { it.show() }
    countOrigins(reads)
}

profile: key=file.profile | value=private/profile.jpg -> disk
invoice: key=file.invoice | value=private/invoice.pdf -> disk
gallery: key=file.gallery | value=none -> default
memory: 0
disk: 2
network: 0
default: 1

Two files came from disk, and the gallery image came from the fallback value. This lesson’s origin is disk: a file in the directory of the application is answered by the disk tier.

DL1. The model derives the directory from a list of names and does not carry the content of the file; the value is only the path. The size of the file, a write being interrupted halfway, and a failure caused by a full disk are absent from the model.

The Media Store and Scoped Storage

The gallery image is not in the directory of the application but in the shared media of the user. The platform limits access to this media with the rule of scoped storage: an application cannot open the files of other applications directly by their path. It reaches shared media through a media store query, with the permission the user gave or with the file the user chose.

// Platform code: GalleryQuery
fun queryImageNames(context: Context): List<String> {
    val names = mutableListOf<String>()
    val projection = arrayOf(MediaStore.Images.Media.DISPLAY_NAME)
    context.contentResolver.query(
        MediaStore.Images.Media.EXTERNAL_CONTENT_URI, projection, null, null, null,
    )?.use { cursor ->
        while (cursor.moveToNext()) names.add(cursor.getString(0))
    }
    return names
}

The query returns the images that the application is allowed to see. Without permission, the list comes back empty or the query raises an error; there is no way to open by its path a file that the application was not given permission for. This snippet is also platform code, and we make no claim about output. The model builds the rule of the platform; the model is not the platform itself.

In the model, the permission rule is not a tier. A file in the media store can sit as a Tier that was not added to the chain, and a tier that is not in the chain does not answer the read.

The Bounding Measurement

We will measure two things. The first is the thumbnail: if the profile photo was shrunk and loaded into memory, the read is answered without descending to disk. The second is the gallery image: the file exists in the media store, but it is not in the lookup chain.

The function that keeps the thumbnail in memory produces a memory value from the path of a file on disk.

// Thumbnails.kt — loads the thumbnail of a file read from disk into memory
fun decodeThumbnails(source: Tier, keys: List<String>): Map<String, String> =
    keys.mapNotNull { key -> source.lookup(key)?.let { key to "thumb:$it" } }.toMap()

// Main.kt — the thumbnail is in memory, the gallery image is outside the chain
fun main() {
    val privateDir = Tier("disk", indexDirectory("private", listOf("profile.jpg", "invoice.pdf")))
    val mediaStore = Tier("disk", indexDirectory("media", listOf("gallery.png")))
    val memory = Tier("memory", decodeThumbnails(privateDir, listOf("file.profile")))
    val chain = listOf(memory, privateDir)
    println("in the media store: ${mediaStore.lookup("file.gallery")}")
    val reads = readFileScreen(chain)
    reads.forEach { it.show() }
    countOrigins(reads)
}

in the media store: media/gallery.png
profile: key=file.profile | value=thumb:private/profile.jpg -> memory
invoice: key=file.invoice | value=private/invoice.pdf -> disk
gallery: key=file.gallery | value=none -> default
memory: 1
disk: 1
network: 0
default: 1

Because the profile image was loaded into memory, it now has the memory origin, and its value carries the name of the thumbnail. The invoice still comes from disk. The gallery image is found in the media store, so the file really exists; but because it is not in the lookup chain, the answer is default. The rule looks not at whether the file exists but at the content of the chain.

If the gallery image is copied into the directory of the application, the situation changes. The file now belongs to the application and sits in the disk tier of the chain.

// Main.kt — when the gallery image is copied into the application's directory, it enters the chain
fun main() {
    val files = listOf("profile.jpg", "invoice.pdf", "gallery.png")
    val privateDir = Tier("disk", indexDirectory("private", files))
    val reads = readFileScreen(listOf(privateDir))
    reads.forEach { it.show() }
    countOrigins(reads)
}

profile: key=file.profile | value=private/profile.jpg -> disk
invoice: key=file.invoice | value=private/invoice.pdf -> disk
gallery: key=file.gallery | value=private/gallery.png -> disk
memory: 0
disk: 3
network: 0
default: 0

All three of the three reads came from disk. The same file gave no answer in the media store, and it gives an answer in the directory of the application; the difference is not in the content of the file but in its place in the chain.

The counterpart on the platform is this: an application cannot read the file of another application directly even if it knows its path. The permission rule decides access to the file, and if permission was not given, the read stays unanswered even when the file sits on disk. In the model this is shown by the file never being added to the chain. What this measurement does not say is why the permission was not given or why the user denied it.

DL2. The model builds permission not as a rule but as membership of the chain. A permission request on the platform, the user revoking the permission later, and temporary access being granted for a single file the user chose are absent from the model.

Summary

• This lesson’s origin: disk. The read of a file in the application’s own directory is answered from the disk tier.
• The app-specific directory asks for no permission and belongs only to the application; shared media is reached through a media store query and permission.
• With two files in the directory and the gallery image not copied into the application, the reads came out as disk 2, default 1.
• When the thumbnail is loaded into memory, the profile read becomes memory: memory 1, disk 1, default 1.
• A file that is not in the chain falls to the default even if it exists in the media store; the rule sees the content of the chain, not the existence of the file.
• When the gallery image is copied into the directory of the application, the same read becomes disk: disk 3.

Next Step

In this lesson, the file was on the disk of the same device. Most data is on another machine and is reached with a request. The next lesson, The Network Client, measures how the response to a request is decoded to build the network tier and what a field that cannot be decoded falls to.