Camera and Media

In the previous lesson the location updates were a continuous stream and the rule dropped the ones in between. In the camera the event is born from a user action: the user touches the shutter and the app expects a photo. One touch is expected to give one result. In this lesson we ask the same question again: how many copies of this sent event reached the receiver?

// Arrival.kt — classifies how many copies of a sent event reached the receiver
class Signal(val key: String, val seq: Int)

class Arrival(val label: String, val sent: Signal, received: List<Signal>) {
    val copies: Int = received.count { it.seq == sent.seq }
    val verdict: String = when {
        copies > 1 -> "duplicated"
        copies == 1 -> "once"
        received.any { it.key == sent.key && it.seq > sent.seq } -> "superseded"
        else -> "lost"
    }

    fun line() {
        println("$label: key=${sent.key} | copies=$copies -> $verdict")
    }
}

fun countArrivals(arrivals: List<Arrival>) {
    for (name in listOf("once", "duplicated", "superseded", "lost")) {
        println("$name: ${arrivals.count { it.verdict == name }}")
    }
}

The classification is the same as in the previous lessons. The rule counts the copies in the receiver’s list that carry the same sequence number; if there is no copy, it checks whether a later event with the same key arrived. It does not see the content or the cause.

Permission and Capture Request

The camera is also a capability that asks for a runtime permission. It is declared in the manifest and the user approves it on first use.

<uses-permission android:name="android.permission.CAMERA" />
<uses-feature android:name="android.hardware.camera" android:required="false" />

The camera library splits a capture into two parts. The app gives a capture request; the library returns the result through a callback. For a capture that ends successfully it reports that the file was saved, and for one that fails it reports an error object.

// Platform code: ShutterController
import androidx.camera.core.ImageCapture
import androidx.camera.core.ImageCaptureException

class ShutterController(private val capture: ImageCapture) {
    fun onShutterTap(options: ImageCapture.OutputFileOptions) {
        capture.takePicture(options, executor, object : ImageCapture.OnImageSavedCallback {
            override fun onImageSaved(result: ImageCapture.OutputFileResults) = openPreview(result)
            override fun onError(error: ImageCaptureException) = logFailure(error)
        })
    }
}

Every takePicture call produces a result or an error. The model builds this rule of the platform; the model is not the platform itself. This snippet claims no output or duration.

Every Touch Is One Result

In the model, each capture request is a Signal. The key is the name of the photo, and the sequence number is the step at which the request was given. The model of the camera library returns a single result for each request.

// Camera.kt — returns a single result for each capture request
fun shoot(requests: List<Signal>): List<Signal> =
    requests.map { Signal(it.key, it.seq) }

// Main.kt — each capture request arrives once as a single result
fun main() {
    val sent = listOf(
        Signal("photo.morning", 1),
        Signal("photo.noon", 2),
        Signal("photo.evening", 3),
        Signal("photo.night", 4),
    )
    val received = shoot(sent)
    val arrivals = sent.map { Arrival("event-${it.seq}", it, received) }
    arrivals.forEach { it.line() }
    countArrivals(arrivals)
}

event-1: key=photo.morning | copies=1 -> once
event-2: key=photo.noon | copies=1 -> once
event-3: key=photo.evening | copies=1 -> once
event-4: key=photo.night | copies=1 -> once
once: 4
duplicated: 0
superseded: 0
lost: 0

This lesson’s delivery is once. All four requests arrived as a single copy. In the previous two lessons the rule counted an event taking the place of another as superseded; here the keys differ from one another, so no event can take the place of another.

SC1. The model takes every capture request as successful. On the platform a capture can fail while the camera is busy, while the permission is revoked or while the library restarts; the error callback is not in the model.

The Error Callback and the Result Contract

The contract of the library is this: for each capture request, only one of the success callback or the error callback runs. Both do not run, and neither running is a violation of the contract. The “a single result for every request” rule of the model is a plain form of this contract: the result is either a file or an error, but it is always single.

For the app this means the error callback is not a luxury. A screen that writes the success path and leaves the error path empty produces a loss that tells the user nothing: the user touched the shutter and nothing on the screen changed. In the measurement, the loss of two captures is only a number for the rule; for the user it is two separate disappointments. The information the error object carries decides what can be told to the user: a redirect to the settings for a missing permission, a suggestion to free space for full storage.

The Lifecycle of the Screen

The camera preview is bound to the lifecycle of the screen. The camera opens when the screen becomes visible and closes when the screen is hidden. The capture request of the library carries meaning at a time when the camera is open; a request given while the camera is closed falls to the error callback.

The relation of this to delivery lies in the moment at which the request is given. If the user touches the shutter and immediately switches to another app, the request has been given but the camera closes as the screen is hidden. For a camera that closes before the capture is completed, the library reports an error; the photo is never formed. In the model this corresponds to a request being lost, as when storage fills up. The difference is that the cause is the lifecycle rather than storage. The rule does not tell these two causes apart; to tell them apart one has to look at the error object.

A Request Made Without Permission

A capture request made while the permission has not been granted also falls to the error callback. The requests the app gave until the user answers the permission dialog are not processed again once the permission is granted. In the model below the permission is granted at step 14 and the requests before it produce no result. The first three requests carry the same photo name: the user is trying to capture a document.

// Permission.kt — drops the result of capture requests made before the permission was granted
fun requirePermission(sent: List<Signal>, grantedAt: Int): List<Signal> =
    sent.filter { it.seq >= grantedAt }

// Main.kt — requests made without permission produce no result
fun main() {
    val sent = listOf(
        Signal("photo.document", 12),
        Signal("photo.document", 13),
        Signal("photo.document", 14),
        Signal("photo.page", 15),
    )
    val received = requirePermission(shoot(sent), grantedAt = 14)
    println("received seqs: ${received.map { it.seq }}")

    val arrivals = sent.map { Arrival("event-${it.seq}", it, received) }
    arrivals.forEach { it.line() }
    countArrivals(arrivals)
}

received seqs: [14, 15]
event-12: key=photo.document | copies=0 -> superseded
event-13: key=photo.document | copies=0 -> superseded
event-14: key=photo.document | copies=1 -> once
event-15: key=photo.page | copies=1 -> once
once: 2
duplicated: 0
superseded: 2
lost: 0

Requests 12 and 13 produced no result, but request 14 gave a result with the same key. For this reason the rule counted the two as superseded rather than lost. This count is consistent with the definition of the rule, but it is misleading from the point of view of the photo: the requests without permission did not take the place of any photo, because no photo was ever taken. In this case the superseded label says only that “a result with the same key came later”; it should be read as a loss that the user did not see and that was made up for afterwards. Had the keys been different, the same situation would have been counted as lost. Reading the rule like a textbook requires remembering that the choice of key changes the number.

SC2. The model takes the permission as granted at a single step and as certain. On the platform the permission dialog can be shown to the user several times, if “do not ask again” is chosen later calls are rejected without showing a dialog, and the permission can be granted later from the settings.

When Storage Is Full

The result of a capture is a file, and there has to be a place for the file to be written. When storage is full, the library cannot save the result and falls to the error callback. In the model, the number of results the storage can take is an integer capacity; the number is a made-up value of the model.

// Storage.kt — drops later results once the storage capacity is full
fun store(results: List<Signal>, capacity: Int): List<Signal> =
    results.take(capacity)

// Main.kt — once storage is full, later captures do not arrive
fun main() {
    val sent = listOf(
        Signal("photo.a", 5),
        Signal("photo.b", 6),
        Signal("photo.c", 7),
        Signal("photo.d", 8),
    )
    val received = store(shoot(sent), capacity = 2)
    println("received seqs: ${received.map { it.seq }}")

    val arrivals = sent.map { Arrival("event-${it.seq}", it, received) }
    arrivals.forEach { it.line() }
    countArrivals(arrivals)
}

received seqs: [5, 6]
event-5: key=photo.a | copies=1 -> once
event-6: key=photo.b | copies=1 -> once
event-7: key=photo.c | copies=0 -> lost
event-8: key=photo.d | copies=0 -> lost
once: 2
duplicated: 0
superseded: 0
lost: 2

The first two captures arrived once; captures 7 and 8 were lost. Because their keys differ, none of them took the place of another and the rule did not count any as superseded. The rule does not see that the cause of the loss is full storage; to know this, the app has to listen to the error callback.

SC3. The model builds the storage with a fixed capacity. On the platform the free space changes according to the other work of the user and the capacity can run out during a capture; this is not in the model.

Media Picker and Shared Storage

Not every photo has to be the app’s own capture. When the user picks an existing photo from the gallery, the selection comes from a shared media store, and the app is given only the address of the selected file. The picker asks for no separate permission, because the app reaches not all the media but only what the user selected. This is a separate case in the delivery model: a selection is an event, but the owner of the event is the user, and the selected file can be counted as a result that was sent and arrived.

The difference between a capture and a selection lies in the responsibility of the error path. In a capture, storage can fill up and the permission can be missing. In the picker the user can give up, and this is not counted as an error: an empty result returns. Giving up is neither lost nor superseded, because no event was sent. This distinction shows the limit of the measurement model: the rule counts sent events, not unsent ones.

The Preview Is Not a Capture

The camera library offers two separate streams: the preview and the capture. The preview is an image updated continuously on the screen that shows the user the frame; the capture produces a single file. Every frame of the preview looks like an event, but it is not a result delivered to the app: frames are drawn on the screen and discarded. In the model of this lesson there is only the capture, because in the preview there is no frame that has to reach a receiver. Because frames can be discarded, the preview is the visual counterpart of the superseded behavior in the location and sensor lessons: a slow screen silently drops old frames, and nobody counts this as a loss.

A capture, on the other hand, cannot be dropped. When the user asks for a photo, the result is either a file or an error. Writing the file takes time, and the success callback does not run until the writing finishes. If the screen is hidden while the file is being written, the writing is mostly completed but the success callback now reaches a screen that is no longer visible. In this case the file exists and the user sees it in the gallery; but the reaction of the screen to the result, for example updating the preview thumbnail, does not happen. For the rule this event counts as arrived; for the user half of the result is lost. This difference, which stays outside the model, requires keeping the result somewhere independent of the screen, for example in a view model.

Common Misconceptions

“The second touch is a copy of the first touch.” A copy is the same sequence arriving more than once. Two touches are two separate requests and the results are two separate ones. If the user wants a single photo, the solution is to block the second touch before the request is given; deduplication on the result side leaves both photos taken.

“If the capture request was given, the photo exists.” The request is only an intention. The photo exists when the success callback runs. If the error callback ran, it does not exist.

“The permission was granted once, the camera always opens.” The camera may be in use by another app, the device may have turned the camera off to save power, or the user may have revoked the permission later. The error path must be ready at every capture request.

“If the file was saved, the user has seen it.” The success callback reports that the file was written, not that the user saw the result. If the screen is hidden at that moment, the result is saved but not shown. If the app does not keep the result somewhere it can be read when the screen becomes visible again, the user finds the photo only in the gallery.

“Full storage cannot be known before the capture.” The free space can be queried before the capture. A check made before the capture makes it possible to tell the user about the two losses in the measurement before the shutter is even touched; the error callback is the last resort.

“If the preview is visible, the camera is ready.” While the preview runs, a capture can still fail: storage may be full, the permission revoked, or a previous capture may still be being written. Readiness is decided by the result of the capture request, not by the preview.

“Superseded applies only to location and sensors.” Superseded, in the definition of the rule, means that an event is replaced by a newer event with the same key. As in the measurement of the request without permission, it can be seen in a capture too. The category depends not on the type of the event but on the keys and the sequences.

“The media picker also asks for a permission.” The picker gives access to the file the user selected; a broad storage permission is not needed.

The Bounding Measurement

What happens if the shutter is touched twice in a row? When the user touches the shutter twice quickly, the app gives two separate capture requests. These are not two copies of the same event: they are two separate requests and each carries its own sequence.

// Main.kt — two touches on the shutter arrive once each as two separate sequences
fun main() {
    val sent = listOf(
        Signal("photo.cover", 9),
        Signal("photo.cover", 10),
        Signal("photo.detail", 11),
    )
    val received = shoot(sent)
    val arrivals = sent.map { Arrival("event-${it.seq}", it, received) }
    arrivals.forEach { it.line() }
    countArrivals(arrivals)
}

event-9: key=photo.cover | copies=1 -> once
event-10: key=photo.cover | copies=1 -> once
event-11: key=photo.detail | copies=1 -> once
once: 3
duplicated: 0
superseded: 0
lost: 0

Captures 9 and 10 carry the same key, but because their sequences differ the rule counts each by its own sequence. Both arrived once; neither was counted as duplicated. Duplicated is the same sequence arriving more than once. If the user asked for two photos, the user gets two photos. If we want the second touch to be ignored, this is the app’s own rule and must be applied before the capture request is given.

SC4. The model builds the two touches as two independent requests. On the platform the library may allow only one capture to be processed at a time and may queue or reject the second request; this is not in the model.

Summary

• This lesson’s delivery: once. Each capture request arrived as a single result: once 4.
• Captures whose keys differ do not take each other’s place; this is why there is no superseded in this lesson.
• Once storage filled up, the next two captures were lost: lost 2. The rule does not see the cause.
• Two requests made without permission were counted as superseded because a later request with the same key arrived: once 2, superseded 2. The choice of key changes the number.
• Two touches on the shutter are two separate sequences and both arrive once; duplicated is the same sequence arriving more than once.
• The camera permission is a runtime permission; the result callback separates success from error.

Next Step

In this lesson the event was born from a single action and each request received a single result. In sensors the event is produced in great numbers every second, and a slow receiver cannot keep up with all of it. The next lesson, Sensors, measures the delivery of the sensor events that a slow receiver skips.