---
title: 'Indexing and Slicing'
source: 'https://academia.sh/en/courses/data-analysis-with-python/indexing-and-slicing'
course: 'Data Analysis with Python'
language: en
updated: '2026-10-07T04:31:10+00:00'
license: 'CC BY-SA 4.0'
objectives:
    - 'Separates label-based from position-based access: shows by row count that the end of one is inclusive and the end of the other is exclusive'
    - 'Counts how many rows the same two ends return in four spellings and measures how a non-unique index multiplies the result'
    - 'Shows that a label slice depends on the assumption of a sorted index and computes the number of rows returned once the order is broken'
    - 'Measures the gain of the index in pass count: linear scan against binary search'
---

# Indexing and Slicing

The same two ends, 10,100 and 10,200, return four different row counts in four separate spellings: sliced by label it is 95 rows because a label slice includes the end, sliced by the same two positions it is 94 rows because a position slice leaves the end out, written with the ends mistaken for positions it is 100 rows, and in a long table with a non-unique index it is 230 rows. When the frame is sorted by total consumption the label index stops being sorted and the same slice returns 617 rows; an end that is not in the index gives a slice of 22 rows in a sorted index and raises an error in an unsorted one. A linear scan without an index takes 2,658 passes, while binary search in the label index reaches the same 95 rows in 21 passes.

The dataset has entered the frame and has been read with known column types. The next question is how to
reach a row. When "the subscribers between 10,100 and 10,200" is requested from a frame, there are two
separate paths: asking for the row by its **label** and asking for the row by its **place in the order**.
Both take a range, both return a subtable, and in code they look almost the same.

Although they look the same, they do not do the same thing, and the difference is gathered into a single
decision: whether the last end of the range is included in the result. That is the measure of this lesson:
how many rows the same two ends return in different spellings. A second question stands beside it: a label
slice assumes the index is sorted, and this assumption is written nowhere.

- **DF10. Label-based access** finds the row by its name in the index; **position-based access** finds the
  row by its place in the table. The two coincide only if the index is an integer sequence running `0, 1, 2, …`.
- **DF11.** A label slice **includes both ends**, a position slice **leaves the last end out**. This is not a
  library preference but the result of two different addressing logics: in a range of names the last name is
  also a name, while in a range of positions the last position is a boundary.
- **DF12.** The index **may be non-unique**. If the same label occurs in more than one row, a label slice
  brings all the rows of that label, and the number of rows returned is larger than the number of labels.
- **DF13.** A label slice assumes the index is **sorted**. If it is sorted, the two ends are found by binary
  search; if it is not, the slice comes to mean "between the positions where the two ends first occur", and
  that is a completely different set.

## Two Access Modes

```python
# dataset.py — MODEL: the long-format reading table of the previous lesson is
# regenerated from the same seed. The dataset is fiction; the detail of the
# generator is in the first lesson of the topic.
import math

M32 = 0xFFFFFFFF
SEED, PERIODS = 20260218, ["2026-01", "2026-02", "2026-03"]
REGION = [("north", 0.28, 21), ("south", 0.22, 17), ("east", 0.18, 26),
          ("west", 0.14, 14), ("central", 0.18, 23)]

def rng(seed: int):
    s = ((seed ^ (seed >> 16)) * 2246822507) & M32
    s = ((s ^ (s >> 13)) * 3266489909) & M32
    s ^= s >> 16
    def next_value() -> float:
        nonlocal s
        s = (s * 1664525 + 1013904223) & M32
        return s / 4294967296
    return next_value

def pick(u: float, weights: list[float]) -> int:
    t = 0.0
    for i, p in enumerate(weights):
        t += p
        if u < t:
            return i
    return len(weights) - 1

subscribers, dropped = [], []
for i in range(1400):
    r = rng(SEED + i)
    name, _, base = REGION[pick(r(), [b[1] for b in REGION])]
    r(); r()  # the household and satisfaction draws are not used in this lesson
    (dropped if r() < 0.046 else subscribers).append(
        {"subscriber_id": 10001 + i, "region": name, "base": base})

valid = {k["subscriber_id"] for k in subscribers}
pairs: dict[tuple[int, str], float] = {}
for k in subscribers + dropped:
    r = rng(SEED + 7000 + k["subscriber_id"])
    for d in range(pick(r(), [0.05, 0.12, 0.21, 0.62])):
        t = 0.0 if r() < 0.038 else math.floor(k["base"] * math.exp(
            (r() + r() + r() - 1.5) * 0.62) * (1 - d * 0.05) * 100 + 0.5) / 100
        if k["subscriber_id"] in valid:
            pairs.setdefault((k["subscriber_id"], PERIODS[d]), t)
        r()
LONG = [{"subscriber_id": a, "period": d, "consumption_m3": t} for (a, d), t in pairs.items()]

PASSES = {"count": 0}

class LabelIndex:
    """A list of labels and a label -> position dictionary. Whether it is sorted
    is a property of the index itself and determines the slicing behavior."""

    def __init__(self, labels):
        self.labels = list(labels)
        self.position = {e: i for i, e in enumerate(self.labels)}
        self.is_sorted = all(self.labels[i] <= self.labels[i + 1]
                             for i in range(len(self.labels) - 1))

    def __len__(self):
        return len(self.labels)

totals = {}
for o in LONG:
    totals[o["subscriber_id"]] = round(totals.get(o["subscriber_id"], 0.0) + o["consumption_m3"], 2)
summary = [{"subscriber_id": k["subscriber_id"], "region": k["region"],
            "total_m3": totals.get(k["subscriber_id"], 0.0)} for k in subscribers]
index = LabelIndex([s["subscriber_id"] for s in summary])
long_index = LabelIndex([o["subscriber_id"] for o in LONG])
print(f"summary frame {len(index)} rows, labels {index.labels[0]}..{index.labels[-1]}, "
      f"{index.labels[-1] - index.labels[0] + 1 - len(index)} gaps in between")
print(f"long table {len(long_index)} rows, {len(set(long_index.labels))} unique labels; "
      f"is the index sorted: {'yes' if long_index.is_sorted else 'no'}")
```

```
summary frame 1329 rows, labels 10001..11399, 70 gaps in between
long table 3199 rows, 1260 unique labels; is the index sorted: yes
```

Both frames come from the same dataset, and their indexes differ from each other in two directions. In the
summary frame a row is a subscriber, the index is unique, but it is **not continuous**: there are 1,399
numbers between 10,001 and 11,399, and 70 of them carry no subscriber. These are the records dropped in the
cleaning step of Data Collection and Preparation, and they remain as a gap in the index.

In the long table the index is not unique: 3,199 rows carry 1,260 distinct subscriber labels, because the
three periods of a subscriber are three rows. These two properties, the index with gaps and the non-unique
index, directly determine the behavior of slicing. Both are properties of the index itself; the code that
builds the frame declares them nowhere.

## The Same Slice, Four Numbers

```python
# slice.py — the same two ends, different spellings. A label slice includes both
# ends, a position slice leaves the last end out.
def binary_search(labels, target, right=False) -> int:
    """Looks up a position in a sorted list; the comparison count is added to PASSES."""
    low, high = 0, len(labels)
    while low < high:
        PASSES["count"] += 1
        mid = (low + high) // 2
        if (labels[mid] <= target) if right else (labels[mid] < target):
            low = mid + 1
        else:
            high = mid
    return low

def label_slice(index: LabelIndex, start, stop):
    """Slice by label: both ends are inclusive. If the index is not sorted, the
    positions of the two ends themselves are taken and the rows between them are returned."""
    if index.is_sorted:
        return binary_search(index.labels, start), binary_search(index.labels, stop, right=True)
    return index.position[start], index.position[stop] + 1

def linear_slice(labels, start, stop):
    """The same slice without building an index: every label is checked twice."""
    selected, passes = [], 0
    for e in labels:
        passes += 2
        if start <= e <= stop:
            selected.append(e)
    return selected, passes

START, STOP = 10100, 10200
PASSES["count"] = 0
i, j = label_slice(index, START, STOP)
label_rows, label_passes = j - i, PASSES["count"]
WIDTH = [40, 10, 10]
show = lambda h: "".join(str(v).rjust(WIDTH[k]) if k else str(v).ljust(WIDTH[0])
                         for k, v in enumerate(h))
print(show(["access mode", "rows", "passes"]))
print(show([f"by label {START}..{STOP} (end inclusive)", label_rows, label_passes]))
print(show([f"by position [{i}:{j - 1}] (end exclusive)", (j - 1) - i, 0]))
print(show([f"by position [{START - 10001}:{STOP - 10001}]", len(summary[START - 10001:STOP - 10001]), 0]))
PASSES["count"] = 0
ui, uj = label_slice(long_index, START, STOP)
print(show([f"by label in the long table {START}..{STOP}", uj - ui, PASSES["count"]]))
selected, linear_passes = linear_slice(index.labels, START, STOP)
print(show(["linear scan without index", len(selected), linear_passes]))
```

```
access mode                                   rows    passes
by label 10100..10200 (end inclusive)           95        21
by position [95:189] (end exclusive)            94         0
by position [99:199]                           100         0
by label in the long table 10100..10200        230        24
linear scan without index                       95      2658
```

Five rows, the same two ends, and four different numbers. The difference between the first two is exactly
one row, and that row is the **last row** of the slice: the label slice includes subscriber 10,200, the
position slice written with the same two positions does not. A difference of one row is invisible in a
report, but in a threshold calculation it leaves out the record at the boundary, and the record at the
boundary is usually what the calculation is about.

The third row is a more common confusion. Because the labels are integers they have been written as
positions, and that would be right if the index were continuous. It is not: because of the 70 gaps in
between, the label `10100` is at position 95, not at position 99. The result is 100 rows, its first row is
not 10,100, and no error has been raised.

The fourth row shows the price of the index being non-unique. In the long table the same label slice
returns **230 rows**, because every subscriber in the range carries between one and three rows. That the
number is larger than 95 is not a mistake; what is wrong is reading it as "I selected 95 subscribers". A
label slice does not select subscribers, it selects **rows**.

The last row gives the gain. Reaching the same 95 rows without building an index takes 2,658 comparisons,
because each of the 1,329 labels is checked twice. In the label index two binary searches finish the job in
**21 passes**: 126 times fewer comparisons and a one-line slice in place of a six-line loop.

## Row and Column Together

```python
# axis.py — label-based access works on two axes: first the row, then the column.
# The measure is the number of cells that enter the result.
COLUMNS = ["subscriber_id", "region", "total_m3"]
wanted = ["region", "total_m3"]
part = summary[i:j]
print(f"frame {len(summary)} rows x {len(COLUMNS)} columns = {len(summary) * len(COLUMNS)} cells")
print(f"row slice first, then column selection: {len(part)} x {len(wanted)} = "
      f"{len(part) * len(wanted)} cells")
print(f"column selection first, then row slice: {len(summary)} x {len(wanted)} = "
      f"{len(summary) * len(wanted)} cells in the intermediate table, then {len(part)} rows")
single = [s["total_m3"] for s in part]
print(f"if a single column is selected the result is a series: {len(single)} values, "
      f"total {round(sum(single), 2)} m3")
```

```
frame 1329 rows x 3 columns = 3987 cells
row slice first, then column selection: 95 x 2 = 190 cells
column selection first, then row slice: 1329 x 2 = 2658 cells in the intermediate table, then 95 rows
if a single column is selected the result is a series: 95 values, total 4348.3 m3
```

Label-based access does not work on a single axis: the same expression names both the row and the column.
This shortens things because it combines two separate selections in one step, but it slips an order decision
in between. If the row slice is applied first, **190 cells** enter the result; if the column selection is
applied first, an intermediate table of 2,658 cells is built and the row slice is taken from it. The result
is the same, the table built is not. In a small frame this difference is immaterial; when the scale grows,
the item measured by the `scale-and-performance` topic is exactly this.

One detail also changes the **kind** of the result. If two columns are requested, what comes back is a
frame; if a single column is requested, what comes back is a series. When the same spelling takes a
one-element list of columns instead of a column name, the type of the result changes, and the code that
follows has been written against either the series interface or the frame interface. How many columns the
selection returns is therefore not only a number but a type decision.

## When the Order Breaks

```python
# broken_order.py — the same label slice, in an index whose order has been broken.
# What breaks the order is not an error but a legitimate sorting decision.
sorted_by_total = sorted(summary, key=lambda s: -s["total_m3"])
broken = LabelIndex([s["subscriber_id"] for s in sorted_by_total])
print(f"frame sorted by total: is the label index sorted: "
      f"{'yes' if broken.is_sorted else 'no'}")
bi, bj = label_slice(broken, START, STOP)
print(f"the same slice {START}..{STOP}: {START} is at position {bi}, {STOP} is at position {bj - 1}, "
      f"slice {max(bj - bi, 0)} rows")

absent = next(e for e in range(START, STOP) if e not in index.position)
gi, gj = label_slice(index, START, absent)
print(f"\nend {absent} not in the index: in the sorted index the slice returns {gj - gi} rows")
try:
    label_slice(broken, START, absent)
except KeyError as error:
    print(f"the same end, in the unsorted index: KeyError {error}")

print(f"\npasses: scan without index {linear_passes}, binary search in the label index {label_passes}")
print(f"code  : linear slice 6 lines, label slice 1 line")
print(f"rows  : the same two ends return {label_rows}, {(j - 1) - i}, "
      f"{len(summary[START - 10001:STOP - 10001])} and {max(bj - bi, 0)} rows in four spellings")
```

```
frame sorted by total: is the label index sorted: no
the same slice 10100..10200: 10100 is at position 492, 10200 is at position 1108, slice 617 rows

end 10122 not in the index: in the sorted index the slice returns 22 rows
the same end, in the unsorted index: KeyError 10122

passes: scan without index 2658, binary search in the label index 21
code  : linear slice 6 lines, label slice 1 line
rows  : the same two ends return 95, 94, 100 and 617 rows in four spellings
```

Sorting the frame by total consumption is a legitimate request and brings the biggest consumer to the top.
The side effect it has is that the label index stops being sorted. In an unsorted index a label slice can no
longer mean "the labels between 10,100 and 10,200", because such a range does not sit together anywhere in
the table. Its meaning is forced to shift to this: the rows between the positions where the two ends are
found. The result is **617 rows**, and the labels of most of these rows lie outside the range.

The same expression, the same two ends, 617 rows instead of 95. There is no error message between the two,
because both conform to the definition of the slice; what changed is the index. This is one of the clearest
examples of the rule of the course: an abstraction shortens the operation but does not make visible the
assumption it works on.

An end that is not found produces two different behaviors in the two indexes, and here the difference
reverses direction. Subscriber 10,122 is not in the index. In the sorted index this is not a problem: binary
search finds the position where that label **would be** and a slice of 22 rows is returned. In the unsorted
index that label has no position and the slice raises an error. So the abstraction does not stay silent here,
it speaks; but the case in which it behaves loudly is the case of **missing information**, not of a wrong
result. In the 617-row case, where it stays silent, it has produced an answer.

## Summary

- Label-based access finds the row by its name, position-based access by its order; a label slice includes
  the last end, a position slice leaves it out.
- The same two ends return 95, 94, 100 and 617 rows in four spellings; the one-row difference between the
  first two is the record at the boundary of the slice.
- If the index is not continuous, mistaking the label for a position slides silently: because of 70 gaps the
  label 10,100 is at position 95, and the slice written by position returns the wrong 100 of 100 rows.
- In a non-unique index a label slice selects rows, not subscribers: in the long table of 3,199 rows the same
  range brings 230 rows.
- The gain of the index is read in pass count: a linear scan takes 2,658 comparisons while binary search
  reaches the same 95 rows in 21 passes; the hidden default is that a label slice assumes the index is sorted.

## Next Step

Slicing gives a range, but what is wanted in analysis is not a range but a condition: "the readings whose
consumption is above 25 m³". A conditional selection arises from a comparison, and a comparison assumes
that every value in the dataset is comparable. It had been seen that the meter reading column has 111 empty
cells and 80 cells with "n/a" written in; a value whose being below or above a threshold is unknown falls on
which side of the filter. The next lesson runs the same condition in three separate spellings and counts how
many rows each of the three returns.