Skip to content
academia.sh
tr
Log in

Course Advanced

Natural Language Processing and Computer Vision

Published Updated

By the end of this course

Start course

01

Natural Language Processing

Counting what each decision adds when short Turkish field notes are turned into numbers: three tokenization decisions yielding three different vocabularies and out-of-vocabulary rates, and the agglutinative structure making stemming hard; what bag of words adds to hand-counted text features and what term weighting adds on top of it; embeddings being learned from context and analogy most often landing on the most frequent word; the same output giving a high number at the token level and a low one at the entity level, and the surprising height of the baseline that writes the most frequent label on every token; the share of each step in a classification pipeline; an embedding pretrained with masked prediction pulling ahead as labeled examples decrease; the high number that a procedure copying the input gets on an overlap-based metric, and the same output receiving four different rankings on four metrics.

  1. 01 Text Preprocessing Measuring how much the decision moves the number when short Turkish field notes are split into tokens: the domain baseline of splitting on whitespace gives a vocabulary of 748 tokens and an out-of-vocabulary rate of 0.0565, while dropping punctuation and lowercasing brings the vocabulary to 669 and adding stemming brings it to 641 and the rate to 0.0557. 0.9467 of the out-of-vocabulary tokens are numbers; when numbers are folded into a single token the vocabulary becomes 203, 124 and 96. The rule-based stemmer splits 85 roots into 103 stems, scatters 19 roots over several stems and merges two roots in one stem; table-based lemmatization cuts the vocabulary to 82 but has a coverage of 0.9967 and can do nothing with a form it does not recognize.
  2. 02 Numerical Representation Counting what each decision adds when short field notes are turned into columns, with the same model and the same held-out set: while the domain baseline of five hand-counted numbers gives 0.7383 and the most frequent class stays at 0.7017, a bag of words of 96 stems reaches 0.8217 and passes the line by 0.0833. Vocabulary size adds contribution from 8 to 64 and adds exactly zero from 64 to 96. Raw counts and inverse document frequency weighting do not move the accuracy at all (0.0833), dropping stop words loses 0.0067, and the ceiling that knows the event kind is 0.8250 — the bag is 0.0033 below the ceiling.
  3. 03 Word Embeddings Giving every item of the vocabulary a vector learned from context and counting neighborhood against the setup's own phrase pools: while the expected rate of a random neighbor is 0.2543, the raw co-occurrence vector gives 0.4938 and the eight-dimensional learned embedding gives 0.6049. The total number misleads — 49 of the 81 forms are rare nouns that occur in a single pattern, and there even raw counts give 0.7755, while on frequent forms random is 0.0462, raw counts 0.0625 and the learned embedding 0.2500. Analogy holds in 0.0972 of 72 tests, and the ones that do not hold land not on the most frequent word but on the forms with the narrowest context.
  4. 04 Sequence Labeling Taking the label from the document and giving it to every token separately, and measuring what accuracy counts: writing the most frequent label to every token, the domain baseline, gives a token accuracy of 0.5061 but an entity-level F1 of 0.0000 — not a single entity is found. The procedure that looks at the last three characters gives the same token accuracy of 0.9824 under both labeling schemes while it gives 0.9267 and 0.9600 at the entity level; the procedure that memorizes the token has a token accuracy of 1.0000 and stays at 0.9665 in the type scheme. In part of speech the baseline of the same form gives 0.5473 and does not mislead there.
  5. 05 Text Classification Giving a single event kind decision to the whole of a field note and counting separately how many points each of the three steps of the pipeline adds: the most frequent class baseline is 0.4420, seven hand-counted measures give 0.5500, the bag of words with a linear model gives 0.9620. Measured over twelve setups, the representation step adds 0.3787 on average, the classifier step 0.0500, term weighting 0.0140; both tokenization decisions give a negative contribution, and suffix clipping costs 0.0165 on average while bringing the vocabulary down from 109 to 89. The contribution of a step is not a single number: weighting rises to 0.0340 with the class mean and falls to minus 0.0060 with the linear model.
  6. 06 Pretraining and Fine-Tuning Carrying over to the labeled task an embedding table learned by masked prediction from unlabeled notes, and reading the difference it adds together with the number of labels: the proxy task raises its own baseline of 0.0419 to 0.3450 and spends zero labels for it. While the domain baseline is 0.4420, training from scratch with twenty labels gives 0.5850, pretraining and fine-tuning give 0.7480; training from scratch reaches this number only with forty labels. With one hundred sixty labels the difference turns to minus 0.0090. The control setup that does not learn — freezing a random table — stays below 0.51 in all five measurements and its two runs differ by 0.1320; the two runs of the pretrained table differ by 0.0020.
  7. 07 Text Generation and Summarization Counting what the metric rewards on tasks where the output is text rather than a label: a procedure that gives the input back as it is and has learned nothing gets 1.0000 on the recall side of the overlap-based metric and ranks first among four procedures. The same procedure gets 0.6362 on the precision side and 0.7777 in balanced form, and falls to third place. When eight unrelated words are appended to the end of the copy, recall stays at 1.0000 while precision drops from 0.6362 to 0.2700. The generator procedure, which reports the content of the reference correctly but writes it with other words, gets 0.0684 in balanced form and 0.0114 on 2-grams; on the content check it is second with 0.9320.
  8. 08 Natural Language Processing Metrics Reading a single output at four metrics and the four metrics giving four different rankings: the same four systems rank B > A > C > D at the token level, A > B > C > D at the entity level, C > B > A > D at the document level and D > B > A > C on the recall side of overlap, and the first of each metric is a different system. Each metric is read against its own baseline: the most frequent label at the token level 0.3970, the most frequent class at the document level 0.4420, marking no span at all at the entity level 0.0000, marking the whole input on the recall side of overlap 1.0000. In the last row the difference between the baseline and the best system is exactly 0.0000.

02

Computer Vision

Measuring each task against its own baseline on fictional images generated from the coverage grid: the contribution of normalization and augmentation and the transform that breaks the label in augmentation, end-to-end training against the mean image baseline and how many examples it takes for transfer to add something, the number a fixed box placed in the middle of the image gets on the overlap metric and the fact that both the method and the fixed box drop as the threshold rises from 0.3 to 0.7, the baseline that assigns every pixel to the dominant class scoring high because of pixel imbalance and semantic segmentation being unable to separate two adjacent objects, the inconsistency of independent per-frame decisions against a decision that uses time across consecutive frames, the success of a joint representation that matches a note with its image against random matching, and the share a single modality gives on its own.

  1. 01 Image Data and Preprocessing Measuring how much the way a pixel is turned into a number carries: when the same two procedures are read on three representations, dataset-wide scaling moves no number (0.5500 and 0.6937 stay as they are), while per-image normalization raises the threshold rule from 0.6937 to 0.9250 and lowers the mean image baseline from 0.5500 to 0.4875. Of five augmentation transforms, the quarter turn flips the label in 160 of 160 examples, and the shift breaks it in 1. With forty labels, nearest neighbor reads 0.5938 and 0.6250 without augmentation, two flips read 0.6875 and 0.6250, and a quarter turn added with its label kept reads 0.5188 and 0.5000: below the 0.5312 of the most frequent class.
  2. 02 Image Classification Measuring end-to-end training and transfer learning against the domain baseline: while the most frequent class reads 0.5312 and the mean image 0.5500, the network with 1,169 parameters reads 0.6000 in both runs on 144 pixels, and the network with 305 parameters reads 0.6687 and 0.7063 on 36 pooled numbers; both memorize the training set completely. With twenty labels, training from scratch stays below the domain baseline at 0.5188 and 0.5250, while freezing a pretrained model produced from a fictional source task gives 0.6312 and 0.5500, and fine-tuning gives 0.5938 and 0.5625. At sixty labels the advantage closes, and at one hundred sixty training from scratch pulls ahead with 0.6125 and 0.6875; the gap of 0.0812 between the two runs of freezing is larger than the difference transfer adds.
  3. 03 Object Detection Reading intersection over union and its threshold together with a procedure that looks at no image: a fixed 4x4 box placed in the middle of the grid gets a mean overlap of 0.3011 and counts as a detection in 0.5125 of the test images at a threshold of 0.3, but falls to 0.1938 at a threshold of 0.5 and to 0.0125 at 0.7. A fixed-size window with 81 candidates that searches only position reads 0.9750, 0.6875, and 0.2375 with a mean of 0.5687; a window with 2,025 candidates that also searches size reads 0.9812, 0.9625, and 0.8500 with a mean of 0.9076. The two procedures are separated by 0.0062 at a threshold of 0.3, and the gap between them rises to 0.6125 at a threshold of 0.7. When the confidence threshold is pulled to the score of the first quarter of the validation set, precision rises from 0.9812 to 1.0000 and recall drops to 0.7063.
  4. 04 Segmentation Reading the decision at the pixel level against its own baseline: because 2,193 of the 17,280 pixels in the test set are object, the baseline that assigns every pixel to the dominant class gets 0.8731 in pixel accuracy, while the object intersection over union of the same output is 0.0000. A linear model deciding from a single pixel value gives 0.9540 and 0.6412, and the same model with a 3x3 neighborhood window gives 0.9851 and 0.8843; the difference comes not from the model but from the representation. 45 of the one hundred twenty images carry adjacent blocks, and even with a perfect mask, counting connected components gets the instance count wrong in these 45 images; rectangle decomposition fixes 24 of them and stops at 21, while on the predicted mask the wrong count rises from 46 to 79.
  5. 05 Video and Temporal Models Counting the instability over time of the per-frame decision: on the 432 frames of thirty-six test sequences, the baseline that writes the dominant class into every frame gives 0.7014 accuracy and 0 instability, while the true label sequences carry 71 transitions. The independent frame decision produces 99 transitions with 0.8634 accuracy, which is 28 more than the truth. When a transition penalty is applied to the same scores, accuracy rises to 0.8657 at a penalty of 0.05, instability settles on the truth's 71 at a penalty of 0.15, and at a penalty of 4.00 the decision falls to the baseline itself, 0.7014 and 1 transition: on this metric the domain baseline is not a separate procedure but the extreme value of the penalty.
  6. 06 Multimodal Models Measuring a joint representation, which brings the note and the coverage frame of the same event together in a single space, against its own baseline: in rooms of twenty-five candidates, random matching gives 0.0400 while the joint representation learned with a contrastive objective gives 0.1660, and the ceiling of the shared information carried by the fiction is 0.2659. On the event kind decision, the most frequent class baseline is 0.4420, note only 0.8940, frame only 0.6140, joint representation 0.8660: the representation that wins at matching stays below what the note alone gives in classification. The course wrap-up collects the domain baseline of fourteen lessons, the method's number, and the difference in a single table.

Final exam

Natural Language Processing and Computer Vision — Final Exam

In preparation

This course's question bank is not complete yet. The exam opens here automatically once enough questions are added.

Start typing to search.

↑↓ ↵ Esc navigate · open · close