stn 04 · rec 2024 · universidad de los andes — coursework
Shipping a 120-Breed Dog Classifier End to End
Built a 120-breed dog classifier the whole way out — scraper, two transfer-learning architectures, a FastAPI service, a Cloud Function and a Streamlit front end — then found the pretrained backbone had already solved it before the first epoch finished.
sec 01 · brief
The problem
Fine-grained image classification is the part of computer vision where the classes are not dog and cat but a hundred and twenty breeds of dog, most of which differ by the set of an ear and the length of a coat. It is the standard demonstration that a classifier has learned features rather than silhouettes, and it has historically taken two things a student does not have: a very large labelled dataset, and enough compute to train a deep network on it from scratch.
Transfer learning is the answer to the second problem and pretrained backbones are the answer to the first, which is what makes the task assignable at all. The published work sets the bar: three papers in the literature review report 96.75%, 93.53% and 93.2% on this family of problem, all of them using pretrained convolutional networks or vision transformers over the same Stanford Dogs dataset.
The brief was not to beat any of them. It was to carry one model the entire distance — dataset to deployed endpoint — because the gap between a notebook that reports an accuracy and a service that answers an HTTP request is where most student projects stop, and it is the part that is actually engineering.
sec 02 · approach
What I built
A pipeline, not a model. Six pieces, each of which could have been the whole assignment: a Selenium scraper for additional imagery, a side trial of OpenAI's Vision API as a zero-shot baseline, two transfer-learning architectures over Stanford Dogs, a FastAPI service for local inference, a Google Cloud Function that lazily pulls its weights from a storage bucket on cold start, and a Streamlit front end that talks to it. The classifier is the middle of the pipeline rather than the point of it.
Two backbones, two different heads, and the difference was hardware. InceptionV3 (2015, 42 layers, ~23.9M parameters) and InceptionResNetV2 (2016, 164 layers, ~55.9M parameters) were both loaded frozen with a small trainable head appended. The heads are not the same size, and the paper is direct about why: with no GPU available, the larger backbone filled the notebook's memory, so its head was cut from two 240-unit layers to two 120-unit ones. The architecture was chosen by what would fit.
The Vision API trial was scoped down to the only thing it could measure. The original intent was to compare a hosted multimodal model against the trained classifier, and that turned out to be impossible by construction: there is no way to constrain the model's output to the 120 Stanford Dogs labels without fine-tuning, so the two systems cannot be scored on the same task. What remained measurable was cost, so cost is what was measured.
sec 03 · method
Method
01
Auxiliary dataset and scraper
Three Selenium functions: one walks a Google Images result set and opens each result to recover the image at original scale rather than the thumbnail, one downloads through Pillow, and one iterates a list of canine Instagram accounts and pulls their photographs. The target was deliberately unusual — several hundred images of a small number of individual named dogs, rather than many images of many breeds — because the question behind it was whether a model could tell two dogs of the same breed apart. That set never entered training; the classifier was fit on Stanford Dogs alone, and the paper says so plainly.
02
Vision API trial
One hosted vision-preview model, called three ways: a single image at low detail on a 65-token budget, a single image at high detail where the model generates 512-pixel crops on a 129-token budget, and a batch of six images against one prompt. Responses came back as paragraphs of hedged prose rather than labels, so a second call to a text model parsed each one down to breed names, and both calls' token usage was written back to the dataframe. Because the label space could not be constrained, no accuracy was computed and none is claimed.
03
Dataset and augmentation
Stanford Dogs: 120 categories, 20,580 images at 256×256, built from ImageNet photographs and annotations. A single Keras image generator — rescale 1/255, rotation ±15°, zoom 0.2, width and height shift 0.2, horizontal flip — with a 0.2 validation split, flowed twice from the same directory to give 16,508 training and 4,072 validation images across the 120 classes. Batch size 256, which at integer division gives 64 training steps and 15 validation steps per epoch.
04
Two transfer-learning heads
Each backbone loaded with its ImageNet weights and frozen, then global average pooling, flatten, and an alternating stack of dropout and dense layers ending in a 120-way softmax. InceptionV3 took two 240-unit layers and was compiled with SGD at learning rate 0.01 and momentum 0.99 against categorical cross-entropy. InceptionResNetV2 took two 120-unit layers and was compiled with Adam. Both ran 20 epochs. Neither ran on a GPU: the notebook's own device query returned an empty list, which is the constraint the paper describes, recorded by the artifact rather than merely asserted.
05
Local inference service
A FastAPI service that accepts an uploaded file, decodes it through PIL into a numpy array, expands it to a batch of one, runs prediction, and returns the predicted breed with its confidence. CORS middleware is configured for a browser front end on a different origin. This is the version that holds the weights in process and answers immediately.
06
Cloud deployment and cold start
The same inference path rebuilt as a Google Cloud Function, with the difference that matters: the model is not bundled. On a cold start the function downloads the weights from a Cloud Storage bucket into the runtime's temp directory and caches them in a module-level global, so the first request after a scale-to-zero pays the download and the rest do not. A Streamlit front end sits in front of it — upload a photograph, it goes out as an array, the breed and the confidence come back.
sec 04 · legend
Instruments
legend · modelling
legend · data
legend · serving
legend · trialled
sec 05 · readings
Results
Breeds classified
120
Stanford Dogs, all classes
Images
20,580
16,508 train · 4,072 validation
Best validation accuracy
86.64 %
epoch 10 of 20 · InceptionResNetV2
Validation accuracy, epoch 1
81.04 %
before the head had learned anything
Net change over 20 epochs
−0.01 pt
81.04 % → 81.03 %
Validation accuracy, spread
80.17 – 86.64 %
20 epochs · median 83.62 %
Runs with recoverable metrics
1 of 2
the InceptionV3 log was not retained
Training wall clock
≈ 4 h 24 m
20 epochs × 64 steps · no GPU
Vision prompt tokens, single image
111
1 retained call, low detail
Vision prompt tokens, per image batched
89.3
150 calls · 536 ÷ 6 images
plate 01
Twenty epochs, one flat line
epoch 01
epoch 02
epoch 03
epoch 04
epoch 05
epoch 06
epoch 07
epoch 08
epoch 09
epoch 10
epoch 11
epoch 12
epoch 13
epoch 14
epoch 15
epoch 16
epoch 17
epoch 18
epoch 19
epoch 20
Validation accuracy after each of the 20 epochs, for the one run whose training log survives. It opens at 81.04% and closes at 81.03%. It passes through 86.64% at epoch 10 and 80.17% at epoch 16 and arrives back where it started, which is what a frozen backbone with a converged head looks like: the features were fit before training began, and the head had nothing left to learn after the first pass.
source
Keras training log retained in the committed notebook, InceptionResNetV2 run. The axis runs the full 0–100 because compressing it to 80–87 would manufacture a trend out of six points of noise — the flatness is the reading, so the axis has to be able to show it.
plate 02
What the literature reports, and what this run did
Borwarnginn 2019
Shah 2020 (a)
Wang · ViT
Shah 2020 (b)
this run
The three accuracies the paper's own literature review cites, against the best epoch of the run measured here. The comparison is not like for like and the gap is not the point — those figures come from different splits, different protocols and in two cases different datasets, and 86.64% is a best epoch on a validation split rather than a held-out test result. It is charted because the paper's conclusion claims high precision without ever printing a number, and this is the honest version of that claim.
source
Literature figures as cited in the course paper; the last bar is the notebook's retained training log. Direct-labelled because the four published bars are within six points of each other and length alone cannot separate them.
sec 07 · findings
What the data said
finding · negative result
Twenty epochs of training moved the number by one hundredth of a point
Validation accuracy at epoch 1 was 81.04%. At epoch 20 it was 81.03%. In between it wandered as high as 86.64% at epoch 10 and as low as 80.17% at epoch 16, with a median of 83.62% and no trend in any direction. The entire run — roughly four and a half hours of CPU — bought a net −0.01 points, and the peak it passed through at epoch 10 was not held, not selected for, and not saved.
finding · result
The backbone had already solved it, and the two datasets share photographs
An 81% top-1 accuracy across 120 classes after a single epoch is not a model that learned quickly; it is a model that did not need to learn. The frozen backbone was pretrained on ImageNet, and Stanford Dogs is built out of ImageNet images and annotations — so the features were fit on the same photographs the classifier is scored against. Everything above chance was already present in the frozen layers before the head was initialised.
finding · negative result
The one run with surviving numbers had its loss function misconfigured
The model ends in a 120-way softmax and was compiled with a cross-entropy loss set to expect logits. Those two statements contradict each other: the flag tells Keras the outputs are unnormalised scores and to apply its own softmax, so the loss was computed on a double-softmaxed distribution. Gradients still point in a usable direction, which is why the run trains at all, but the reported loss values do not mean what they appear to mean. The 86.64% figure is real; the loss curve beside it is not interpretable.
finding · negative result
The two runs failed in opposite directions, and neither is complete
The InceptionV3 run's training log was not retained in the committed notebook — its cells were re-executed or cleared — so the paper's claim that both architectures reached similar results cannot be checked against anything. The InceptionResNetV2 run kept its full log but its weights were never committed. The repository ships the model it has no numbers for and the numbers for the model it does not ship.
finding · negative result
The paper reports no result for either of its own models
Its results section is two paragraphs. It states that the models ran for 20 epochs, that the results were similar, and that the gap between the training and validation curves suggests overfitting, smaller for the deeper backbone. It prints no accuracy, no loss, no precision, no recall and no confusion matrix. The only percentages anywhere in the document belong to other people's models. Every performance figure on this page had to be recovered from the notebook.
finding · negative result
The scraper was built, and never used for what it was built for
Several hundred images of individually named dogs were collected specifically to test whether a model could distinguish two dogs of the same breed. That capability was never tested — the paper says so directly — and the images never entered training either, because the classifier was fit on Stanford Dogs alone. What the stage produced was a working scraper and an unused dataset.
finding · result
The comparison the trial existed for was impossible before it started
A hosted multimodal model cannot be scored against a 120-class classifier unless its output can be constrained to the same 120 labels, and without fine-tuning it cannot be. The trial returned free prose that needed a second model call just to parse into breed names. Recognising that the accuracy comparison was unavailable and re-scoping to the one dimension that was measurable — token cost — is the correct move, and batching six images against one prompt cut prompt tokens per image from 111 to 89.3.
finding · negative result
The deployed endpoint no longer answers
The Streamlit application the paper links is no longer reachable: the URL returns a 303 redirect to a sign-in page rather than the app. The Cloud Function behind it, and the storage bucket it pulled weights from, are not verifiable from outside either. The deployment is documented in the source and in the committed function, and it is not currently demonstrable.
sec 08 · forward
Recommendations
01
Fix the loss configuration and re-run before trusting any curve. A cross-entropy set to expect logits, sitting behind a softmax output layer, is a one-line correction — and until it is made, the only defensible number from that run is the accuracy.
02
Checkpoint on best validation accuracy. The run passed through 86.64% at epoch 10 and ended at 81.03%; saving the best epoch rather than the last would have kept the better model for free.
03
Unfreeze the top of the backbone, or stop training after one epoch. Those are the two honest options a flat curve leaves. A frozen backbone whose head has converged by epoch 1 has nothing left to learn, so the compute spent on epochs 2 to 20 either goes to fine-tuning real layers or is not spent.
04
Report on a held-out test split, not the validation split. The current figures are the best epoch measured on the same split used to watch training, which is the number most likely to flatter.
05
Commit the training history, not just the weights. A history dump is a few kilobytes and would have made the second run's numbers recoverable instead of lost.
06
Use the individual-dog dataset that already exists, or delete it. The scraper collected the right data for a re-identification task and the task was never attempted. That is either the natural second phase or dead weight in the repository.
sec 09 · notes
Notes
disclosure
Nothing on this page is pseudonymized. This is coursework, not client work: the university, the dataset, the architectures, the three repositories and the deployed application are all named as they are. The student ID printed on the paper and the university email address attached to every commit are deliberately excluded — they identify a person and inform no reader.
limits
- There is no test split. Every accuracy figure on this page is measured on the validation split, which is also the split used to monitor training. The best-epoch figure is the most optimistic reading available of an already optimistic measurement.
- Validation images were augmented. One image generator carrying rotation, zoom, shift and horizontal flip is flowed twice from the same directory, once per subset — so the validation set was measured through the same random transformations as the training set. That is not standard practice, and it makes these figures not directly comparable to any published Stanford Dogs number, including the four in plate 02.
- With a batch size of 256 and step counts computed by integer division, each epoch evaluates 3,840 of the 4,072 validation images and trains on 16,384 of the 16,508. The remainders are dropped every epoch.
- Only one of the two runs has any recoverable figures, so nothing here compares the two architectures. The paper's claim that they performed similarly is unverified and is reported as a claim, not as a result.
- The loss values from the surviving run are not interpretable — the loss function was configured for logits and given probabilities.
- Epoch wall-clock times in the log alternate between roughly 25 minutes and roughly 35 seconds, ten of each, with no corresponding change in step count. The ≈4 h 24 m total is the sum of what the log reports; the alternation is unexplained, and the notebook does not say why.
- The Vision API trial produced no accuracy of any kind, by construction. Its token figures rest on 150 calls in one mode and a single call in the other.
- The deployment is documented and not currently reachable.