Object Detection Dataset Checklist: Before You Label Image #1

Most annotation projects do not fail in the labeling; they fail in the decisions nobody wrote down before labeling started. A class added in the middle re-numbers every YOLO label. A missing guideline turns "tiny box" into a debate on image 3,000. This checklist is the pre-flight for a detection dataset — ten minutes now, days saved later.

1. Freeze the class list (and how to grow it)

• Write the exhaustive class list with a one-line definition each ("person = visible human body or major part, excludes reflections").
• Decide now what is explicitly not labeled (reflections, posters, screens showing people?) — unwritten rules become inconsistent labels.
• If your export is YOLO txt, plan to append new classes at the end of classes.txt forever; the format indexes classes positionally.

2. Coverage beats volume

• Each class needs examples across: scales (small / medium / large), lighting, angles, occlusion levels and backgrounds.
• Build a tiny coverage matrix (class × condition) and collect images against the empty cells, not against round numbers.
• Rare classes starve: 200 well-spread examples of one class beat 2,000 of another plus 50 of it.

3. Image hygiene

• Resolution: keep native resolution for small objects; downscaling before labeling throws away the pixels your model needs.
• Duplicates: burst shots and video frames inflate the dataset and leak between splits. Deduplicate before labeling, not after.
• Naming: keep original unique names or a stable scheme; sidecar annotation files are matched by filename, so renaming mid-project means touching every label.

4. Splits before labels

Decide the train/val split now — by folder or a manifest — and make it leakage-aware: frames of the same video, same session or same physical object must not straddle splits. Labeling first and splitting later is how test sets quietly become training data.

5. Budget the QA, not just the labeling

Plan for a second pass from day one. At minimum, run automated checks before training: out-of-bounds boxes, implausibly tiny boxes, same-class duplicates, unknown classes and format conflicts. FlyLabelImg's checker runs these rules over the whole folder in one click, and can re-run the AI model over annotated images to surface likely misses — a cheap "second pair of eyes" pass. The review-first drafting workflow also means your first pass already produced judged, confirmed boxes rather than raw output.

6. Label in reviewable batches

• Label 100–500 images, then stop and audit: class definitions, edge cases, the guideline questions that came up.
• Write the answers down — the guideline document is part of the dataset.
• Only scale up when a batch passes the checks with no debates. AI pre-annotation makes the batch loop fast enough that the audit, not the drawing, is the bottleneck — which is exactly where you want the bottleneck.

7. Format and tooling

• Pick the export format your training framework wants (see VOC vs YOLO) and prefer a tool that stores sidecars next to the images — datasets stay portable.
• Keep the annotation files under version control with the images, or at least back them up on the same schedule.

FAQ

How many images do I need per class?

Rule of thumb for fine-tuning a modern detector: several hundred well-distributed examples per class is a reasonable starting target, but coverage diversity matters more than the raw count. Start with 200–500, evaluate, and add against the coverage matrix, not the total.

Should I label every object in every image?

Consistency beats completeness if the guideline says so — but a dataset that randomly omits instances of a class teaches the model that the class is uncertain. If you skip instances of a declared class systematically (e.g. only label nearest objects), write that rule down.

Can AI label the whole dataset for me?

It can draft it. Treat model output as a first pass to judge, not as ground truth — and use a tool whose dedup keeps your confirmed boxes intact when you re-run a better model later.