Expected MMSI
The library label remains visible beside every prediction so errors can be inspected directly.
Vessel Identity · Model evaluation
The batch verifier runs an active vessel-identity profile against available validation and test recordings, preserves the expected MMSI and exposes the best alternative—not just a single headline accuracy number.

Verification workflow
Leakage protection: if an encounter contains any training recording, all validation or test clips from that encounter are excluded automatically.
Per-file evidence
The library label remains visible beside every prediction so errors can be inspected directly.
The strongest model match is reported as an MMSI, or as Unknown vessel when it falls below the cutoff.
The score measures proximity to the learned vessel prototype in the model’s acoustic embedding.
The runner-up MMSI and score reveal close competitors that a simple correct/incorrect label would hide.
Validation or test origin and encounter ID remain visible for every evaluated recording.
Unreadable or incompatible files are marked as errors without concealing the rest of the batch.
Summary metrics
| Output | Question answered | How to use it |
|---|---|---|
| File accuracy | How many individual clips received the expected MMSI? | Useful for reviewing difficult windows, but adjacent clips are correlated. |
| Encounter-majority accuracy | How many independent passages were identified by the majority of their clips? | Use as the more conservative operational summary. |
| Unknown results | How often was no candidate similar enough? | Tune the cutoff while balancing false identities against missed known vessels. |
| Confusion counts | Which expected vessels are repeatedly mapped to which alternatives? | Find acoustically similar identities and under-represented training classes. |
| Alternative score | Was the winner clear or nearly tied? | Prioritise close decisions for listening and signature comparison. |
Unknown-vessel rejection
Closed-set accuracy assumes every recording belongs to a vessel represented during training. Real deployments contain untrained vessels, overlapping traffic and non-vessel sound. Analyze therefore applies an adjustable similarity cutoff.
Requires a closer match and reduces unsupported identity assignments, but may reject more recordings from known vessels.
Returns a known candidate more often, but increases the chance of confidently assigning unfamiliar audio to the wrong identity.
The cutoff should be selected using independent known-vessel and out-of-library recordings representative of the deployment environment.
Good evaluation practice
No. They provide useful acoustic variation within an encounter but do not replace independent passages recorded at different times, ranges and operating states.
No. It indicates similarity to a learned prototype. Confounding vessels, hydrophone response and environmental noise still require review.
Repeated near-ties expose pairs that need more training encounters, better separation or a coarser class such as vessel type.
Yes. Named profiles and model history allow the same held-out material to be rerun after retraining or restoration.
Continue exploring
We can walk through thresholds, alternatives, confusion patterns and encounter-level results.