Coastal & Marine Spatial Analysis Pipelines

Detecting Dark Vessel Gaps in AIS Tracks

A vessel stops reporting for three hours and reappears forty miles away. That is either a transponder switched off, a vessel that sailed out of reception, a satellite revisit gap, or a receiver outage — and in the message stream all four look like the same absence of messages. The discrimination is possible, it uses data the feed already carries, and it is never conclusive from one signal. This page builds the evidence set, within the anomaly detection workflow.

Root cause: absence is not a message, so it carries no metadata

Every other AIS anomaly is a message that says something implausible. A gap is the absence of messages, which has no MMSI, no timestamp and no receiver ID. Everything known about it has to be inferred from what surrounds it: the last fix before, the first fix after, and what every other vessel was doing meanwhile.

That third source is the one most pipelines never consult, and it is by far the strongest. A receiver outage or a coverage hole affects everything in an area; a switched-off transponder affects one vessel while its neighbours report normally throughout.

import pandas as pd


def neighbours_reporting(df: pd.DataFrame, centre, t0, t1, radius_m: float = 50_000):
    """How many other vessels reported inside a radius during the gap window."""
    window = df[(df["timestamp"] >= t0) & (df["timestamp"] <= t1)]
    if window.empty:
        return 0, 0
    d = window.geometry.distance(centre)
    near = window[d <= radius_m]
    return near["mmsi"].nunique(), int(len(near))

Two vessels reporting throughout a three-hour gap is weak evidence; forty vessels reporting throughout it is strong. The count matters as much as the fact.

Four Causes, Three Pieces of Evidence A decision table with four rows, one per cause of an AIS gap, and three evidence columns. For a receiver outage, no neighbours reported during the window, the gap did not begin at the reception horizon, and the re-appearance is reachable; the verdict is an infrastructure event affecting everyone. For a vessel leaving reception, neighbours near the vessel also stopped while neighbours further inshore continued, the gap began at the horizon, and the re-appearance is reachable; the verdict is coverage. For a satellite revisit gap, all vessels in the footprint stopped simultaneously and resumed simultaneously, the horizon is irrelevant, and the re-appearance is reachable; the verdict is a scheduling artefact of the constellation. For a switched-off transponder, neighbours reported normally throughout, the gap began well inside coverage, and the re-appearance may or may not be reachable; the verdict is that this is the only row consistent with intent. A footer notes that no single column separates all four and that the combination does. cause neighbours during the gap started at the horizon? re-appearance receiver outagenone reportednoreachable left receptionnearby stopped tooyesreachable satellite revisitall stopped togethern/areachable transponder offreported normallynosometimes not No single column separates all four. The combination does. The shaded row is the only one consistent with intent, and it is identified by what its neighbours did — which is the one signal a per-vessel gap detector never looks at. Collecting all three costs one spatial query per gap, against data the pipeline already holds.

Step-by-step detection with production code

1. The expected interval, not a fixed threshold

The data quality topic sets out why a single gap threshold is wrong: a moored vessel legitimately reports every three minutes and a fast one every two seconds. The gap detector needs the same state-dependent expectation.

import logging

logger = logging.getLogger(__name__)

def expected_interval_s(nav_status: int, sog_kn: float, is_class_b: bool) -> float:
    """Nominal AIS reporting interval for this vessel state."""
    if is_class_b:
        return 180.0 if sog_kn < 2.0 else 30.0
    if nav_status in (1, 5):            # at anchor, moored
        return 180.0
    if sog_kn > 23.0:
        return 2.0
    if sog_kn > 14.0:
        return 6.0
    return 10.0


def find_gaps(track: pd.DataFrame, *, tolerance: float = 20.0) -> pd.DataFrame:
    """Intervals exceeding `tolerance` times the expected reporting interval."""
    t = track.sort_values("timestamp")
    dt = t["timestamp"].diff().dt.total_seconds()
    expected = [expected_interval_s(s, v, b) for s, v, b in
                zip(t["nav_status"], t["sog_kn"], t["is_class_b"])]
    ratio = dt / pd.Series(expected, index=t.index)
    gaps = t[ratio > tolerance].copy()
    gaps["gap_s"] = dt[ratio > tolerance]
    gaps["gap_ratio"] = ratio[ratio > tolerance]
    logger.info("%d gap(s) beyond %.0f× the expected interval", len(gaps), tolerance)
    return gaps

Expressing the threshold as a multiple of the expectation rather than in minutes is what makes one detector work across the fleet. A three-minute silence from a vessel doing 24 knots is ninety times its expected interval; the same silence at anchor is one.

2. Score, with the evidence attached 

from dataclasses import dataclass, field


@dataclass
class GapAssessment:
    mmsi: int
    gap_s: float
    neighbours: int
    started_inside_coverage: bool
    reappearance_speed_kn: float
    factors: list[str] = field(default_factory=list)

    @property
    def score(self) -> float:
        s = 0.0
        if self.neighbours >= 10:
            s += 0.4; self.factors.append(f"{self.neighbours} neighbours reported throughout")
        if self.started_inside_coverage:
            s += 0.3; self.factors.append("gap began well inside reception")
        if self.gap_s > 3600:
            s += 0.1; self.factors.append(f"gap of {self.gap_s / 3600:.1f} h")
        if self.reappearance_speed_kn > 25.0:
            s += 0.2; self.factors.append(
                f"re-appearance implies {self.reappearance_speed_kn:.0f} kn")
        return min(s, 1.0)

Accumulating the reasons alongside the score is the part that makes the output usable. An analyst who receives “0.7” investigates from scratch; one who receives “0.7 — 34 neighbours reported throughout, gap began 40 nm inside reception, 3.2 h” can dismiss or escalate in seconds, which is the difference between a queue that gets worked and one that does not.

3. Re-appearance geometry as corroboration 

from pyproj import Geod

GEOD = Geod(ellps="WGS84")


def reappearance_speed_kn(last_fix, first_fix) -> float:
    """Straight-line speed required to get from the last fix to the first one after."""
    _, _, dist_m = GEOD.inv(last_fix.lon, last_fix.lat, first_fix.lon, first_fix.lat)
    dt_s = (first_fix.timestamp - last_fix.timestamp).total_seconds()
    if dt_s <= 0:
        raise ValueError("re-appearance is not after the disappearance")
    return (dist_m / dt_s) * 1.94384

Note what this can and cannot show. A plausible implied speed proves nothing — a vessel that went dark and sailed normally produces exactly that. An implausible one is strong evidence, because it means the vessel did not take the direct route, which is consistent with a deliberate detour. The asymmetry is why this signal contributes only when it fires.

4. Assemble the assessment 

def assess_gap(feed, gap) -> GapAssessment:
    """Collect every available piece of evidence about one gap."""
    last, first = gap.last_fix, gap.first_fix_after
    n_vessels, n_msgs = neighbours_reporting(feed, last.geometry, last.timestamp,
                                             first.timestamp)
    inside = coverage_fraction_at(last.geometry, last.timestamp) > 0.8
    speed = reappearance_speed_kn(last, first)
    a = GapAssessment(
        mmsi=int(gap.mmsi), gap_s=float(gap.gap_s), neighbours=n_vessels,
        started_inside_coverage=bool(inside), reappearance_speed_kn=float(speed),
    )
    logger.info("gap mmsi=%d %.1f h · neighbours=%d · inside=%s · reappear=%.0f kn · score=%.2f",
                a.mmsi, a.gap_s / 3600, a.neighbours, a.started_inside_coverage,
                a.reappearance_speed_kn, a.score)
    return a

Every field on the assessment is a number an analyst can check independently, which is the property that makes the score reviewable rather than oracular. When a reviewer disagrees with a verdict, the disagreement is about one of five stated facts rather than about the model.

Same Duration, Same Waters, Opposite Verdicts Two plan views of the same coastal area with a receiver at the shore and a dashed arc marking the reception horizon. In the first, a vessel's last fix sits on the arc and its nearby neighbours also stop reporting around the same time and place; the assessment records two neighbours reporting, the gap starting at the horizon, and a reachable re-appearance, giving a low score. In the second, the vessel's last fix is deep inside the arc, thirty-four other vessels continue reporting throughout the same three hours, and the re-appearance implies twenty-nine knots; the assessment gives a high score. A caption states that the gap durations are identical and that a duration-only detector cannot separate them. gap A — 3.1 h, score 0.10 reception horizon receiver last fix, on the arc 2 neighbours · began at the horizon · reachable gap B — 3.1 h, score 0.90 last fix, deep inside 34 neighbours · began inside · re-appearance 29 kn Identical durations. A detector that thresholds on duration alone cannot separate them, and most do.

Verification and acceptance test 

import pytest


def test_a_coverage_gap_scores_low(feed, coverage_gap_case):
    a = assess_gap(feed, coverage_gap_case)
    assert a.score < 0.3, f"coverage gap scored {a.score:.2f}: {a.factors}"


def test_a_dark_gap_scores_high(feed, known_dark_case):
    a = assess_gap(feed, known_dark_case)
    assert a.score >= 0.6, f"known dark event scored {a.score:.2f}: {a.factors}"


def test_every_alert_carries_its_reasons(feed, gaps):
    for g in gaps:
        a = assess_gap(feed, g)
        if a.score >= 0.5:
            assert a.factors, "an alert was raised with no stated reason"

The third test is a policy assertion rather than a correctness one, and it is worth keeping. A scoring function that can emit an alert with an empty reason list will eventually do so, and an unexplained alert is one an analyst learns to skip.

A Satellite Feed's Normal Is a Terrestrial Feed's Alarm Two overlaid histograms of AIS gap duration in hours. The terrestrial feed's distribution falls away steeply, with almost all gaps under twenty minutes and a thin tail beyond an hour that contains the events worth reviewing. The satellite feed's distribution has a broad mode between two and five hours, produced by the constellation's revisit interval, so the range that is suspicious on the terrestrial feed is entirely routine on the satellite one. A vertical marker at a three-hour threshold, tuned on the terrestrial feed, is shown falling in the middle of the satellite feed's main mode, where it would flag most of the fleet. gap duration (hours) gaps terrestrial satellite — the revisit interval a 3 h threshold, tuned terrestrially — here it flags most of the fleet 01.534.56 Subtract the revisit period before scoring, or run the neighbour test alone — it is the one signal that survives.

Edge cases and gotchas

  • Satellite feeds have structural gaps. A polar-orbiting constellation revisits mid-latitudes every few hours, so every vessel outside terrestrial range has regular gaps that are pure scheduling. Where the feed is satellite-only, the neighbour test is the only one that still works, and the revisit period should be subtracted from every gap before scoring.
  • A vessel that changes MMSI mid-voyage looks like two vessels with a gap each. The identity model in the section overview matters here: a gap detector keyed on raw MMSI reports a disappearance and an appearance rather than one continuous track with a re-identification.
  • Do not treat a high score as a finding. These signals distinguish likely deliberate from likely infrastructural. They say nothing about why, and a dark period has many lawful explanations — equipment failure, a security transit, a documented maintenance window. The output is a queue for review, not a conclusion.

Up: Anomaly Detection in AIS Trajectories