Vision-Based Navigation

Optical Flow Estimation

The Dynamics of Pixel Motion Analysis

When an unmanned aircraft enters a GNSS-denied zone, visual sensors offer a completely passive, high-bandwidth method to track movement. Unlike active radar or laser systems that emit radio frequencies, a camera gathers rich environmental data without releasing electromagnetic energy, allowing the aircraft to maintain total radio silence. The most fundamental application of camera-based movement tracking is optical flow estimation.

Optical flow is defined as the pattern of apparent motion of objects, surfaces, and edges in a visual scene, caused by the relative motion between the camera lens and the surrounding environment. For a downward-looking camera mounted on a UAS, optical flow acts as a high-frequency visual speedometer, tracking the displacement of ground features from one image frame to the next.

To process this motion, the system relies on the brightness constancy assumption. This principle states that the visual intensity or brightness of a specific pixel point in a scene remains constant between two closely spaced, consecutive image frames, even if its position shifts across the image sensor. As the aircraft flies, the ground features beneath it shift their pixel locations on the camera's complementary metal-oxide-semiconductor (CMOS) sensor.

By calculating spatial gradients (how brightness changes between neighboring pixels in a single image) and temporal gradients (how brightness changes at a single pixel location between consecutive frames), the computer determines the raw pixel velocity vectors across the image plane.

Resolving Scale and Height Ambiguity

While optical flow is highly effective for local speed tracking, it possesses a fundamental physical limitation: it cannot measure absolute physical speed (such as meters per second) on its own. A camera is an angular sensor that measures the rate at which pixels move across a two-dimensional plane, expressed as a ratio of ground speed to altitude.

An aircraft flying very fast at a high altitude can produce the exact same pixel displacement rate as an aircraft flying very slowly just a few meters above the ground.

To break this scale ambiguity and resolve the true ground velocity vector, the APNT system must integrate data from an independent vertical reference, such as a laser rangefinder or barometric altimeter. By dividing the isolated translational optical flow by the true height above ground level, the system scales the raw pixel movements, transforming abstract velocity-to-height ratios into precise physical velocity measurements.

Visual SLAM Mapping

Simultaneous Tracking and Map Construction

While optical flow is restricted to tracking relative speeds, Visual Simultaneous Localization and Mapping (SLAM) is a much more advanced approach. Visual SLAM allows an aircraft to explore completely unknown, unmapped environments by building a three-dimensional map of the surroundings on the fly, while simultaneously tracking its own position within that newly created map.

A robust Visual SLAM pipeline operates using a split architecture divided into a high-rate Front-End and a low-rate Back-End:

• The Tracking Front-End: This module operates in real time, processing incoming image frames at high speeds. It extracts distinct visual landmarks (such as high-contrast corners, rocks, or building edges) and tracks their movement across consecutive images. It provides a continuous, short-term estimate of the aircraft's local trajectory, acting much like a visual odometry engine.
• The Optimization Back-End: While the Front-End handles local tracking, the Back-End works to maintain the structural integrity of the generated map. It uses an optimization process known as Bundle Adjustment. This algorithm refines both the calculated three-dimensional positions of the environmental landmarks and the historical positions of the aircraft, minimizing the collective geometric alignment errors across a sequence of recorded images.

Loop Closure and Drift Cancellation

The most critical feature of Visual SLAM for long-range navigation is Loop Closure Detection. As the aircraft flies, the Back-End converts incoming images into compact visual fingerprints. It continuously compares these fingerprints against a historical database of places it has already visited during the current mission.

If the aircraft loops back and flies over a previously mapped location, the system recognizes the match. By calculating the exact geometric alignment between its current position and its historical record, the computer executes a pose graph optimization. This spreads the accumulated tracking error evenly across the entire historical flight path, instantly resetting the navigation drift to zero and anchoring the local map.

Terrain Relative Navigation

Profile Matching Against Point Clouds

For missions traversing long distances, the aircraft can achieve absolute, drift-free localization by utilizing Terrain Relative Navigation (TRN). This technique matches active physical measurements of the ground beneath the aircraft against a pre-loaded three-dimensional point cloud or elevation map stored in the system's memory before takeoff.

As the aircraft flies, it uses active distance sensors, such as a downward-pointing laser scanner (Lidar) or a radar altimeter, to continuously measure the exact profile of the ground below. By subtracting this ground distance from the aircraft's estimated altitude above mean sea level (provided by high-accuracy barometric and inertial state estimators), the flight computer calculates a real-time, three-dimensional elevation profile of the terrain. This live elevation profile is passed to correlation algorithms that compare the measured contour segment against the pre-loaded onboard terrain database. The system slides the live profile across the database search space. When a unique geometric correlation peak is identified, the system resolves the aircraft's absolute, drift-free horizontal coordinates. This update is fed directly into the primary navigation filter to bound the inertial drift, allowing the aircraft to pinpoint its position without any external radio transmissions or satellite signals.

Visual Map Matching

Global Alignment via Satellite Imagery

While Terrain Relative Navigation relies on active distance profiling, Visual Map Matching provides a completely passive method for absolute global localization. This technique works by comparing live imagery from an onboard downward-looking camera with ortho-rectified satellite / drone maps stored in the APNT system's database.

The visual map matching pipeline operates through three key stages:

First, the live camera image is passed through a perspective correction process. Using the aircraft's current estimated roll, pitch, and altitude, the software removes perspective distortion, transforming an angled view into a flat, top-down view that matches the exact scale and orientation of the satellite map database.

Second, the system uses invariant feature descriptors to isolate distinct, permanent visual landmarks in both the live frame and the database search window. Rather than matching simple pixel colors, which change with shadows and weather, the system identifies structural layouts such as road intersections, riverbanks, and runway boundaries.

Third, once identical landmarks are matched, the computer calculates a transformation matrix. Because every pixel in the pre-loaded satellite map is tied to an absolute latitude and longitude, solving this matrix transformation translates the center point of the live camera feed into an absolute global coordinate lock. This provides the system with an un-jammable horizontal position update, ensuring the aircraft can navigate across continents with absolute precision.