Skip to content
Darley Defense Day · Dec 1

Real-time 3D mapping

Real-time 3D mapping from live drone video.

Real-time 3D mapping means building a 3D model of an area while the drone is still flying, instead of processing its imagery after it lands. SpatialGen reconstructs 3D from live ISR video on the edge, on one NVIDIA Jetson Orin or any edge computer, and adds it to the same 3D map it builds from satellite imagery.

Last updated

Fig. 02 EdgeSee it in the film

Why real-time 3D mapping belongs at the edge

Conventional drone mapping is a batch job. The aircraft flies a pattern, lands, and its photos are processed on a workstation or uploaded to a cloud service. The 3D model arrives after the flight, and only if the imagery can get to the compute.

At the tactical edge neither is a safe assumption. Links are narrow and contested, and the decision the model informs is often made during the mission. SpatialGen runs reconstruction beside the video feed, on edge compute, so the 3D picture updates as the drone flies.

The drone is also not the starting point. SpatialGen builds the base 3D map from satellite imagery before launch, and live ISR video lands on that map automatically. A unit has a 3D map before the first aircraft is overhead, and a better one once it is.

Post-flight photogrammetry and real-time reconstruction

AttributePost-flight photogrammetryReal-time reconstruction (SpatialGen)
When the 3D model existsAfter landing and processingWhile the aircraft is flying
Where it is computedA workstation or a cloud serviceAn edge computer beside the video feed
What has to cross the data linkThe full set of photos, or the finished modelThe video already being sent, with 3D results carried inside it as metadata
Map before the first flightNoneYes, built from satellite imagery

How it works

  1. 01

    Start from the satellite map

    SpatialGen turns satellite imagery into a 3D map of the area before launch: 10 km² in 10 minutes.2

  2. 02

    Take in live video

    The processor reads frames from the live ISR video stream on the edge computer and selects the keyframes worth reconstructing.

  3. 03

    Reconstruct on the edge

    A neural reconstruction model turns those keyframes into overlapping 3D submaps on one NVIDIA Jetson Orin or any edge computer. No cloud upload.

  4. 04

    Send 3D inside the video stream

    Results travel as KLV metadata inside the MPEG-TS video stream. Viewers merge the submaps into one model, and can join late or replay a recording.

Performance

NVIDIA Jetson Orin. Or any edge computer.
1
10 km² of satellite imagery to 3D.2
10 min
  1. 2Validated with Satellogic imagery. The US Army's One World Terrain took 3 hours to generate 1 km² (Army, 2019).

Terms

KLV
Key-length-value, the metadata encoding carried alongside full-motion video from unmanned aircraft. It lets data such as sensor position travel in the same stream as the pictures.
MPEG-TS
MPEG transport stream, the container most ISR video links use to carry video and its metadata together.
Keyframe
A video frame chosen for reconstruction because it adds a new view of the scene.
Submap
A small 3D reconstruction of part of the scene. Submaps are aligned and merged into one model.

Questions

What is real-time 3D mapping?

Real-time 3D mapping is building a 3D model of an area from a sensor feed as the feed arrives, so the model is usable during the mission instead of after it. SpatialGen does this from live ISR drone video on edge compute and adds the result to a 3D map built from satellite imagery.

How is real-time 3D mapping different from photogrammetry?

Photogrammetry is usually a batch process: collect overlapping photos, then compute a model from all of them afterward. Real-time reconstruction processes video as it arrives and extends the model continuously. Photogrammetry suits survey products made after a flight; real-time reconstruction suits decisions made during one.

Do you need a drone flight before you have a 3D map?

No. SpatialGen builds the base 3D map from satellite imagery, so it exists before any aircraft launches. Live drone video is added to that map when it is available. This is the main difference from tools that build their 3D model only from drone-collected imagery.

What compute does real-time 3D mapping need?

SpatialGen generates the 3D map on the edge with minimal compute: one NVIDIA Jetson Orin or any edge computer. Haivision is a SpatialGen technology partner for ISR edge compute. Reconstruction runs where the video is, so native video frames do not have to be uploaded anywhere.

How does the 3D data get from the edge to the people who need it?

Inside the video stream. SpatialGen packages 3D results as KLV metadata in the MPEG-TS stream that already carries the ISR video, so no second data link or upload service is required. A viewer can join mid-stream, or replay a recorded stream later and rebuild the same 3D model.

Does it work in ATAK?

SpatialGen's 3D map is formatted for ATAK and iTAK, and the SpatialGen ATAK plugin plans drone routes on it. Live ISR drone video lands on the same map. Request a briefing to see the current workflow from live video to TAK.

Darley Defense Day · December 1

See it. Win it. December 1.

Live at Darley Defense Day.