Skip to Main Content
Skip Nav Destination

Lidar, camera, action!

26 July 2013

Radar's visible-light analogue finds uses in movie, TV, and video game production.

You're watching a movie. A cluster of people is leaning over a ship's rail. The camera pulls back, and a huge monster appears, nibbling the stern. With perfect scaling and perspective from every angle, the gap between the unaltered footage of the real ship and the computer-generated footage of the fictional monster is bridged, thanks to lidar.

Lidar is a technology that exploits the constant speed of light to map three-dimensional spaces. The simplest method bounces laser beams off targets and determines the reflections' time of flight directly. A more sensitive method, based on heterodyne detection, mixes the beam with a local oscillator of lower frequency and determines the time of flight via interference.

In either method, each time-of-flight from each direction yields a cloud of millions of x, y, and z coordinates, which track outlines, boundaries, and textures to millimeter accuracy. The clouds are processed into mesh surfaces or wire frames, ready for digital modeling. Lidar-derived models are used for building topographical maps, documenting crime scenes, and surveying historical buildings. They're also used for creating visual effects.

From lidar to visual effects

Ben Cole is the Vancouver-based lead software engineer for MPC. Earlier this summer, I met him to discuss the role of lidar data in visual effects.

Cole explained that modern TV features lots of 'hidden 3D,' whose visual effects cross into all genres and budgets in ways most audience members will not even notice. The quality of graphics and animation is constantly improving, making it ever-easier to perform visual cheats. But at the same time, the increasing fidelity of TV monitors has made those cheats easier to identify. Lidar visual effects are more accurate in appearance and motion, and thus prevent the disappointing film-going moment when 'almost perfect' still looks strange.

Although lidar surveys can be as varied in execution as they are in application, surveys for movies and TV are usually conducted in the same way. Multiple scans are taken from multiple locations to recreate environment, with each scanner position carefully recorded. Outdoors, GPS can be used to obtain positions. For sets built inside a studio, relative positions are determined by distributing round spheres of established dimensions and reflective properties about the set. Those 'known features' are then used to align multiple scans from different angles or positions. A fairly simple set with only four scanning positions may be surveyed in less than half an hour, although it will take significantly longer to process and merge the data into a massive, all-encompassing virtual environment.

Lidar surveys for the entertainment industry resemble those for other purposes, except that filming has a highly transient nature. What is on the set today could be gone tomorrow, so any survey data collected must be collected correctly and completely the first time, usually on short notice. To get a sense of those and other challenges, I spoke with Huseyin Caner, head of film and entertainment for Plowman Craven in the UK, and Ron Bedard, president of Industrial Pixel Visual Effects in Canada.

In the film industry, timelines are tight and decisions can change quickly, so a survey team must be ready to optimize their limited window to scan a location. The Russian Antonov An-124-100 is far too valuable an aircraft to destroy, even for a big-budget movie. To help create the climax of the 2002 James Bond movie Die Another Day, Plowman Craven scanned one of the aircraft. Working inside a museum with only a half-day to collect all the lidar data, the survey crew managed to scan only half the massive cargo plane. They ended up mirroring the data to build a complete digital model. Plowman Craven's newest piece of lidar equipment, Caner tells me, is capable of collecting 1 million points per second.

Capturing 1 million points per second yields mind-boggingly dense data sets of hundreds of gigabytes. But size isn't all that matters. Although modern equipment is no longer limited to scanning flat, matte-white surfaces, scans from wet, dark, or highly reflective surfaces are still noisy. Running noise-processing software, matching multiple scans, removing artifacts, and building surfaces are time-consuming tasks, even when performed automatically by computer. When processing is complete, the massive datasets are shrunk drastically to strike a balance between the client's needs for resolution and manageability.

In the past, diverse data formats frustrated the transfer of lidar data between software suites made by different vendors. Now, thanks to the adoption of wavefront object files (.objs) as the industry standard, files are compatible with all the software suites used in the industry throughout all the steps of the modeling process.

Lidar equipment is upgraded annually to take advantage of technological improvements. Older scanners could accurately scan only matte white surfaces. The scanners, which were extremely heavy and awkward, cost upward of $250 000. As a graduate student, I used one of the monstrous chunks of metal to scan rock faces. My colleagues and I had to lug the scanner out of the back of a truck, and manhandle it into position, hoping all the while we didn't break it. Now all sorts of surfaces may be scanned with portable equipment costing about $50 000. The smallest models can even be held in one hand, although they still require a steady tripod.

Lidar is rarely used in isolation. The technology can be paired with photography to add color, intensity, or exposure values to the x, y, and z positional information. Motion capture and reference photography also supplement lidar in creating models.

Neither Industrial Pixel nor Plowman Craven originated in the entertainment industry. Industrial Pixel split from its parent, an engineering company, when it became clear that the growing number of survey requests from Vancouver's active film industry could sustain a separate business. Bedard joined the new company, bringing with him his experience in forensic photography.

Plowman Craven remains a full-service, all-industry survey company. It expanded into the entertainment industry after a member of its marketing staff sat next to a visual effects supervisor at a social event. Caner was one of the company's first surveyors to learn how to use lidar.

Both companies prefer to hire employees with an aptitude for the work and then add on in-house training for the specific skills necessary. Where the companies differ is what skills they consider more difficult to teach after hiring.

Bedard explained to me that although people with physics or engineering backgrounds have desirable problem-solving skills and technical aptitude, he finds it more useful to hire people who have a background in film and understand the different workflows of engineering and film. Conversely, Caner prefers to hire people with a surveying background and then teach them how to survey a film set. For positions data processing, however, he prefers to hire candidates with a visual-effects background who can apply artistic judgment in creating models and texture maps.

Lidar is most commonly used for high-end productions, on sets that will be repeatedly used, or in situations where cameras can't be positioned to film what a director envisions. Bedard told me that his company usually provides lidar scanning for feature films, but also occasionally for TV if a recurring or complicated character or set makes lidar a more efficient use of a visual effects budget.

The balance between accuracy and artistry lies at the heart of visual effects. Imagine the painted backdrop from the Wizard of Oz, or the projected backgrounds common to the 1970s and 1980s. Now, try to spot the exact moment of shifting between a fully real and fully digital background in any of this summer's blockbuster movies, one of which features the sea monster that opens this article.

Whether using the accuracy of a lidar scan to exactly match scales and angles, or artistic license to play up perspective and perceptions, visual effects studios lean heavily on the laws of physics to enhance your viewing experience.

Mika McKinnon is a disaster researcher, entertainment science consultant, and irrepressible educator. She writes about disasters at GeoMika.com , and science in fiction at SpaceMika.com .

Close Modal

or Create an Account

Close Modal
Close Modal