Virtual production technically covers a number of things, but what people normally mean by it is shooting on an LED volume. This is a stage where the walls are giant LED screens displaying real-time backgrounds for photographing the talent in front of. The background may be a simple 2D plate shot from a moving vehicle, for a scene inside a car, or a more elaborate set of plates shot with a 360° rig.
The most advanced set-ups do not use filmed backgrounds at all, but instead use 3D virtual environments rendered in real time by a gaming engine like Unreal. A motion-tracking system monitors the position of the camera within the volume and ensures that the proper perspective and parallax is displayed on the screens. Furthermore, the screens are bright enough that they provide most or all of the illumination needed on the talent in a very realistic way.
I have never done any virtual production myself, but earlier this year I was fortunate enough to interview some DPs who have, for a British Cinematographer article. Here are some tips about VP shooting which I learnt from these pioneers.
1. Shoot large format
To prevent a moiré effect from the LED pixels, the screens need to be out of focus. Choosing an LF camera, with their shallower depth of field, makes this easier to accomplish. The Alexa Mini LF seems to be a popular choice, but the Sony Venice evidently works well too.
2. Keep your distance
To maintain the illusion, neither the talent nor the camera should get too close to the screens. A rule of thumb is that the minimum distance in metres should be no less than the pixel pitch of the screens. (The pixel pitch is the distance in millimetres between the centre of one pixel and the centre of the next.) So for a screen of 2.3mm pixel pitch, keep everything at least 2.3m away.
3. Tie it all together
Several DPs have found that the real foreground and the virtual background fit together more seamlessly if haze or a diffusion filter are used. This makes sense because both soften the image, blending light from nearby elements of the frame together. Other in-camera effects like rain (if the screens are rated weatherproof) and lens flares would also help.
4. Surround yourself
The most convincing LED volumes have screens surrounding the talent, perhaps 270° worth, and an overhead screen as well. Although typically only one of these screens will be of a high enough resolution to shoot towards, the others are important because they shed interactive light on the talent, making them really seem like they’re in the correct environment.
5. Match the lighting
If you need to supplement the light, use a colour meter to measure the ambience coming from the screens, then dial that temperature into an LED fixture. If you don’t have a colour meter you should conduct tests beforehand, as what matches to the eye may not necessarily match on camera.
6. Avoid fast camera moves
It takes a huge amount of processing power to render a virtual background in real time, so there will always be a lag. The Mandalorian works around this by shooting in a very classical style (which fits the Star Wars universe perfectly), with dolly moves and jibs rather than a lot of handheld shots. The faster the camera moves, the more the delay in the background will be noticeable. For the same reason, high frame rates are not recommended, but as processing power increases, these restrictions will undoubtedly fall away.
This is the second part of my report from my time at Camerimage, the Polish film festival focused on cinematography. Read part one here.
Panavision: The BEauty of 8K Large Format
It was a chilly but bright morning as I strolled into Bydgoszcz and made straight for the MCK Orzeł, where I planned to spend most of the day. With only a few minutes to go until the scheduled start time, the queue for Panavision’s large format seminar had spilt out onto the street. The tickets ran out before I reached the desk, but there was a live video feed in the cinema’s bar. In many ways this was better than going into the auditorium – sitting in a comfy chair with the bar close at hand, and a table to make notes on.
My article for Red Shark News about the future of large format cinematography has been surprisingly popular, and it contains plenty of detail about this Panavision seminar, so I won’t repeated myself here. I will say that it converted me from a high resolution sceptic to a believer, because the speakers demonstrated that footage acquired in high rez – even when downscaled – retain much of its smoothness, high bit depth and dynamic range. “More resolution evokes the imagination of the brain,” was how colourist Ian Vertovec summed it up.
At the end of the session, Red Shark’s David Shapton and Matt Gregory emerged from the auditorium and joined me for lunch at the bar. We had all found the seminar very interesting, and Matt was quick to get us all tickets to the Panasonic 4K seminar coming up later in the day. The pair then went off to other things, while I headed to the kiosk to get a ticket for the imminent John Toll seminar. But of course I’d left it too late, they were all gone, and so I returned to my comfy chair in the bar to watch via video feed again.
Panavision workshops: A conversation with John Toll, ASC
John Toll, ASC was the recipient of this year’s Lifetime Achievement Award at Camerimage, and this seminar was an epic journey through this career. He explained how he learnt his craft as a camera operator for the late great Conrad Hall, ASC and Jordan Cronenweth, ASC.
The talk then focused on some of Toll’s biggest movies, beginning with the period drama Legends of the Fall starring Brad Pitt and Anthony Hopkins. The movie was largely daylight exterior (something that was to become a theme across Toll’s work) so the cinematographer insisted on twelve weeks of prep, the same as the production designer. This allowed him to be part of selecting locations and choosing orientations for the buildings to get the optimal sun path. Toll said he was lucky that the 1st AD was willing to be flexible with the schedule, observing the mood of the weather each day and shooting scenes that matched that mood.
Gaffer Jim Plannette joined Toll on the stage to discuss the huge night exterior battle sequence. This employed three Musco lights (a Musco being fifteen 6K pars on a 100ft boom arm) which three-quarter-backlit every angle. To get crisp, grain-free blacks, Toll overexposed and printed down.
Braveheart was covered next, with 1st AC Graham Hall joining the panel. Hall had a difficult time with the film’s battle scenes, featuring as they did so much movement, improv and slo-mo. Toll revealed how the immersive style of the action was based on a sixties TV documentary about Culloden that coincidentally both he and director Mel Gibson had seen. A lot of colour timing was required to give consistency to the battles, which were shot over weeks, at all times of day.
The discussion then turned to Terrence Malick’s The Thin Red Line. The Australian locations featured very difficult, uneven terrain, so Toll used an Akela Crane. The crane’s arm was so long (75ft) that the arc of its movement couldn’t be detected on camera. Its use had to be carefully planned though, because each time it needed to be moved it had to be disassembled and reassembled on a special platform. 80% of the filmwas actually handheld, and Toll operated himself so as to respond to the spontaneity and improv which Malick encourage from his cast.
Toll told a funny story about a shot of shadows moving across long grass which was praised by critics. It was inserted to cover a continuity error, the build-up to the battle having been shot in heavy cloud, while the battle itself was shot in full sun. A happy accident!
Time was running short, so the moderator powered through Almost Famous (where parallels were drawn between the explosive battle scenes of Toll’s earlier movies and the crowd scenes at the rock concerts, punctuated by flashing cameras), Vanilla Sky and Billy Lynn’s Long Halftime Walk. This latter film, directed by Ang Lee, is notable for its acquisition in 120fps 4K stereo. You can read more about it on the British Cinematographer website.
Panasonic Seminar: Varicam Experience
I remained at the MCK and, meeting up with Dave and Matt again, finally got into the auditorium, for the Panasonic seminar. Moderated by British Cinematographer‘s Ronnie Prince, the panel brought together a trio of DPs who shoot Netflix shows on the Varicam: Bobby Shore, CSC (Anne with an E), Pepe Avila del Pino (Ozark) and Patrick Alexander Stewart (Arrested Development).
Both Shore and del Pino admitted that they were most comfortable with the Arri Alexa, but due to Netflix’s strict rules on 4K acquisition (the Alexa tops out at 3.2K) they had to find another camera. They plumped for the Panasonic Varicam, a 4K camera best known for having dual native ISOs: 800, in common with the Alexa, Red and many others, and 5000, two and two-thirds stops faster. Being native, both ISOs have the same dynamic range, the same log curve and in theory are equally clean.
Panasonic, I think, were keen to push ISO 5000 in this seminar, but unfortunately Shore and del Pino shot almost exclusively at 800. Stewart did shoot Arrested Development at 5000, interiors and night scenes at least, but down-rated it to 2500. Otherwise, he said, the sets would have been too dark for the actors to feel like they were in a believable daytime environment. I think that’s a fascinating, unexpected side effect of low light sensitivity! Stewart lit the stages with large Quasar softboxes and paired the Varicam with Fujinon zooms and occasional Xeen primes.
Del Pino chose Zeiss Super Speeds for night scenes and Hawk V anamorphics for day scenes on Ozark. He liked the claustrophobia of cropping the anamorphic images to 2:1, the show’s delivery ratio, while the Super Speeds produced a creaminess he found appealing. He also mentioned that the Varicam’s sensor handled greens very well, which was important for him, given how much of Ozark takes places in woods.
Anne with an E was lensed on vintage Panavision Standard and Super Speeds – “the oldest, craziest lenses we could find,” says Bobby Shore. He liked their low resolving power, their weird flares and how they fell apart when wide open. Part of the show’s signature look are ECUs of the freckled title character, which were captured on a 27mm Primo for a more detailed, tactile image.
Inspired by the work of Robbie Ryan, BSC (Wuthering Heights), Shore kept the lighting naturalistic, mixing tungsten and HMI sources on a set that was treated like a location. He also shot through a Panaflasher, a special lens filter with built-in lighting which reduces contrast and adds a colour tint of your choice, but this effect was dialled out in post. Indeed, Shore raised the issue of DPs’ images being altered after the fact, an issue I explore fully in another Red Shark article.
By the way, I’ve been watching Anne with an E since the festival and I can thoroughly recommend it.
When the seminar was over, I went for dinner with Dave, Matt and Chris Bouchard, at a very nice (but once again cheap) Asian fusion restaurant. Then we met up for drinks with some of the reps from Red, and I got talking to a DP who had shot squid from a submarine for an episode of Blue Planet. After another drink or two with Chris at the Cheat bar, I called it a night.
Tune in next week for the final part of my Camerimage blog.
This week I attended Camerimage for the first time. Centred around the Opera Nova theatre beside the river Brda in Bydgoszcz, Poland, Camerimage is an international film festival celebrating the art of cinematography. It’s a bit like Cannes for DPs, but colder. This is the first part of my account of my three days at the annual hub of motion picture imaging.
The Ryanair flight was dirt cheap but trouble free, and at 9:50am I found myself on the tarmac of Bydgoszcz airport. There I met David Shapton and Matt Gregory, founders of Red Shark News, for the first time. I’ve been contributing articles to Red Shark for a few months so it was nice to finally meet these gentlemen in person.
A taxi (also dirt cheap) dropped me at the Opera Nova – only about three miles from the airport – where I picked up my pass and goodie bag. Bizarrely, said goodies included an Ikea catalogue. How did they know that us DPs love flat-pack furniture so much?
Canon Workshop: Stephen Goldblatt
From the Opera Nova I hurried to a college across the river, where the sports hall formed the venue for a Canon workshop run by Stephen Goldblatt, ASC, BSC, the man behind the lens for the likes of Lethal Weapon 2 and Batman Forever. The blurb for this workshop described Goldblatt as “a master of low light shooting”, and it was certainly pitch black when I walked in a few minutes late, and gingerly picked my way around to the far side of the hall to find a seat.
On a purpose-built bedroom set, Goldblatt was recreating lighting from the Robert Redford / Jane Fonda romantic drama Our Souls at Night, shot on the Sony F55 and Canon C300 Mark II. To practical lamps on either side of the bed he added egg-crated tungsten soft-boxes to beef up each one. He simulated moonlight through an imagined off-camera window by placing a lace curtain in front of a blue-gelled lamp and blowing it gently with a fan. An additional egg-crated soft-box provided a low level of blue toplight.
As he worked, Goldblatt revealed how he doesn’t miss ceulloid, loving how relatively easy it is now to light night exteriors or moving car scenes. “But just because you don’t need much light,” he cautioned, “it doesn’t mean you don’t want to control it.” Other developments coming down the pipe do not inspire him so much; he feels that high resolutions and HDR are unnecessary, pushed by marketing people rather than creatives.
He placed great emphasis on the importance of the eyes. “A common failing of newer DPs is that they worry more about the set than the eyes,” he said, before explaining how he will often walk beside the handheld camera with a torch, providing eye-light. He also stressed the importance of eye-lines. Although in any one shot it’s not that important how wide or tight the eye-line is, or how high or low, across the two hours of a feature film the decisions have a cumulative effect.
Goldblatt no longer uses a light meter. “Trust your eye, develop your eye,” he advised, adding that you must have a strong voice to remain in control of the images through postproduction.
After grabbing lunch, I returned to the Opera Nova to browse the exhibition hall. This closely resembled a mini BSC Expo or Media Production Show, with all the major camera and lens manufacturers displaying their wares, along with several lighting companies. I had a play with some of the cameras, including the actual Alexa 65 used on Rogue One.
Then I met up with Chris Bouchard, one of The Little Mermaid‘s two directors, who had arrived in Poland the previous day. We sauntered over to another venue, the MCK Orzeł, an independent cinema with a nice, chilled, film-buff-friendly atmosphere. The auditorium itself was packed though as we settled in for a seminar on “The Future of Digital Formats”.
Red Seminar: The Future of Digital Formats
Promoting Red’s Monstro sensor, the session was mostly about the benefits of shooting in high resolutions, and giving yourself the maximum flexibility in post. You can read my thoughts on both of those topics in upcoming Red Shark articles.
One of the speakers, Christopher Probst, ASC (DP of Mindhunter and technical editor of American Cinematographer magazine) made some interesting points about ISO. “Traditionally, low ISOs were used for bright scenes like day exteriors, and high ISOs were used for darker scenes like night exteriors,” he explained. “That was based on reducing the grain, getting the cleanest possible image on film.” He advised the opposite for digital capture. “Use a low ISO for nights to get more shadow detail, and a high ISO for days to get more highlight detail [in the sky, for example].”
Another interesting nugget came from Markus Förderer, BVK. On Independence Day: Resurgence he switched between spherical, 1.3x anamorphic and 2x anamorphic lenses depending on the situation. For example, flatter lenses were better for wide shots – where anamorphics would distort straight lines – and for VFX work.
Hawk Vantage Seminar: Top cinematographers tell their Hawk stories
I ducked out of the Red session early so that I could pop back to the Opera Nova for the Hawk Vantage seminar, bumping into my Perplexed Music gaffer Sam Meyer on the way. Hawk were launching three new sets of lenses: MiniHawk (T1.7 hybrid anamorphics), Hawk Class-X (T2.2 2x anamorphics) and Hawk65 (T2.2).
The MiniHawks in particular seem very exciting. Daniel Pearl, ASC showed us some stunning frame grabs from the upcoming Dennis Quaid vehicle Motivated Seller, shot using these lenses on Alexa Mini. Whilst having key advantages of spherical lenses (speed, small size, low weight, extremely close focus) the MiniHawks have a unique and beautiful cigar-shaped bokeh.
While Pearl had used the latest Hawks, Magdalena Górka, PSC had shot with some old ones, the C series, for Brad Silberling’s drama An Ordinary Man. “I had to frame everything centrally because that’s the only place that was sharp!” she laughed. Also addressing focus fall-off, Andrzej Bartkowiak, ASC (Speed, The Devil’s Advocate) stated, “I like anamorphic because the shallow depth of field allows you to direct the viewer’s eye more.”
Stuart Dryburgh, ASC (The Secret Life of Walter Mitty, Bridget Jones’s Diary) talked about shooting 1.3x anamorphic. He has done this on three-perf 35mm (to achieve a Scope aspect ratio), on an Alexa in 16:9 mode (again for 2.39:1) and on an Alexa in 4:3 mode (to get 1.85:1). He also recommended shooting on Super-16 with 1.3x glass, citing the example of Ed Sheeran’s “Thinking Out Loud” video, which Pearl shot.
Peter Flinckenberg, FCK (Upswing, Concrete Night) noted that, with the shift to digital acquisition, the DP is no longer a magician, “but you can bring back that magic with lighting and glass that has character.”
CW Sonderoptic: Exploring Large format cinematography & Leica lenses
I took my leave, dashing back to the MCK Orzeł for another lens-themed seminar, this time by CWSonderoptic, the makers of Leica. The first half of this panel revolved around a short film shot by Darius Khondji, ASC, AFC (Seven, Delicatessen) on an Alexa 65 with the new Leica Thalias.
The second half was all about Tod Campbell, DP of Stranger Things and Mr. Robot, focusing on the latter show. The second season of Mr. Robot was shot on Leica Summicrons after Campbell found that the Cookes used on season one distorted the many straight lines which became such a key part of the show’s unique look. “I look at season two as kind of the birth of the photography for the show,” he said. With a laugh he added: “Sorry that the lighting looks like shit in season one. I was learning!” (See my spherical lens tests for my own thoughts on Cookes and Leicas.)
Campbell revealed that season three of Mr. Robot has a different look again, using much more camera movement and “twice as much atmos”. For this season he paired Canon K35 glass with an 8K camera, but due to the Canons’ low resolution he employed Leica Summiluxes for the wide shots.
He also shared some interesting information about his testing process, admitting that he doesn’t really know how other DPs test. He doesn’t use charts, he just makes it up. He always includes a candle, a practical lamp, some kind of highlight in the background, and random foreground objects (as background bokeh can differ from foreground bokeh).
Christopher Doyle Seminar
When the Leica seminar ended I went back to the Opera Nova, where Chris and I had dinner at the nice (and once again cheap – are you detecting a theme?) restaurant. Despite having got up at 4am (3am local time) I wasn’t feeling too tired, so we headed upstairs to the 10pm seminar by Christopher Doyle, HKSC (Hero, Lady in the Water). Many people were nursing beers, including Doyle himself, and the lecture theatre was dimly illuminated by mood lighting. Clearly this session was not going to be like the daytime ones.
“We’re going to fuck things up,” Doyle began, dispelling all doubts. He proceeded to talk disjointedly but entertainingly about his work on The White Girl and what I think was a separate film about a camera obscura. His oratory was liberally sprinkled with great one-liners, a few of which I reproduce here for your edification:
There are only three people in filmmaking: the actor, the audience and the cinematographer in between them.
If actors don’t feel loved, the performance will not come across on camera.
Give the idea the image it deserves.
[Vittorio] Storaro [legendary DP of Apocalypse Now amongst others] can’t tell you how to do it. You have to find it for yourself.
People in space – that’s what cinematography’s about.
The location is very important. It gives the energy, it imposes the style.
The lens doesn’t matter; it’s what it shows that’s important.
You never sleep because you care too much – that’s what filmmaking is.
Doyle also picked up on a piece of dialogue from a clip he screened: “What is it?” / “I don’t know yet.” It was a great summation of finding the essence of a shot, he said.
Having had our fill of aphorisms, Chris Bouchard and I slipped out to get a drink. The Cheat, the pop-up bar across the road, was absolutely packed, and my early morning was finally catching up with me, so I called it a night. The highstreet of Bydgoszcz was quiet and chilly as I walked briskly to my hotel, curiously located down a service road behind the city’s football stadium, reflecting on all that I had learnt that day.
Tune in next week for tales from my second day at Camerimage.