Unreal Engine 4.20 officially launched today with heaps of updates. In the VR realm, the new version brings native mixed reality video compositing for VR applications, and an update to a much newer Oculus SDK. On the AR front, UE 4.20 adds supports for Apple’s ARKit 2.0 and Google’s ARCore 1.2, along with Magic Leap support.

Mixed Reality Compositing

Mixed reality video compositing for VR allows applications to pull in video from the real world and mix it with the virtual world, which can lead to a really cool footage, like the work done for the Fantastic Contraption (2016) trailer; it’s also the preferred way for streamers to show themselves in VR.

Up to this point, mixed reality video compositing has generally required third-party programs and game implementations. Unreal Engine 4.20 brings this sort of functionality directly into the engine, simplifying the work required for developers to make their game support such features.

The feature supports video input, calibration, and in-game compositing (including occlusion), though it doesn’t directly handle capture, so a third party tool will still be needed for capture or streaming. The system also works with moving cameras, as long as they are tracked by a Vive Tracker or similar device. Documentation for the feature can be found here.

Updated Oculus SDK

For developers using Unreal Engine to build titles for the Rift, UE 4.20 moves to the newer Oculus Runtime 1.25, meaning that developers will be able to take advantage of capabilities and improvements that have come since the prior version (1.17 as of UE 4.19).

SEE ALSO
Sharp & Japan's Largest Telecom Unveil Lightweight AR Glasses 'MiRZA'

Updated AR SDKs & Magic Leap

As with the Oculus Runtime, Apple and Google have since updated their ARKit and ARCore SDK—the foundation for building AR applications on those platforms—and the newest versions are now available with UE 4.20:

ARKit 2.0

We’ve added support for Apple’s ARKit 2.0, which includes better tracking quality, support for vertical plane detection, face tracking, 2D image detection, 3D object detection, persistent AR experiences and shared AR experiences. Support for these new features enables you to place AR objects on more surfaces, track the position and orientation of a face, recognize and bring 2D images to life, detect 3D objects, and facilitate new types of collaborative AR experiences.

ARCore 1.2

We’ve added support for Google’s ARCore 1.2, which includes support for vertical plane detection, Augmented Images, and Cloud Anchors. Support for these new features enables you to place AR objects on more surfaces, recognize and bring images to life, and facilitate new types of collaborative AR experiences.

A special version of UE 4.19 with development support for Magic Leap was released earlier this year. Now UE 4.20 is the first “fully supported” release of the engine to support the Magic Leak SDK (Lumin 0.12.0), meaning developers can target the platform with a production-ready version of the engine.

Newsletter graphic

This article may contain affiliate links. If you click an affiliate link and buy a product we may receive a small commission which helps support the publication. More information.


Ben Lang
Ben is the world's most senior professional analyst solely dedicated to the XR industry, having founded Road to VR in 2011—a year before the Oculus Kickstarter sparked a resurgence that led to the modern XR landscape. He has authored more than 3,000 articles chronicling the evolution of the XR industry over more than a decade. With that unique perspective, Ben has been consistently recognized as one of the most influential voices in XR, giving keynotes and joining panel and podcast discussions at key industry events. He is a self-described "journalist and analyst, not evangelist."
  • Firestorm185

    and they demo it with Robo Recall, lol. Great, now we can have mixed reality video in a no-longer-updated game. xD

  • JJ

    Unreal Engine has a LOT of problems for VR Development that they should address before adding these additional features. Theyre really just ignoring critical problems people are having to offer new features and such and its really sad.

    • Alex

      Out of interest, could you please clarify what the problems for VR development that they should address first?

      • JJ

        This is a section from one of the current top Bug threads for Unreal Engine vr on their forums that has been confirmed true. Theses issues are causing multiple game studios problems and delaying releases. The 4.19 issue was found to be when the headset goes to sleep the audio feed bugs out and kills the current program. So yet again the only solution is wait until 4.20… or 4.21 but odds are they will also have their own errors like all the previous editions.

        I have also been in contact with the developers of Farpoint because that was made un unreal and they even confirmed these issues and also confirmed that you are not going to get above 90 fps for vr in unreal engine unless you have a very advanced technical art team, and even then its a stretch. When I talk to people about these issues they brush me off, but thats wrong of them because many people in the industry have confirmed these problems exist.

        “I’ve spent years of time developing on vive and now WMR in Unreal and Unity, and Unreal is not fit for WMR at this time and here is why.

        In 4.17 a lot of people are facing an issue where HBZ setup MIP or HBZ MIPMIP are causing lag spiking from 5-50+ ms without a reason why and disabling the ZBuffer does not solve the issue. Some people tracked the issue and it ended with UE Devs stating the issue will be fixed in 4.18 deeming 4.17 useless for VR devs because that lag spike usually will make you miss your 11ms/90FPS mark that is essential to VR Dev. 5-50ms lag spike that cannot be fixed, deems this build unusable for vr devs because you cannot launch a game that lags like that.

        In 4.18 the previous problem is fixed BUT now there is a new problem where the screen is all warped and distorted right out of the box. This is reproducible simply by opening the VR template, touching nothing, and running it. There are no solutions online for this issue, and barely any talk, but anyone using WMR has found this to be the issue with 4.18xx builds.

        So on to 4.19! Nope, Even in the VR template after about 20-30 seconds the game window freezes and you have to terminate UE to get it back. Since 4.19 is beta and just came out there are no discussion topics on this issue yet.

        We here at the office had plans on using UE4 compared to Unity for a few future projects, but seeing as how there are no legit builds that actually work for VR, we will certainly NOT use Unreal engine anymore.’

        • JJ

          So its kind of just this continuing issue of not patching current version, fixing the issue for the next one, but then bringing on more bugs. This is a big problem unless you know how to change unreal engine code, which thank god I can do. Its pretty funny seeing the unreal engine demos that are extremely cropped on the sides knowing that Its possible to rebuild the unreal engine in way that fixes that

          • Andrew Jakobs

            So you have solutions yourself? have you proposed those solutions to the UE-devteam?

          • JJ

            Maybe if UE puts me on payroll, otherwise i’m not going to do their work for them. Other devs can hit me up and its up to my discretion whether or not im going to help. I like to help :) but i also don’t like to give things i’ve spent a lot of work on away for free at this moment.

        • Andrew Jakobs

          And yet also a lot of other studio’s don’t seem to have these problems. So it might be some sort of a configuration/driver problem. (I’m not saying there aren’t any problems).
          But I agree, if there are really bad problems, they should be fixed first.

          • jj

            The other studios have been sticking with previous 4.15 and under builds plus most of them have a large enough and skilled dev team that can rewrite the unreal engine. Most people arent capable of that.

          • Andrew Jakobs

            Even as a skilled team I wouldn’t go start rewriting the unreal engine, bugfixing OK, but rewriting, not really (also isn’t that another license where you have full access to the code).
            But I really don’t understand why they wouldn’t fix those bugs you’re talking about, especially if it’s blocking usage, they would have the same problems themselves with developing and testing the engine.

    • Juan Ritz

      For over a year the number one item on my wish list has been native mixed reality compositing. I personally haven’t encountered any of these critical problems that you mention, but I certainly don’t doubt that there are some. I do however doubt that they’re entirely ignoring any critical problems. It’s obviously a case of resource allocations. They can’t please everyone, and the people that I work with are very happy that they’ve finally tackled MR compositing.

  • Trenix

    VR is pushing forward pretty fast. Going to keep my Samsung Odyssey for a month and see what happens. If a new device gets announces soon, I’ll probably return it ASAP.

    • Andrew Jakobs

      that’s lame… I can understand buying a headset and sell it later, but I think it’s pretty lame to buy a headset use it and then just return it (if it isn’t broken). (OR I’m misunderstanding what you meant).

      • Trenix

        If I get a crappy product, yes I will return it. I’m not spending hundreds on something I’ll never use again and isnt worth the price.

        • JJ

          How is it a crappy product? wouldn’t you know before the month is done…. seams like your just trying to get things for free. Instead do you research maybe test one out at a store or friends before you make that relatively small financial decision and regret it.
          For others like me I enjoy the headset and all the others I own, and if i dont then its my fault for not doing my research before spending my money

          • Trenix

            There is no testing for the Samsung Odyssey anywhere. I checked and read through many reviews. I came in expecting the Samsung Odyssey to be the best headset, far from the truth, reviewers are liars. Also I made the determination that it was a piece of crap in a few hours, if not less. The head strap design is nothing less retarded at best. It’s insanely uncomfortable, can easily fall off, and there is nothing that holds the front of the headset up or down so the image quality is always off.

            I tested it, I pressed the headset closer to my face and adjusted up and down until I got a clear image and tried to use what this headset had available to hold this position, but it simply couldn’t. Even the nose flap is unbearable. It was annoying to wear just to get a game to run and when I did, it wasn’t even supported by my controllers. So VR steam supported headset? Mostly not. This is a bad product.

            All this made me do is consider making my own VR youtube channel which heavily criticizes VR headsets so people can make their own determination. There is nothing perfect, but most reviews make it seem like it’s so. Samsung Odyssey is worse than the Vive and Oculus by FAR and notice, I mentioned nothing about tracking, just VR headset basics. The Samsung Odyssey was the first headset that I packed up ASAP to get my money back.

          • JJ

            To review the headsets you’d have to be able to afford all these headsets and by the way you talk about returning products so fast just tells us you cannot afford to be a vr reviewer. As someone who not only owns every headset but has multiple of each at work, I can say you are extremely over exaggerating the downsides of the odyssey. Out of all of the headsets, nobody out of the dozens at the office make any of the complaints you have made, which leads me to believe you are just looking for excuses to complain.

            Nobody has an issue with the padded headband or any issues with it pressing against your face. Again i feel like you’re just really inexperienced with vr because were past the phase of mystery with these headsets. You’re capable of knowing what to expect when buying an odyssey because there are so many reviews out now. Any issues you have are either already documented or only specific to you and the way you’re using it.

            Headsets also serve better purposes for different things and have different benefits. Right now the Samsung is a top pick for businesses due it how easy it is to set up and use, leading to a lot more sales for trade shows and demos.

          • Trenix

            For some people, it takes years to realize something. For others, it can take a few hours. I’m frankly, not an idiot. You remind me of those people who reply on negative steam game reviews and whine that someone only played a game for 2 hours and they won’t know what they’re talking about cause they didn’t play enough. Then when someone plays for 100 hours, you’d say they must of enjoyed the game if they played that long. You can’t win with people like you, you’re just mindless fans and I can careless of appealing to you whatsoever.

            Sorry, but the Samsung Odyssey is trash. I clearly explained why and everything I say is valid. I can careless what most people think or say, really. I’m also not going to painfully deal with something that simply works terribly. A video of a review would be far better, but I’m not a real reviewer, just a average customer that is waiting for the next generation of VR instead of experimental headsets.

        • Andrew Jakobs

          Yes, but by now you already know if you like it or not, and there aren’t many stores where you can just return a product after a month just because you didn’t like it, mostly that’s only 1 or 2 weeks (or our country if order through the internet, you have 14 days, but you have to return it within those 14 days).

          • Trenix

            Usually you have a month for most stores around here in America. I ended up returning my Odyssey the same day I got it by the way. It’s a piece of crap.

          • Andrew Jakobs

            So you bought it yesterday?

          • Trenix

            I bought it 3 days ago, it arrive yesterday and that’s the same day I returned it.

          • Jerald Doerr

            “For some people, it takes years to realize something. For others, it can take a few hours. I’m frankly, not an idiot.”

            So your saying you knew it was a piece of shit in the 1st hours of use but your going to keep it for a month? Wow your right! Your not an idiot! Your a cheap ass idiot and I hope it all backfires on you and you end up paying for your rental.

          • Trenix

            Uhh… what? I was going to use it for a month to see if it’s worth keeping and if any new product would be released during the time. I never said after exactly a month, I’d return it. Then when I got it, within about an hour, I realized it was a piece of crap and then I returned it. Are you following me or are your panties up in a bunch still?

          • Jerald Doerr

            Ok I’ll take it all back I never saw your last post for some reason and from reading the your previous post I gathered you where saying you plans on keeping it for almost a month before returning it.

            So I apologize..

            “I bought it 3 days ago, it arrive yesterday and that’s the same day I returned it.”

          • Trenix

            No I didn’t edit anything, it’s all good.

  • MasterElwood

    Magic LeaK!

    Hahaha.

  • guillaume brincin
  • oompah

    I hope unreal engine supports ray tracing.

  • Les Grzyl

    Anyone managed to get to make a mixed reality movie in Unreal Engine, either in engine or in packed game? For me after completing all steps and exporting .sav file to game config folder as requested game monitor view becomes unworkable , and in vr game gets extremely low framerate( not the case without mixed reality plugin)