Before we go any further: If you haven’t used it: SynthEyes is a standalone tracking and matchmoving application for camera, objects, geometry and planes – including tools for stabilisation and motion capture. It is used – among many other things – for critter insertion, fixing shaky shots, virtual sets, object removal, stereoscopic production, 360°VR, architectural previews, accident reconstruction, product placement, face and body capture, and many more – and in the 20 years it has been around, it has been one of the premiere trackers in the market – although with a strong focus on features, and not necessarily on ease of use. See the announcement and Dr. Russ Anderson in action here:
blog.borisfx.com/3d-camera-solving-syntheyes
In the current version, a few of the “highlights” are a new “Curves” colour correction tool, a PrepSet Manager, easier roto splines editing, with per-spline Invert and Ignore. As well as a camera view “RotoMask” display, “Greenscreen” Keying panel improvements, “Render Cameras” for independent preview renders and many workarounds and fixes for problems in “adjacent programs” – Syntheyes is well aware that it has to talk to any platform out there.
But enough of the preliminaries – we went to the Beach at the Amsterdam RAI and chatted to Boris Yamnitzki, CEO and Founder of BorisFX and John-Paul Smith, CEO Imagineer Systems. If you want to know what else they are doing: borisfx.com
The Boris FX Team at the Dell Booth in Amsterdam: Third from the left John-Paul Smith and on forth from the left Boris himself
DP: Syntheyes has been around for 20 years now. What can existing users expect now that it has joined Boris effects?
Boris Yamnitzki: Syntheyes is joining the Boris FX suite of products, which means that it will be marketed and sold and – what‘s more important – supported by a larger team. Up to this point it was basically a one-person effort – however brilliant and hard-working Russ (Anderson) is. Now Syntheyes is going to benefit from a larger team and wider technical support. In addition, obviously, there is a big road map for Syntheyes technology and we plan to take it further. Russ is of course staying with the product and is obviously going to lead the technology and the science of 3D camera solving, but there‘s a team of programmers who will be supporting him in that – specifically, the original Mocha Imagineer Team.
Also, we would like to invest in education and training of this product. We would like to enhance support and basically increase what I call the “product availability” – in other words, the product will be available to a wider range of users.
John-Paul Smith: I think the one important point to reiterate for the existing Syntheyes users is we‘re not planning to stop making the standalone version of Syntheyes. We‘re not planning to step away from any of the really important use cases. This involves growing the house of our industry and I know that there are particular niches where it‘s widely used and so far just our exposure to this entire community is that it is really passionate about the product, and we really want to build on that and, you know, make the most of the community engagement and take the products in the direction that benefits this community.
DP: Can we expect to see a plug-in version of Syntheyes in the future?
Boris Yamnitzki: Well, without commenting specifically on a date, yeah.
John-Paul Smith: And I just add on to that, people should see what happened to Mocha when it was acquired by Boris FX and the kind of direction we took that product – something similar will probably happen with Syntheyes.
DP: You‘ve got a pretty broad line-up now, starting with some pretty easy to use tools like Optics or Continuum to really complex applications like Silhouette or Syntheyes. How hard is it to bring these into streamlined UI/UX?
Boris Yamnitzki: Hard! But we do have a standardised development toolchain that really helps us on every platform and streamlines our software development. One example of that would be Particle Illusion – which originated (similarly) many years ago with one developer and had access to the development framework. And then it was brought on the more standard framework that our other products use. And of course, the program itself benefited greatly from that. I expect all current and future products joining the Boris FX family to benefit from that – and the developments of the platform in the future!
DP: One of the biggest trends in our industry at the moment is A.I. which is
of course always connected to the fear of A.I. replacing the artist. What is your stance on the issue? Especially looking at the tasks that artists use Mocha, Silhouette and Syntheyes for?
Boris Yamnitzki: Well, first I can mention that Syntheyes currently has an AI component in it, which was developed earlier by Russ Anderson. So, it definitely will move that forward or improve and retrain and enhance that.
In general, the company is investing heavily in A.I. development and essentially rapidly becoming an A.I. company where we spend a lot of effort in building training sets, adopting models, creating more programing models and releasing new features based on A.I. Features that were solved by traditional programming methods earlier. However, having said that, we have not found that it is a “cure all” for everything.
So there is always this final human push that is necessary for any professional pipeline and results in film or broadcast television – the last 5 percent quality that can only be done by humans. A.I can help streamline the work. It can definitely accelerate and make the projects more cost effective. And it‘s just the technology shift, or rather a technological revolution. That‘s how I would put it bluntly.
I think about it as it‘s like the internet: That was a threshold. It was a changing point for a lot of industries and a lot of markets. And so A.I. is no less and we‘re just in the beginning. I‘m not afraid of A.I., and I wouldn’t discourage anyone from being enthusiastic, but there‘ll be winners and losers, like with every technological shift, like the horse carriages gave way to automobiles. Same thing will happen here.
John-Paul Smith: We don‘t want to replace the creative process with an A.I. process. It‘s very much “assisting”, as far as we‘re concerned. And those are the areas that we‘re investing in – assistive technology. I think it‘s worth saying that our focus on the A.I. side at the moment is very much around things that are currently either awkward, time consuming or bad – meaning where the existing algorithms just don‘t give as good quality results as A.I. can. We‘re focused on making artistic tools, which help create and tell a story or, do whatever it is that they‘re trying to achieve.
DP: So we‘ve seen a lot of integration across the BorisFX line-up. Can we expect similar things for some folks or other products?
Boris Yamnitzki: Again: Yeah, without directly commenting on the roadmap. We
try to take advantage of all technologies that we have in the Boris FX Suite. Mocha made a big impact on Sapphire, likewise on Silhouette. So, you know, definitely something to look forward to in the future.
John-Paul Smith: We‘re demonstrating a kind of experimental data exchange between Mocha and Syntheyes. We showed taking Mocha mesh tracks into scenarios and then solving them in 3D. And that‘s something that we were able to put together in the space of two or three weeks of development – that‘s really promising. It illustrates the kind of thing that we would like to do in the future.
DP: Looking into the rise of smaller content creators, are there plans for simplified versions?
Boris Yamnitzki: That‘s a difficult question. On the one hand, yes, you would want everyone to be able to matchmove. The strength of Syntheyes, though, is in that specialised user interface. That‘s not to say that we won‘t be able to offer maybe a simplified version of that.
John-Paul Smith: The depth in Syntheyes is one of the reasons why it is a successful product. Obviously, the camera solving algorithm and the 2D tracking process is fantastic, but it‘s the 20 years of development of Syntheyes. I‘ve seen so many features added in response to specific customer situations. Anything we want to make sure that we don‘t dumb it down too much. But yeah, there‘s the potential to do something different – we‘re well aware that there‘s a steep learning curve and Syntheyes and we‘re absolutely interested in reducing that. And certainly, as we look at the roadmap of Syntheyes, we are definitely going to be putting effort into the user interface and making it more accessible. Whether it will ever be accessible for a casual user, I‘m not sure.
DP: Anything else our readers should be on the lookout for?
Boris Yamnitzki: By the time everyone reads this, Syntheyes will be included in the Boris FX Suite, which is a one license for all products and all programs, and that also goes for CrumplePop audio plug-ins, which was also recently acquired. So this is what I call „a larger availability for a wider availability“. Whilst people may not necessarily go out and buy a license and start learning it, Syntheyes will already be included in the “Swiss Army Knife” that is the Boris FX Suite.
So they will be entitled to it, it is very much available to them and that may encourage them to look into it and use it. And of course, we are working on training materials for everybody in the Boris FX Family, to use Syntheyes to its fullest potential!
