Integrated PTZ Camera (physical) control and programming with Eos and Augment3d

This observaton/discussion could have found its way into a different forum but the implications are wide enough that I thought it belonged here.

We are a performing arts academy with a black box theater shifting into multi-camera capture and live streaming of performances in HD for now. We have installed robust IP based video transport using Newtek's NDI protocol and have installed a collection of PTZ cameras in the black box. IP control allows for a great deal of control of the various camera attributes but the software currently available leaves much to be desired in theatrical settings.

Recently we replaced our old Smartfade 24/96 with Nomad touch screen monitors and a 2x20 wing. The convergence of these two projects has sparked an idea which, I think could revolutionize and greatly simplify the integration of physical PTZ cameras into both busking and cue-based productions. 

PTZ cameras are in many ways analogous to moving lights. But, instead of projecting light, they are capturing it. The physical movement attributes are pretty much identical. The difference is the need to control, often on a cue by cue basis, things like sensor gain, dynamic range, noise reduction, etc that need to be adjusted in order to achieve maximum image quality in different lighting conditions.

The GUI, workflow, flexibility and moving camera control of Eos are a natural fit for PTZ camera control. Eos already contains needed functionality like move in black, cue stacks, park, etc. that if implemented for PTZ camera control would be reason enough for organizations moving to new media distribution to consider. However, there's much more.

Augment3d already relies deeply on the concept of virtual cameras to provide visualization. Imagine being able to use it not only to design and program what the live audience will see, but also what the live camera will capture.

An integrated stack of lighting and camera cues would allow one to step through the show from both the perspective of the live audience and/or each camera during programming. When live, cameras would always be in exactly the right place at the right time and trigger a video switcher as needed to provide the correct transition and handle any other audio/video centric elements like streaming and capture.

I realize that some of the pieces are in place to do this using OSC/Midi etc. Perhaps there are folks out there who've already created profiles for PTZ cameras but I'm not aware of any.

1) Are ETC thinking about this as a development/business opportunity?

2) Any forum members already dabbling with this?

Cheers all!

  • If I had to guess this isn’t likely on ETC’s R&D roadmap as it’s an industry and technology that already exists for Broadcast. The EOS integration would be UDP string triggers to the playback system of choice for Camera Automation. That “cue” playback is often handled by the TD (studio version not Theater). That may not be available with your current system, but it is definitely a technology that is out there in the world. 

    Hope this doesn’t burst your bubble, you’re definitely on the right thought process and we need talented programmers with cue structured minds for camera and switching automation. If you’re like me you might be able to combine both worlds of a Love for EOS and all things ETC along with broadcast television media. 

  • If this does already exist for broadcast, I assume it's only available for the big players otherwise I'd be able to find it.

    Lots of small to medium size venues that have an Ion and a wing or two are having to pivot to support streaming and performance capture. Products like Tricasters, ATEM Mini's, OBS and vMix along with a truckload of decent quality $1,500 PTZ are finding their ways into these venues yet the control paradigm is simply lacking.

    I believe with a relatively small amount of development work Eos/Augment3d could be dramatically more valuable for those of us who are merging live performance with video streaming and capture.

  • Well, this is where as an integrator and system designer I would ask you... is your goal budget driven or solution driven? It sounds like you've found a solution. However, that solution may not have provided robotic scene recall control. If that had a been known goal at your outset then hopefully your integrator would have sent you down that path. That doesn't mean it can't be added... but I need a little more information and I might be able to point you in the right direction. What is your camera environment and control platform? There might be a solution that exists there without needing a bigger boat...

    Given the charitable assumption by your mention of Newtek is it safe to say you have the Newtek NDI family of cameras? The NDIHX perhaps? Tricaster's can be fully functional production switchers, in fact, I use them in the field. Depending on your flavor of switcher and control surface you can trigger macros that utilize tricaster's PTZ UI to recall presets. Then it's simply a matter of getting your GPI triggers set up and you're good to go. I don't believe the same is true for the non-hardware emulators from Blackmagic, OBS and VMIX... 

    I don't think it's a case of only being available to the big players... If we're talking about the fully robotic studio cameras with smart sensing pedestals that can be run remotely from across the country then of course the budget is going to be big, but any budget can be big if the need calls for it and the resources exist.

  • Our budget is large to us, no doubt tiny to others. We are using Minnray NDI ptz's plus some ZCam E2c Ndi enabled cameras, vMix running on a robust PC, the transport layer is 100% NDI running on managed switches and newly installed Cat6e. I've probably spent about $15k thus far. Most of the switcher functionality is stuff we'll never use for what we're doing. All of the low-end switcher solutions I've seen have been oriented toward, worship, HS sports and public access use cases.

    The EOS implementation I have is Nomad running on a PC, a touchscreen and a 2X20 Wing.  Works great for the mixture of < 50 conventional and a few movers.

    We have three student's helping us with this transition from a local College, two graduating this spring with degrees in Broadcast Media Production and one with a BFA in Film. 

    For the majority of small theatres and performing arts schools, Covid-19 is an existential threat. We are repurposing some grant dollars to create a learning and performance environment that isn't constrained by three studios and an 80 seat black box that's capped at 30% normal capacity. 

    We know of several other organizations near us that are trying to do similar things as a way to prepare for a more hybrid distribution model post covid. 

    I can, no doubt piece together the small amount of tech needed to solve the few control problems that I have with a combination of vMix, camera and integration tools like Stream Deck and Companion. That wasn't the point of the post. 

    I haven't heard anyone react to my proposal that adding the function points needed to support PTZ camera control in EOS  would create a synergy for design, programming and execution that when integrated with Augment3d would be very useful no matter the price point.

    I look at the moving fixture controls in Eos and it's got camera control written all over it. I'd love to only write one cue list to control them both. 

  • Forgive my ignorance, is there a standard for PTZ camera control? Everything I'm finding seems to be manufacturer specific.

    Also, why not build a box that takes sACN, ArtNet, or DMX and converts it to PTZ and just build a fixture profile for it like it were any other fixture in your rig?

  • there was a post in the Eos Programmers facebook group some years ago from someone who had written a software to take sACN and convert it to some form of PTZ. maybe this will help you:

    www.facebook.com/.../

  • There are many. The predominant one seems to be VISCA (developed by Sony). The Visca over IP implementation runs fine over NDI. For now, your suggestion is exactly what I'm planning on trying.

    But... I still say this is an emerging market for ETC and having native PTZ camera support in Eos would open up a big market. For performance facilities fitting up for streaming.

Related