Net3 ACN Gateway Crashing - N34G-4OUT

Hey All,

I'm a dealer as well as a freelancer here in Chicago.

Today I'm out at the set of Empire Season 2 helping them shoot a couple of shots in studio.  They are having an issue that I need help troubleshooting.  I sold them the hardware (new Ion, new gateway, and a bunch of other things) around June of this year, so I have a number of vested interests in getting this fixed for them.

They have this gateway setup on a very simple network:  Netgear Prosafe Switch FS108P, Ion 2k, Airport WAP (for Ipad Mini), and this NET3 gateway.  They have an identical setup for some other stages here and are experiencing the same crashing issues.  These rigs are outputting between 600 and 1200 addresses depending on which stage they are on on a given day.  They are on for an average of 15 hours a day, and 5 or 6 days a week, and are power cycled at least once a day, usually twice a day (during moves).

4 times since July 1 they have had a gateway just completely drop connection to the consoles.  The gateway doesn't loose power, it shows as not having connectivity to the console (all four ports flashing).  They have cycled power on the entire control rig (including switches and consoles) and had it come back up.  This isn't during fades, large changes or anything.  Just sitting idle for hours, and then bang, nothing.  I'm not clear if the dimmers maintain state or turn off, but I can find out if that would inform the situation.

Today being a shooting day, there isn't much I can do to troubleshoot, or really even poke around in the GCE.  However, what should I have them look for?

The first things I thought of to check is:

Turn off RDM

Check ethernet cables

Ensure static IPs

Cycle power routinely (but they say they do already)

What else should I recommend.  It is a pretty new node, and in fact, if I had to guess, it likely isn't the node that is the issue.  To me it sounds like a networking issue.  The FS108P is unmanaged, but either way if it was working great, I wouldn't think that switch would dump connectivity for no reason.

Also, this happens to both rigs, which are both almost completely new, so it sounds like a config issue more than an anomaly.  Is there a firmware update for these gateways?  What else am I missing.

Tech support, if you want to reach out, I can send SNs, software versions, or whatever else you might want to know.

Thanks all,

Casey

Parents Reply Children
  • Corey,
    This is Philip Wheeler, a tech working on this KarenG. This is Gamut Theatre install (#300439700, Job #15037024)
    Ion is not crashing - the 'crash' is all lights out, then the Ion still functions but nothing happens
    Two Net3 2-port Gateways, one Net3 4-port, all in PoE side of the switch, Ion and dRD12 unpowered
    Still haven't confirmed IP address of the NetGear switch - all others (as KG stated) are in correct 10.101.xx.xx range
    Subnet Masks are all 255.255.0.0
    IP Gateways were all 0.0.0.0 (or 1.1.1.1, can't recall @ the moment)
    BUT...all are Dynamic. A very clear next step to me is try with Static IP in the Gateways.
    RDM Disabled on all Gateway outputs
    If I can cause the light dump with any change to the control pathway - which I can - then (I'm guessing) anytime Dynamic IP considers a change it could be a cause.
    All Gateways are PoE using the NetGear switch. Another next step could be wallwarts.

    Once set, what must be altered in the Ion to ensure good continual control comm?
    What can we check/confirm on the Ion to establish/verify this?
  • Hi Philipp,

    Could be an issue with settings on Managed Switch. I recommend unmanaged Switches. 

    Could be an issue with a same IP Address on multiple devices. 

    Could be an issue with the EOs Software or Ion Hardware.

    How often does this issue occur? How long does it last? How does problem go away? 

    Would be good to get sACNview Software running on a PC in network when problem occurs to see if console is sending sACN. 

    - Set all IP Addresses to Static in 10.101. Subnet with Subnet Mask 255.255.0.0 and Default Gateway/Router IP of 10.101.1.1. 

    - Create a Network Plan of all devices and how they are connected and their network address info. 

    - Disable DHCP Servers in Network

    - Look at Settings in Switch. 

    - If problem continues, send us results of sACNview when problem is occurring plus Eos Logfiles, with time/date of occurrence and answers to above questions. 

    - PoE wouldn't be an issue here. 

    Thanks and Good Luck!

  • Corey - thanks so much!
    The problem occurs seemingly randomly, but we've seen it happen with fixture manipulations (e.g. getting a mover up & running) or something that causes a solid level of signal through the system. Once the lights go out, Ion can no longer perform any functions with the fixtures until the P-ACP is reset or the whole system is cold-booted (power cycle of Ion only does not fix)
    - Managed vs. Unmanaged switches?
    I'm working on getting an sACN view, working on getting info on the switch (software''s not cooperating)
    Gonna call to confirm tinier questions on configuring the gateways, then do that.
    Still wondering what confirming I can be doing on the Ion setup/configuration regarding this issue
  • Paradigm is also involved?

    Maybe the Paradigm is causing issues? Who programmed the Paradigm? They need to be involved in this troubleshooting process. 

    Alone that you say that the problem stops when ypu reset Paradigm, points to an issue involving Paradigm. 

    You could set the sACN Priority of Ion to 200 and see if problem stops. This should only be done with caution though - not that this interferes with the Paradigm's control of certain address like relays or such. 

    And then afterwards you need to determine where the issue is. 

    As suggested - you should involve the person who programmed the paradigm here. 

Related