Net3 ACN Gateway Crashing - N34G-4OUT

Hey All,

I'm a dealer as well as a freelancer here in Chicago.

Today I'm out at the set of Empire Season 2 helping them shoot a couple of shots in studio.  They are having an issue that I need help troubleshooting.  I sold them the hardware (new Ion, new gateway, and a bunch of other things) around June of this year, so I have a number of vested interests in getting this fixed for them.

They have this gateway setup on a very simple network:  Netgear Prosafe Switch FS108P, Ion 2k, Airport WAP (for Ipad Mini), and this NET3 gateway.  They have an identical setup for some other stages here and are experiencing the same crashing issues.  These rigs are outputting between 600 and 1200 addresses depending on which stage they are on on a given day.  They are on for an average of 15 hours a day, and 5 or 6 days a week, and are power cycled at least once a day, usually twice a day (during moves).

4 times since July 1 they have had a gateway just completely drop connection to the consoles.  The gateway doesn't loose power, it shows as not having connectivity to the console (all four ports flashing).  They have cycled power on the entire control rig (including switches and consoles) and had it come back up.  This isn't during fades, large changes or anything.  Just sitting idle for hours, and then bang, nothing.  I'm not clear if the dimmers maintain state or turn off, but I can find out if that would inform the situation.

Today being a shooting day, there isn't much I can do to troubleshoot, or really even poke around in the GCE.  However, what should I have them look for?

The first things I thought of to check is:

Turn off RDM

Check ethernet cables

Ensure static IPs

Cycle power routinely (but they say they do already)

What else should I recommend.  It is a pretty new node, and in fact, if I had to guess, it likely isn't the node that is the issue.  To me it sounds like a networking issue.  The FS108P is unmanaged, but either way if it was working great, I wouldn't think that switch would dump connectivity for no reason.

Also, this happens to both rigs, which are both almost completely new, so it sounds like a config issue more than an anomaly.  Is there a firmware update for these gateways?  What else am I missing.

Tech support, if you want to reach out, I can send SNs, software versions, or whatever else you might want to know.

Thanks all,

Casey

Parents
  • If you continue to have problems... Try connecting the console to gateway directly, with a network crossover cable, and powering gateway off of a wallwart. (Thus eliminating the switch and poe, for testing...)
  • Just for your reference and convenience:

    A cross-over cable is not required with any of our current generation consoles or gateways due to Auto MDI-X (https://en.wikipedia.org/wiki/Medium-dependent_interface#Auto_MDI-X

  • Is the more info on this thread? I have a brand new system which is crashing, network error sounds very similar.  I have Ion, 4-port and 2-port Net3 gateways, net gear power safe 8-port network switch with 4-port PoE.  All cat5e has been checked for standard B and length.  IP addresses are in the correct 10.101.range.  i appreciate any insight!  I have not even be able to build a show yet because I can't get the system stable.  Thanks. 

  • Hi Karen,

    What is crashing? Is the Eos Software on Ion crashing? 

    Could you list all three parts of the Ion, and Gateways here? 

    Network Address Settings: 

    Ion IP:

    Subnet Mask: 255.x.x.x

    Default GatewaY: x.x.x.x

    Thanks, 

  • Hello Karen,

    What is the exact model number of the switch?

    Are the symptoms by any chance: works for awhile (about 5mins) then stop?

    Regards,

    Marcus
  • Corey,
    This is Philip Wheeler, a tech working on this KarenG. This is Gamut Theatre install (#300439700, Job #15037024)
    Ion is not crashing - the 'crash' is all lights out, then the Ion still functions but nothing happens
    Two Net3 2-port Gateways, one Net3 4-port, all in PoE side of the switch, Ion and dRD12 unpowered
    Still haven't confirmed IP address of the NetGear switch - all others (as KG stated) are in correct 10.101.xx.xx range
    Subnet Masks are all 255.255.0.0
    IP Gateways were all 0.0.0.0 (or 1.1.1.1, can't recall @ the moment)
    BUT...all are Dynamic. A very clear next step to me is try with Static IP in the Gateways.
    RDM Disabled on all Gateway outputs
    If I can cause the light dump with any change to the control pathway - which I can - then (I'm guessing) anytime Dynamic IP considers a change it could be a cause.
    All Gateways are PoE using the NetGear switch. Another next step could be wallwarts.

    Once set, what must be altered in the Ion to ensure good continual control comm?
    What can we check/confirm on the Ion to establish/verify this?
  • Netgear Prosafe GS108PE
    Main symptom: works most of the time, but some change is attempted and all the lights go out - see also my reply to corey
  • Hi Philipp,

    Could be an issue with settings on Managed Switch. I recommend unmanaged Switches. 

    Could be an issue with a same IP Address on multiple devices. 

    Could be an issue with the EOs Software or Ion Hardware.

    How often does this issue occur? How long does it last? How does problem go away? 

    Would be good to get sACNview Software running on a PC in network when problem occurs to see if console is sending sACN. 

    - Set all IP Addresses to Static in 10.101. Subnet with Subnet Mask 255.255.0.0 and Default Gateway/Router IP of 10.101.1.1. 

    - Create a Network Plan of all devices and how they are connected and their network address info. 

    - Disable DHCP Servers in Network

    - Look at Settings in Switch. 

    - If problem continues, send us results of sACNview when problem is occurring plus Eos Logfiles, with time/date of occurrence and answers to above questions. 

    - PoE wouldn't be an issue here. 

    Thanks and Good Luck!

  • Corey - thanks so much!
    The problem occurs seemingly randomly, but we've seen it happen with fixture manipulations (e.g. getting a mover up & running) or something that causes a solid level of signal through the system. Once the lights go out, Ion can no longer perform any functions with the fixtures until the P-ACP is reset or the whole system is cold-booted (power cycle of Ion only does not fix)
    - Managed vs. Unmanaged switches?
    I'm working on getting an sACN view, working on getting info on the switch (software''s not cooperating)
    Gonna call to confirm tinier questions on configuring the gateways, then do that.
    Still wondering what confirming I can be doing on the Ion setup/configuration regarding this issue
Reply
  • Corey - thanks so much!
    The problem occurs seemingly randomly, but we've seen it happen with fixture manipulations (e.g. getting a mover up & running) or something that causes a solid level of signal through the system. Once the lights go out, Ion can no longer perform any functions with the fixtures until the P-ACP is reset or the whole system is cold-booted (power cycle of Ion only does not fix)
    - Managed vs. Unmanaged switches?
    I'm working on getting an sACN view, working on getting info on the switch (software''s not cooperating)
    Gonna call to confirm tinier questions on configuring the gateways, then do that.
    Still wondering what confirming I can be doing on the Ion setup/configuration regarding this issue
Children
  • Paradigm is also involved?

    Maybe the Paradigm is causing issues? Who programmed the Paradigm? They need to be involved in this troubleshooting process. 

    Alone that you say that the problem stops when ypu reset Paradigm, points to an issue involving Paradigm. 

    You could set the sACN Priority of Ion to 200 and see if problem stops. This should only be done with caution though - not that this interferes with the Paradigm's control of certain address like relays or such. 

    And then afterwards you need to determine where the issue is. 

    As suggested - you should involve the person who programmed the paradigm here. 

Related