Network issues

Hello all.
I have a couple of questions about network issues happening in my venue, a big opera house here in São Paulo, Brazil. Before I get to
them, some context.


We have a full ETC system with two consoles in multiconsole mode. My Primary is an Ion XE and my backup is an upgraded Ion Classic. Our Ion XE is our DHCP server on the network, but most devices have static IP working on a 10.101.x.y subnet. The whole system runs on sACN.
Consoles in 3.2.10 software version (we're currently in season, planning to update afterwards).
We have 9 HSR48 dimmer racks and 2 4-port sACN gateway (legacy).
We also have a bunch of remote plug-in stations distributed throughout stage and FOH.
In the booth, on top of consoles and one of the gateways, we have an Aruba 1930 24p 1G Base-T Class 4 PoE switch.
In the dimmer room, we have 2 splitters and a 3Com 2426-PWR Plus switch along with our second gateway.
This system was built around 2011, before I worked here and has worked somewhat okay for many years. But in the beginning, we didn't use our second gateway or have much complications in the network part, basically because we didn't understand it quite well. Since then, some
gMA2 users have brought their consoles and we expanded our parameter count recently. Also, the switch in the booth isn't the original one,
which wasn't okay at some point.
Now, to my questions:

1) About managed switches, what configurations do you recommend I should take a look? I left IGMP snooping enabled and disabled EEE for
I read those were important. Anything else? As a curiosity, I do think this switch takes a lot longer to boot up in relation to our old one
(we turn off the booth system at the end of the night).

2) For some time now (I can't put my finger when it started), I turn on both consoles and my primary comes online, it controls the rig, but
in the lower right corner, it shows only "User 1" and not "User 1 | Primary | Master" as it should. While in this state, the backup
doesn't connect. I have to exit to the shell and come back for it to come in the right state and establish connection between the two. Any
ideas as to why this happens?

3) We have been receiving increasingly more gMA2 users who bring their consoles and need to send data to our dimmers. Any suggestions in this scenario? We've had a strange issue in which the gMA2 console would output only 24 addresses per universe and this would only resolve with a console reboot.

4) I made a test the other day in which I connected my pc running dot2 software just for test purposes and it controlled the rig. The thing
is it still considered HTP relation between dot2 and Eos despite dot2 having a higher sACN priority (also shown in sACN Output Viewer in the
console and the sACNView software). The shown values in software didn't accord to the real output values. Any ideas as to why?

Sorry for the long post, Hope you guys can give me some ideas.
Best regards.

Parents
  • HI,

    First we have a put a lot of good information about network design and setup on our support site here - Network Design - Electronic Theatre Controls Inc

    Tried to answer each question for you in order.

    1) IGMP snooping alone can cause you issues and you also need to have an IGMP Querier in your system to manage the Multicast. 
    The documents on the support site have details on recommended setup for this.
    Managed switches do always take a long time to boot, some of them upto 10 minutes so that is to be expected.

    2) This could be a couple of things, first if the switch has not fully booted and is allowing traffic when EOS boots up then the network connection wont be made, so dropping to the shell and coming back allows EOS to detect the network correctly and will start using it. I suggest booting your switches up then give them some time before you start up your consoles.
    The other issue might be related to if you don't have the IGMP Querier running as if there multicast traffic isn't getting between the consoles then they wont be able to connect correctly, again dropping to the shell and back in is likely refreshing the network connection and allowing that traffic.

    3) If the MA system or any other consoles is able to send sACN out then it should have control of your dimmers, that said which processor is running in your sensor racks and what software version do they have? Installed in 2011 will have been at the cross over point of CEM + and CEM 3 if I have my dates correct.
    It would be good to see what sACNview is reporting when your EOS console is turned off and if that shows values for all addresses, both when connected to your network and directly to the back of the console and if there is a difference then we know its network related, I am not 100% sure exactly what might be the issue would be that might take some digging.

    4) The values should only HTP if you have matching Priorities, that said you will need to take a closer look at the universe Priority and the Address Priority. This could be the difference you are seeing. Most ETC products by default will output per address priority and depending on each manufacture they may or may not do this. If sACNview shows you the priority of each source in () then they have per address enabled and you need to click on each address to see if there is a source with a per address priority higher. That could be causing the mixed outputs.
    Screen shots of what sACNview is showing you would be helpful.

    Regards,

    Tom

  • Hi Tom.

    Sorry for the delay, it's been a crazy week and I've been studying and doing tests around here.

    The information you've given was precious. The link to Network Design gave me a lot of specific information.

    After a firmware upgrade, I found the configuration to set up the Querier and a few other recommended parameters for IGMP snooping.

    The whole multiconsole thing is exactly what you said: our switch takes something between 2 and 3 minutes to boot up. During this time, there won't be any communication in the network. I configured the network ports in both consoles to wait for network traffic. So the time is the same, but at least now, when I arrive in the afternoon, they're usually already talking to each other.

    As a method for trying to minimize this time, Brett Dikeman suggested in the ETC Eos Programmers Facebook group that I should look out for FastPort/Edge Port which I did find after a while. Didn't see any noticeable difference. So I'm trying not to turn off the switch at the end of the night.

    As for the MA problem, we have one right now in the house running a concert. We are working with CEM+ processors. My IONXE has Per Address Priority enabled. After a few tweaks, I found out a few things that worked and that I should pay attention:

    - The MA console doesn't need to have the same IP scheme as the rest of the network.
    - On MA, sACN has to be put in OutputMulticast Mode;
    - Priority has to be equal or higher than our console (this one I already knew);
    - And for the tricky one that took me a while to find out: the LocalStart and sACN Universe in sACN config has to be setup correctly according to the patch. In today's case, the guy made the dimmer patch in universe 8, but LocalStart was on 1. When we changed it to 8, everything came into place;

    As for the sACN priorities, I couldn't replicate the problem from the other day. So on this I guess I'm gonna have to keep an eye on.

    So, all in all, things are fairly stable now. I appreciate very much the time you put in ansewring my points.

    Thank you.

Reply
  • Hi Tom.

    Sorry for the delay, it's been a crazy week and I've been studying and doing tests around here.

    The information you've given was precious. The link to Network Design gave me a lot of specific information.

    After a firmware upgrade, I found the configuration to set up the Querier and a few other recommended parameters for IGMP snooping.

    The whole multiconsole thing is exactly what you said: our switch takes something between 2 and 3 minutes to boot up. During this time, there won't be any communication in the network. I configured the network ports in both consoles to wait for network traffic. So the time is the same, but at least now, when I arrive in the afternoon, they're usually already talking to each other.

    As a method for trying to minimize this time, Brett Dikeman suggested in the ETC Eos Programmers Facebook group that I should look out for FastPort/Edge Port which I did find after a while. Didn't see any noticeable difference. So I'm trying not to turn off the switch at the end of the night.

    As for the MA problem, we have one right now in the house running a concert. We are working with CEM+ processors. My IONXE has Per Address Priority enabled. After a few tweaks, I found out a few things that worked and that I should pay attention:

    - The MA console doesn't need to have the same IP scheme as the rest of the network.
    - On MA, sACN has to be put in OutputMulticast Mode;
    - Priority has to be equal or higher than our console (this one I already knew);
    - And for the tricky one that took me a while to find out: the LocalStart and sACN Universe in sACN config has to be setup correctly according to the patch. In today's case, the guy made the dimmer patch in universe 8, but LocalStart was on 1. When we changed it to 8, everything came into place;

    As for the sACN priorities, I couldn't replicate the problem from the other day. So on this I guess I'm gonna have to keep an eye on.

    So, all in all, things are fairly stable now. I appreciate very much the time you put in ansewring my points.

    Thank you.

Children
No Data
Related