Network slows down after enabling ACN with CEM+ modules

First of all, I'd suggest upgrading your Ion to the latest version (1.9.6). That may be it, as I believe there were several improvements to how full-ACN traffic is handled between 1.9.1 and 1.9.6

Which version of CEM+ and RFR (handset and base) firmware are you using?

Jay

I agree with Richard in that you should be running the latest version of Ion Code V1.9.6.  With that said you should also be running the latest version of CEM+ code V3.1.1, to receive rack feedback.  There was also a point in which we moved from the Draft version of ACN to a Ratified version.  the other thing is to try a complete reboot of all devices including switches on the network.  There have been known issues with network crashes partially taking down the network.  The other question is are these manages switches.  If so you may want to try to turn off the management side of the switches or put in a simple switch.  Net3 uses multicasting to communicate with other Net3 devices on the system.  This could be part of the bottle neck.

P-

Ok,

I upgraded the console to v1.9.6.9.0.3

CEM+ Modules are all v3.1.1.9.0.36

The switches I am using are unmanaged (not managed)

The RFR was an older revision but I used the GCE to update it to v2.07.

As of right now my problem is the same.  If I unplug the racks from the switch, both my RFR and iRFR respond fast as before.  When the racks on on the switch the RFR and iRFR both respond with a very HIGH lag time.  so much that it's not useful.

My network settings in the ION: Local IP Settings

• (DHCP OFF)
• IP Address 10.101.90.101
• subnet mast 255.255.0.0
• Gateway 10.101.1.1

• Network Services - on
• Address Service (DHCP) - on
• First Address 10.101.50.101
• Number of addresses = 100
• subnet Mask 255.255.0.0
• Enable Routed Network = OFF
• Default IP Gateway = blank

Protocols

• ANC + Local DMX
• Sensor Rack Feedback = ON

For the rest you can refer back to my attached .pdf in the first post (keeping in mind I updated the ION & RFR)

Hi Jaylaca,

First I have a clarification question. In your PDF, you have rack 5 and rack 6 with the same IP address. Is this a typo or do they actually have the same IP address?

Your PDF mentions a static IP address for your WAP but Dynamic Subnet Mask and Gateway. Can you confirm which of these settings is correct and what the current values are?

Have you tried swapping ports on the switches to eliminate a bad port?

Have you tried connecting each Sensor+ Rack one at a time to see if there might be an issue with only one of the racks that could be interfering with your network?

You mentioned a 250ft run of network cable. Is that total, or in addition to what is installed in the wall?

You may benefit from talking this through with your local ETC Tech Services:

Americas - 1.608.831.4116 or 1.800.688.4116
London - +44 (0)20 8896 1000
Germany - +49 (8024) 47 00-0
Hong Kong - (+852) 2799 1220

starksk said:

First I have a clarification question. In your PDF, you have rack 5 and rack 6 with the same IP address. Is this a typo or do they actually have the same IP address?  That's a typo.  Rack 5 = 10.101.90.105  Rack 6 = 10.101.90.106

Your PDF mentions a static IP address for your WAP but Dynamic Subnet Mask and Gateway. Can you confirm which of these settings is correct and what the current values are? The linksys WAP is set get its TCP/IP configuration from the DHCP server on your network (the ION's Network Services). I believe I set it's ip to 101.101.15.101 to set it up from my laptop.  Then I set it's  DHCP to "Client".  After I did that I could no longer access it directly since I don't have client services on my laptop.  But it works great on it's own with my iPod.

Have you tried swapping ports on the switches to eliminate a bad port?  Yes, tested all ports with on-lot tech

Have you tried connecting each Sensor+ Rack one at a time to see if there might be an issue with only one of the racks that could be interfering with your network?

You mentioned a 250ft run of network cable. Is that total, or in addition to what is installed in the wall?  That's total. 3 ft cable from 8-port switch to board.

I'll call tech support tomorrow, they' closed now.

So what was the fix for the issue?  Inquiring minds want to know....

• 1. If you unplug the racks from the switch, the response time improves and is acceptable. Can you try turning 'Off' Sensor Rack Feedback"?
• 2. Are there any errors on the racks during this sluggishness?
• 3. Could you set all Devices Gateway IP's to to 10.101.1.1 and see if this makes the suggishness go away?         
• 4. Are there any other devices on the network (lighting gear or pc)?
• 5. Is there a particular show file involved that we could get?
• 6. Could we also get the CEM+ config?

thanks

@ Corey - I'll work on getting this information posted

But so far no resolution.  My work around is to have the Sensor Racks connected directly to my laptop and run DimStat and also connect via ETConnect through Internet Explorer. 

Just to note, it all works.  Only the RFR and iRFR  lag out when the racks are connected to the ION.  I've got an updated RFR on the way with v2.07.  But my iPod is also super slow to respond, so I don't think it's the RFR firmware.

I had re-imaged the ION to v 1.9.6, so everything is running the latest versions.

The last I could figure out before my show wrapped was that with StreamingACN and Sensor Rack Feedback enabled in the Ion's Networking setup, it just slowed the network down too much for either the RFR or iRFR (iPod) to work.  The latency was 5 to 15 seconds for any button-push/command on either RFR.  And it didn't matter if the RFR was USB or Ethernet.

The functions on the console all worked.  The Ion could see, control, and change any setting with the Sensor CEM+ racks themselves (such as Curves, etc)  But it made the RFR's just way too slow and completely unusable.  When I unplugged the network cable going to the dimmer room (MiLan switch and racks), everything worked fast again.

My work-around was to keep the racks plugged into my PC laptop running DimStat software and keep that network separate from the board/rfr setup.

Anyone wanting to run DimStat on their laptop, you will need to reference this limitation: http://community.etcconnect.com/wikis/products/knowledgebase-sensor-connect-with-internet-explorer-8.aspx

Is this Challenge resolved yet?  If not.....U say the dimmer racks and Console are connected via DMX cabling AND Net3. 2 me sounds like the items running only on the network(RFR & iRFR) want the console to talk  via Net3 to the rack while the console wants to talk via DMX. this could cause a signal loop. Try unplugging the DMX from the console; forcing everything to use Net3 to communicate. 

Jake.

You know, I never thought to do that.  But what you say makes sense in hindsight because when I unplugged the Ethernet  from the Racks (to the ION), and just had the DMX connected, everything went back to full speed with the RFR's and iRFR. 

The show is wrapped, so I'll test that next time I have a setup to play with....

So. I have nearly the exact same problem. 5 racks SP48 2.4k, SP6 (6k), 3 SP12 (12k) All CEM+ running 3.1.2

Ion is running the latest Beta.1.9.11 Build 23

Rack networking, but each rack is in its own group. I would have preferred to set up a group of all 5 racks, but they simply wouldn't recognize each other, and I could at least get the Ion to recognize all 5 and make adjustments from the console.

Wireless is an Apple Airpot Extreme (AAE). iPhone and iPad. Ethernet is connected from console to Ethernet LAN port (WAN is empty). Additional LAN ports feed a switch at Racks, and laptop to configure the router. 10.101.124.101 and router address 10.101.1.1. Bridge mode.

I have been through nearly every combination of trying to get the console to be the DHCP server with the AAE in bridge mode, or have the AAE as the DHCP server. Router addresses as the device's own IP and as 10.101.1.1 Subnet mask is always 255.255.0.0.

Currently iPad & iPhone each work (and only work) with static IPs 10.101.125.101 & 102 Router 10.101.1.1 Console is 10.101.100.101 AEE is 10.101.124.101, with 10.101.1.1 in

At the console networking is off, console has a manual address, with gateway as 10.101.1.1

As soon as I connect the cable running to the racks (through unmanaged ethernet switches) and ACN & DMX on, the i devices start to have a multiple second lag, and I bet the signal strength of the console would drop from 5 bars to 1 though all the CEM+s show up on the ION About screen.

This shoot lasts 4 days (and this is day 3) but I set up all of my shows with networked Sensor racks, an Ion and a wireless router. This all worked as expected until the 5.0+ phone updates. Devices are on 5.1 currently.

So I guess I am asking what am I doing wrong?

Console: 

Should network services be on? DHCP? Routered network? Gateway addresses? I have set them to the IP settings here (http://community.etcconnect.com/wikis/products/knowledgebase-etc-network-ip-addresses.aspx)

Base Station

I couldn't get it to work either receiving or sending DHCP. What should the router address be? Always 10.101.1.1 for all devices?

Finally, I suppose the cable to the racks could be the problem, but the similarity between my case and the case above makes me think not. I cannot turn off DMX since I don't have a gateway, we are shooting today, and I am controlling our racks, a bunch of Kinoflo Image 85s, and 5 installed Sensor racks of the house system on old CEMs. I am using the lower part of one universe and the upper part of another for our racks and theirs.

If this relates to why the CEM+s wouldn't talk to each other I am all for it. Last time it happened on a different job it was because of a bad CEM+ that I kept kicking up the chain as I had only one spare instead of a spare for each rack, but, I was satisfied with putting each rack in its own group. 10.101.10x.101 with gateways the same.

Any assistance would be welcome.

James McNeal

ETCP Certified Entertainment Electrician

IATSE Local #52, ACT

Programmer, HBO Films Project

As an additional note. I can access all racks through Web Connect independent of the network. In the past I could use Web Connect, ACN, iRFR all at the same time.

James

Your description of the network sounds very confused, I think you've been running in circles for a while and lost sight of the big picture.

Next time, grab a bit of paper and sketch out exactly what is connected to where and the IP address it has. This really helps find any mistakes.

My guess is duplicate IP addresses or even a circular link somewhere.
- It is very important to have a strict star configuration with no loops*. Loops can have all kinds of strange effects.

I'm unsure what you mean by:

mcliln said:

As an additional note. I can access all racks through Web Connect independent of the network.

Do you mean that if you unplug the rack from the network and connect your laptop directly then it works, while on the main network it does not?
Most likely cause of that would be a bad cable, having eliminated that it may be duplicate IP addresses somewhere or no route from laptop to Sensor+ rack.

Ping is very useful when diagnosing network issues as it can often tell you which of those happened.

* Strictly speaking redundant links are possible with fully-managed switches, but you'd really want a computer networking specialist to set those up as they can be quite complex to fault find.

Sorry for the confusion.