Large Radio Network Considerations and Solutions

The purpose of this guide is to help various engineers and decision makers understand limitations of the different protocols as well as guide deployment teams to design their networks around said limitations.

Large Networks do not behave in the same manner of small or medium-sized networks.  While this statement is true, the definition of network size has changed somewhat during the last 10-15 years. 

What was once considered a large network is now only of middling size as Digi’s XBee customers have begun deploying larger networks or many networks within a specific geographic region.

At this point in time, the following would be appropriate in terms of network size description.

2-20 nodes = Small Network

20-60 nodes = Medium Network

60 – 180 nodes = Large Network

180 + = While possible, it is not recommended / split up the network

It is true that there are networks that are functioning as desired that run larger than 180 nodes. However, these are hard to maintain and generally don’t allow for much data per node to traverse the network.

Let us look at a few scenarios and discuss what would be needed / appropriate for each in terms of protocol type and size of network.

Scenario 1:  Remotely-Located Solar Farm with 10,000 radio nodes

In this scenario, which is actually more common than one would think, we have a solar farm in a remote location.  This is relevant as outside RF interferers are less likely than in an urban environment.

This farm has 10,000 solar panel pylons that can track the sun on 3 axes.  Each panel pylon requires an XBee radio to transmit or receive positional data as well as positional commands.

Each inverter platform handles up to ~400 of these panel pylons.  To get a better idea of the scale of each section, along with distances involved, please refer to the image below:

On this particular solar farm, there are 50 inverter platforms.  (They are seen above as little rectangles just off the roads going between sections)

Originally the customer wished to use as few networks as was possible.  However, after trial and error, they found that too many nodes per network didn’t allow for timely or even reliable data transfer to occur.  In the end, each platform hosted two gateways.  Each gateway would handle up to 200 nodes as separate networks.

The main factors are as follows:

• Node Density
  • Because the land is flat and the distances are fairly short between nodes, either PRO and non-PRO radios can be used.  If using PRO, then turning down the power on the remote nodes is appropriate.  When nodes are this close, full power can create too much noise.  The analogy of people talking in a small space would be apropos.  By turning down the power output, there is a much better chance of the signals not interfering one with another and prevents inter-network interference.
• Node Layout
  • The solar panels do move.  This causes routes to change throughout the day.  However, the movement of the panels is slow and as such the update of routes occurs frequently enough that transmission failures are not very common. 
  • Nodes that lose direct LOS to the coordinator retain LOS to neighboring nodes that do have direct LOS to the coordinator. 
    • In ZB network this can be more interruptive than in Digimesh networks that use directed broadcasting.
    • If this were a smaller deployment (both numbers as well as geographically) using a non-routing protocol such as 802.15.4 might be appropriate.  However, since the geography, and variability of line-of-sight are factors, it is suggested that one use a routing protocol such as ZigBee or DigiMesh.

• Data Transmission Frequency
  • Solar farms don’t generally transmit more than 1 panel position per minute per panel-array.  This is well within acceptable data frequency ranges.
    • There are times, however, when this frequency may increase.  These might include the following:
      • Emergency weather event broadcasts
      • System firmware updates
      • XBee module firmware updates
      • Module deployment or replacement
• Data Transmission Packet Size
  • Each TX packet isn’t larger than 60 bytes, thus avoiding packet fragmentation.  This particular customer kept all data transmissions at or under this threshold so as to avoid congestion caused by packet-fragmentation.
• Broadcast Event Frequency
  • Broadcast events (non-ZB stack maintenance) don’t occur except in the case of weather events that require all panels to rotate to a stow position.  This is particularly important when using the ZigBee protocol.  The ZigBee protocol only allows up to 8 broadcast events to be present on the network at any given time.  In our experience, a broadcast can occur roughly 1 time per second barring any ZB stack maintenance broadcasts are happening.
    • Of interest, the DigiMesh protocol does not have this limitation.
• Channel Selection
  • Each section has a 2 channel offset from any adjacent section.  This helps avoid interference from one network to the other.  While a 1 channel offset should be enough, 2 channels were thought to be a better option as a precaution.
  • Because this deployment is not in a populated area, chances that Wi-Fi would be a problem are extremely unlikely.

Scenario 2:  Indoor Lighting on Single and Multiple Floors

Most of the same issues arise for this particular scenario with a few exceptions.

• PRO variant radios should only be used in coordinator roles.
  • All remote radios should use non-PRO as the density can be too great and interfering signals (mostly network maintenance traffic) can degrade network performance. 

• Special attention to the Wi-Fi bands being used in the area should be given.
  • If there are only going to be a few networks, then deploying them with channel masks to only scan on non-Wi-Fi network channels is suggested. 
  • Avoid physical co-locating Wi-Fi access points if possible.  These create great levels of noise and degrade network performance.
• Bluetooth will likely be in high use in many of these applications.  While it is a factor, there isn’t much that can be done about it.  In the case of mission critical applications, controls on the use / function of BT devices in the area may need to be implemented.  This has been implemented by some customers having large manufacturing plants where safety is impacted by the use of BT devices.  Lockers acting as signal attenuators for phones and other devices were used.  This kept the noise floor relatively low.
• As lighting is predominately located within the ceiling structures of most buildings, the chance for physical interferers is very high.  These usually take the form of HVAC ductwork as well as structural ironworks.  Clear line of sight should be encouraged during installation.  However, it isn’t always an option.  Using the Network Mapping Tool in XCTU can help installers know where network links are weak or less-than-optimal.

Scenario 3:  Outdoor monitoring of various sensors in a highly dynamic environment

Dynamic environments, especially those that involve the nodes to be moving constantly, are not ideally addressed using the ZigBee protocol.  The Digimesh protocol is much better suited to handle such environments.  Nearly every other concern, previously discussed in the other applcations, is applicable.  The biggest challenge is keeping network size and location manageable. 

As node routes will constantly be changing, Directed Broadcasts are the suggested method that should be used to reach transmission destinations.

As with any new product or system, always deploy a test network that mimics as closely as possible the networks that will be deployed for use in real-world applications.  This cannot be emphasized enough.  A very large percentage of cost and headache comes when networks are deployed without proper testing.  This stage will be money and resource well spent as it will help avoid costly updates, troubleshooting and replacement / repairs.