Difficult-to-Explain Outages

WTC Communications is an independent telecommunications company based in Ontario, Canada, that has been providing voice, data and networking solutions to business and residential clients for almost 90 years. For its video service, consisting of 300 HD/SD programs, WTC had been receiving source signals solely via satellite to a local headend. Kevin Cristy, Video Manager at WTC, found the satellite feeds to be pretty reliable, however occasional signal outages would occur due to heavy wind or rain. Kevin was also concerned about having all eggs in one basket. When the satellite feeds did exhibit errors or a complete outage, there was no backup service in place which left viewers with a poor video experience.

WTC are also a wholesaler and provide video signals to four other video providers who are geographically located significant distances away.

“When it’s pouring rain, people usually know that video signals can be affected,” says Kevin. “But when you’re 300km away and the sun is shining, it’s difficult to understand video outages. Once you’re serving a region of a certain size, it’s not acceptable to have these outages.”

There were two things that WTC were looking to do:
1. Find a more reliable source for video ingest feeds.
2. Configure some kind of redundancy to maximize service uptime for video subscribers.

To solve the first problem, Kevin turned to his own network and learned that many other video operators were switching to fiber-based sources. Fiber-optic video transmission has some immediate advantages over satellite as video stream delivery is usually more reliable during harsh weather conditions. However, service outages can still occur.

“For us, the fiber feeds would be coming to us from Toronto which is a few hundred kilometres away,” explained Kevin. “That distance is obviously a potential risk.” This confirmed that video redundancy was still required, and so WTC started investigating the options.

The Search for Redundancy

There was the option of configuring redundancy within the network using backup feeds over fiber. In this scenario, each service would have an A feed and a B feed delivered via the same fiber-optic transmission. “The problem here is that the A and B feeds are sourced from the same pot,” said Kevin. “This means that fiber cuts at certain points in the provider’s network can still take out both feeds.”

WTC also explored the redundancy capabilities built into the set-top box implemented by their middleware provider. The problem with this option was that the failover is post-encryption. It wasn’t possible to failover to a backup feed before the stream goes into the network, which meant that the entire line-up had to be encrypted twice. This doubles the traffic being sent into the network and would increase overall costs significantly.

WTC knew they wanted a robust solution with a seamless failover experience and recognized that there was already an investment made in the satellite headend. It made sense to bring the fiber feeds in as primary sources and use the satellite as secondary sources. This is where transcoding became a vital part of the concept.

Transcoding is Key

The satellite provider used by WTC is in the process of transitioning to MPEG-4 but currently they provide a mix of MPEG-2 and MPEG-4 AVC video sources. The fiber provider also delivers a mix of MPEG-2 and MPEG-4 AVC and is transitioning as well, but at a slower pace than satellite.

“There are a lot of MPEG-2 streams on the fiber side to feed legacy systems, and many of these are not groomed,” explained Kevin. “The fiber feeds come in at approximately 18 to 20 Mbps, and one is even at 30 Mbps. We can’t send the streams out that way. We need all video outputs delivered as MPEG-4 AVC with a total payload of 8 Mbps, for HD services. The transcoding is key.”

The solution needed to easily switch between the two multicast sources when errors are detected, transcode them to the required MPEG-4 output format and deliver them efficiently to subscribers. This is where Automatic Service Failover (SFO) in the Inca 4400 Modular Series came in as the optimal solution. With Automatic Service Failover, the Inca chassis monitors up to 16 PIDs in the primary source, with user-configurable parameters and thresholds. The chassis can be configured to automatically fail over to a secondary or backup source when any of the trigger conditions are met. “Automatic Service Failover provided a nice solution that could feed easily into our encryption system,” added Kevin. “It fit the bill perfectly.”

When WTC realized that failover within the transcoder was the way to go, they didn’t research any other vendors. “We made our transcoding decision a few years ago with Inca,” explained Kevin. “We’ve been so happy with the Inca equipment. We haven’t bothered to look at anything else.”

Straightforward Deployment

Automatic Service Failover (SFO) is configured in the 4400 Modular Series using Inca’s award-winning user interface VidiOS™ for management and analysis. “Set-up was pretty straightforward,” reports Kevin. “Just like everything else in the UI, it’s pretty self-explanatory.”

Since Automatic Service Failover has been set-up on WTC’s transcoders, there have been a couple of outages. WTC uses the Inca All Seeing Eye for confidence monitoring and have this displayed on a large TV for the support team. “Anytime a source goes out, everybody’s got eyes on it,” said Kevin.

The first outage that occurred after Automatic Service Failover configuration was caused by a fiber cut. During this event, the advantages of SFO were evident right away.

“By the time we noticed that the sources were down, the streams configured for Automatic Service Failover were already running with the backup source,” he explained. “In fact, we sort of exposed that we need to finish this project. Some of the streams that were not set-up for Automatic Service Failover triggered customer calls. We had to go in and manually change the source settings to pull the satellite feeds for those channels.”

WTC later learned that other video operators who had configured redundancy using the A and B feeds from the same fiber provider lost both primary and backup sources when the fiber cut took place. In other words, they had a complete outage. WTC’s careful research and smart decision-making paid off. Using the existing satellite feed as the backup source in case the primary fiber transmission went down was an ideal solution for WTC. Using Inca’s 4400 Modular Series with Automatic Service Failover to deploy this solution worked seamlessly and demonstrated advantages immediately when it really mattered.