Streaming Stick WiFi Performance Compared

Streaming Stick WiFi Performance Compared

June 3, 2020 Family Internet Travel Tech Blog 0

Building on our Best Streaming Sticks For Travel post, where we review the features that make a device good for travel, we tested the same three streaming sticks to determine which ones performed the best under less than ideal WiFi conditions. Just like you might experience when traveling.

While the challenge of poor performing WiFi isn’t unique to travel, it is certainly more common to experience slow and inconsistent WiFi on the go. Unlike your home or office WiFi network, your ability to do something to improve the connection is greatly reduced when traveling or on the go.

The Devices

Just as with our best streaming sticks for travel we decided to focus on the Roku Streaming Stick, Amazon Fire TV Stick and the Google Chromecast. Not only are these the only true candidates for a “travel” streaming devices, they also are three of the most popular.

Amazon Fire TV Stick

As one of our favorite travel devices, the Amazon Fire TV Stick is affordable and has many of the features you’d desire. It’s also a solid choice for home use. It is reliable and can do more than just stream shows. There are even games you can play on it. It doesn’t make a great gaming platform but if you have younger children having the additional ways to entertain them is always a plus.

If you are well integrated into the Amazon eco-system this is also a no brainer. Prime integration is fantastic, as you would expect, and if you are an Alexa user you can pretty do anything you need to without the remote. Though there are a few apps that don’t support the Alexa control as well.

It does have some draw backs such as not every app or service works on it. Vudu and several of the other movie streaming services are good examples. This is generally due to the fact that Amazon or Vudu view each other as competition and choose not support the platform. There are also those who have philosophical differences with Amazon.

Roku Streaming Stick

The Roku Streaming Stick is another device we are big fans of as a travel streaming stick. While it doesn’t have all of the features of the Amazon Fire TV Stick, its pretty close. It’s also a great choice for home use if you need a stick that does double duty.

One big advantage Roku has over the other players is it is neutral. While they recently entered the streaming service game, they are not seen as a threat or competitor in the same light. So almost all platforms support the Roku. This is one of the big reasons why it is our favorite – there isn’t much you can’t do with it.

The disadvantage with the Roku Streaming Stick is it is an afterthought for a lot of services. Apps tend to be behind or out of date, even major platforms like HBO Go/HBO Now often lag behind with updates. Roku has gained popularity recently, so this is less of a problem than it use to be, but it’s still an issue you see.

Google Chromecast

The Google Chromecast is a go to choice as a streaming device because of the cost and simplicity. Those who do everything with their phone usually have well integrated into the Google eco-system. The challenge with the Chromecast is it just works so much differently , and that often isn’t ideal for travel. That is is a real shame, because when paired with a travel router it is actually a great travel device. It’s cheap, you don’t need to keep up with a remote.

However, if you don’t have a travel router its almost painful to use the Chromecast when away from home. It also is less resilient than the other devices we tested. App support is decent, but not impressive compared to the Roku or even the Amazon Fire TV. Many platforms still don’t support Chromecast streaming. And you need to install apps on your own phone for each platform you do want to use.

Other Devices

While the test would have been very interesting on a variety of other devices which are not well rated for travel, we wanted to stick with the core offerings. However, feel free to reach out on social media if you’d like to see us test with other devices in the future.

The Methodology

We wanted to be as consistent as possible. While nobody will confuse this blog post for a full lab test, we did want to ensure we were putting on a fair comparison. We used the same conditions for each device. This includes the same WiFi end point, same service, same show, same resolution, same TV, and same playback and buffer time allowed.

We used Netflix which is generally regarded as a very forgiving streaming service. We are actually currently comparing all of the different streaming services to see how they work on poor performing internet. So we already know Netflix was among the leaders. This meant we were going to have an opportunity to see which device really made a difference in the results.

We picked the same show and resolution. As with the other tests we opted for 1080p rather than 4K. There simply isn’t enough 4K TVs in hotels and Airbnb’s to maker that comparison worth it. Also, 4K takes 4 times the bandwidth and we felt it would be too easy to have problems or downgrades, even if the connection wasn’t that bad.

The chosen show was Netflix’s Space Force. We figured by picking a popular show from a popular service that was practically brand new, anything incorporated into the codec or compression done by Netflix would be applied. We felt an older show or stream left too much to chance that there may be a lower quality stream.

We used the same internet connection and router. This ensured there were no differences in the performance. In terms of how we simulated bad internet we applied the same impacts to the speed and quality across all three for consistency. We’re going to get a bit technical, so if you don’t care about that part skip to the bottom.

For the technies who may be interested, the way we simulated bad internet was to add latency, packet corruption and packet loss. This ultimately created some very harsh conditions. In addition to these three types of unreliability and delay, like you might experience, we further saturated the WiFi network with traffic.

The result was we turned a 30MB WiFi stream with < 10ms and no packet corruption or loss, which would be a dream to find in most hotels, into one in some cases than ran <250KB with over 1 second of delay and 30% packet loss and corruption. Needless to say at that point, none of the streams were still working.

If you are interested in more of the technical details we outline them at the bottom of this blog post.

The Results

Our results had a few surprises. If you are interested in the technical breakdown you can see it following the summary below.

The Roku and the Amazon were nearly tied in how resilient they were. While unscientific, it felt as though the Roku had a less degraded stream under the same conditions as the Amazon. The quality drop was less simply less pronounced. But the difference was minor.

Another area that gets impacted as the network got worse was the user interface. Since each device’s user interface needs some kind of data to function, the usability declined as the WiFi got worse. The Amazon device felt more responsive than the Roku under these conditions.

Of course, we are still largely splitting hairs. Both degraded as the network got worse and worse until they simply no longer could buffer and the user interfaces became nearly useless.

The Google Chromecast was far and away the easiest to break. It took about half of the delays, corruption and data loss to break the Chromecast. Also, the Chromecast didn’t seem to degrade very extensively, which likely explains why it wasn’t as resilient.. Once it started struggling it seem to just stop working.

Conclusion

Bottom line for travel it is clear that the Amazon and the Roku were equal in terms of the resilience for a poor WiFi network. The Amazon stick was still our feature winning since it does the best job of handling the other pieces you need when traveling, such as Captive Portals. This test did nothing to change that recommendation.

The Roku however was equally good and if you are planning to take a travel router or similar device where captive portals are not an issue for the device to handle, we would highly recommend it.

Technical Details

Streaming StickDegradedBarely UseableBroken
Roku Streaming StickLatency: 1s
Packet Loss: 20%
Corruption: 20%
Latency: 1.5s
Packet Loss: 25%
Corruption: 25%
Latency: 2s
Packet Loss: 30%
Corruption: 30%
Amazon Fire TV StickLatency: 1s
Packet Loss: 20%
Corruption: 20%
Latency: 1.5s
Packet Loss: 25%
Corruption: 25%
Latency: 2s
Packet Loss: 30%
Corruption: 30%
Google ChromecastLatency: 500ms
Packet Loss: 10%
Corruption: 10%
Latency: 750ms
Packet Loss: 15%
Corruption: 15%
Latency: 1s
Packet Loss: 20%
Corruption: 20%

As we mention above, in order to simulate a bad network like you might see in a hotel, Airbnb or conference center we added: congestion, latency, packet loss and packet corruption.

However, we didn’t just add it linearly. We used a tool to simulate some variation. So while below we may note a latency of 500ms, it was actually 500ms give or take 50ms, 25% of the time. We also did a similar variation with packet loss and corruption. This helps ensure there was some unpredictability to the network traffic, which is more realistic.