CGO's October 14th Reply To The CRTC

The Canadian Gamers Organization (CGO) received a letter frrom the CRTC requesting more info on games that are throttled by Rogers.  The original date for reply was set on October 10th, which was thanksgiving day.  CGO asked the commission to extension the deadline for a reply and asked for justification and reasoning behind the CRTC's request for more info.  The extension was granted (October 14th, 2011) and the CRTC ignored the request to explain further why they were requesting more information from CGO.  The below is CGO's reply to the commission sent in October 14th, 2011:

Re: Your file #545613

Ms Baldassi,

The Canadian Gamers Organization (CGO) wishes to submit this reply to your letter dated September 30th, 2011.

In Rogers letter dated September 2, 2011 Rogers stated:

1.    In the case of World of Warcraft, we found, in the tests we conducted that no consumer impact is identified unless P2P file sharing applications are running at the same time, thereby driving traffic to over 80/KBPS.  World of Warcraft runs well below 80 kbps.  It therefore would not otherwise be impacted by our ITMP which is activated for traffic at 80 kbps and above.  Similarly, in respect of other games, gaming systems or not-related gaming applications that have been reported to us by our customers, we have tested them and found that there is no impact on consumers when P2P applications are also not running.  Like World of Warcraft, other games also run well below 80/kbps and therefore would only be impacted if they were misclassified and other P2P applications were running at the same time.

From the statement above, Rogers has stopped short in explaining to the commission why any games are captured by its ITMP whatsoever. In the proceeding that led to the development of the ITMP framework, Rogers testified before the Commission that its ITMPs are carefully calibrated to ONLY capture those peer to peer file-sharing applications they were developed to capture:

4962   We try to be very conservative in the way we use the heuristic sort of approach, so that we don't capture, in an unintended way, packets that may not be of that nature.

4963   So we definitely -- what we always do is, we test all of those classifiers that are analyzed in the traffic heuristically, and ensure that only those packets which are definitely peer-to-peer are actually classified as peer-to-peer, and then routed into the separate allocation.  

Note that none of the other traffic that our customers send upstream is rate shaped. The only impact is on users of other networks that source content using P2P from our customer’s computers. These users still get their content – only not quite as quickly as they might without rate shaping.

Rogers’ ITMP operates on the basis of flow management. Traffic flows identified by its deep packet inspection equipment as targeted applications are diverted into distinct network channels with “separate allocations” of bandwidth (specifically, 80 kbps allocated bandwidth). Traffic generated by non-targeted applications should never be identified by Rogers’ deep packet inspection equipment and should never be diverted into this rate-limited channel.

Yet it is apparent from Rogers’ letter of September 2 that other applications and games are being misclassified by its ITMP and pushed into this rate limiting channel. At a basic level, this means that Rogers’ ITMPs are not calibrated as they should be and as represented in Rogers’ submissions to Telecom Public Notice CRTC 2008-19, cited above. This is problematic regardless of direct customer impact.

But customer impact exists. Rogers states openly in its September 2 letter that World of Warcraft and unspecified “other games” are diverted to this rate-limited channel by its DPI. This would be consistent with reports the CGO has received recently from our members. It means anything running above 80/kbps with file sharing applications running is being actively impacted and misclassified by Rogers ITMP.

Even if games or other applications ran below the 80/kbps threshold individually; if they were running simultaneously that combines upload traffic to exceed 80/kbps with P2P applications running, they would also be impacted by Rogers ITMP from what Rogers has disclosed to the commission.  Rogers’ ITMP is activated “for traffic at 80 kbps and above”, and through Rogers disclosure is only active when P2P file sharing applications are open. As the commission well knows, Canadian consumers can have several gaming systems, wireless devices, and PCs hooked up simultaneously in any given home network set up.

This aggregate impact on gaming is in direct conflict to Rogers’ own stated objective for its ITMP, which is to slow down “[h]igh-volume, low time-sensitive traffic…to ensure all customers have a high level of service.   These benefits are to be experienced by P2P and non-P2P users alike, as noted by Rogers, again, in its comments to TPN CRTC 2008-19:

Note that there is also a benefit to the P2P user who will experience better performance of non-P2P applications while they are supporting P2P uploads. Heavy P2P users need to set up complex queuing rules in their home routers in order to prevent their P2P uploads from crowding out their web surfing or games. Our network management practices alleviate this burden for the P2P user, so even they benefit.

Yet customers who wish to use P2P file-sharing applications and play online games (a highly time-sensitive activity) at the same time appear not to warrant this benefit.

Further, CGO wishes to submit that not all games run below 80/kbps.  World of Warcraft is an older game, and since development and release of this game, newer games are increasingly becoming more sophisticated requiring much more bandwidth.   World of Warcraft recently updated to provide a setting to users which would make the game server respond faster to user actions, but at the cost of the user's upstream bandwidth increasing.  World of Warcraft also has a built-in VOIP for in-game team chat on top of regular game data transmitted to developer’s servers and to other peers on the network, however the bulk of World of Warcraft users prefer to use alternate VOIP solutions (due to better sound quality/voices being much clearer), which can be additionally taxing on the upstream bandwidth.

The developers of the game Homefront,  for example, state it requires users to have consistent upload traffic between 100 - 120/kbps in order for the game to run smoothly with a direct connection to their servers, stating that most lower end DSL connections in the US support upload of 768/kbps.  The interview with THQ developers on the dedicated severs is here.  Time index 10:50 THQ explains what we’ve stated above. This is above the threshold where Rogers ITMP kicks in with P2P applications running.

In our original complaint we stated:

We believe the affects of ITMP are a lot more widespread than with the PC version of World of Warcraft, and extend to other games, gaming systems, and non related gaming applications, and could impact internet services even when P2P applications are not running, due to the ITMP bug actively misclassifying non-P2P applications.

In addition to the September 2 letter cited above, Rogers has also provided information to the commission with disclosure to ongoing misclassification of non related gaming applications.
In its letter dated September 27th, 2011, Rogers stated:

B) In terms of further disclosure, in our ITMP policy on our website, we disclosed that our customers may experience problems if they run P2P applications at the same time as they use other applications including games.

In Rogers “updated disclaimer” located: http://www.rogers.com/web/content/network_management states:

“3. Are there other applications that could be impacted by Rogers traffic management measures?

If your Internet connection cannot attain full speed while using an application, such as encrypted FTP, please ensure that you are using the standard port assigned for the application/protocol in question (as per the IANA: http://www.iana.org/assignments/port-numbers).

If you cannot find the application/protocol listed in the IANA's website or you're not currently using the assigned port listed, it is possible that the application/protocol being used may be impacted by traffic management if your Internet connection is using a P2P file sharing application at the same time.

To resolve this issue, please close the affected application and check that all P2P file sharing applications are not running on the Internet connection. When you turn the affected application back on, ensure you are using the standard port assigned and there are no P2P file sharing applications running.

Note: Allow up to 10 minutes after terminating the P2P applications before you restart the affected application to ensure the application is not affected by traffic management. Verify that all computers connected to the Internet follow the trouble shooting process above.

Through this disclosure, Rogers suggests that it has reports confirming that encrypted FTP communications and other non-gaming related applications are being actively misclassified when these applications are not using standard protocols and P2P filesharing applications are running at the same time.

In the commissions’ letter dated September 16th, the commission stated:

In Telecom Regulatory Policy 2009-657, the Commission stated that when noticeable service degradation occurs to time-sensitive traffic, it amounts to controlling the content and influencing the meaning and purpose of the telecommunications in question. The Commission found that use of an ITMP resulting in the noticeable degradation of time-sensitive Internet traffic would require prior Commission approval under section 36 of the Telecommunications Act.

Further, in our letter to the Commission of September 30, we noted that even under the Commission’s ITMP framework for assessing unjust discrimination under section 27(2), “the Commission considers that establishing that the ITMP is carefully designed and narrowly tailored is important in an evaluation of whether or not the discrimination or preference is unjust or undue.”   Rogers has openly stated that its ITMPs capture time-sensitive games in addition to the peer-to-peer file-sharing applications that are its intended target. This exhibits neither ‘careful design’ nor ‘narrow tailoring’ and hence amounts to unjust discrimination against gaming traffic such as World of Warcraft. The overbreadth of the ITMP is, specifically, that it diminishes users’ ability to play World of Warcraft and certain unspecified other games’ while also using their P2P applications. Rogers has no legitimate reason to impose this penalty on its users.

This policy needs to be enforced to be effective.

Requests put to the Commission in a letter from the CGO dated September 30th, 2011 for further justification and reasoning based on the new ITMP complaints procedure were not answered.

The new guidelines state: http://www.crtc.gc.ca/eng/info_sht/t1043.htm

How to make an Internet performance complaint

Before you complain to the CRTC about an Internet traffic management practice, you should     first contact your Internet service provider to see if it can resolve the issue.

If your service provider doesn’t address your complaint to your satisfaction, and you believe that your service provider’s traffic management practices are not compliant with the CRTC’s policies, you can complain to the CRTC. Before doing this, make sure that you know your rights.

In the commission letter dated September 30th, 2011 your staff requested that the CGO provide the following:

In addition, in order to proceed with the analysis of your claim that the Rogers’ ITMPs extend to other games, gaming systems, and non-related gaming applications, we require more specific information about the nature of the problems you and/or other you represent have experienced than you provided in your complaint dated 22 August 2011.

Rogers’ admission that ‘other games’ are caught by its rate-limiting practices demonstrates that its ITMP discriminates against an unspecified set of games. According to the Commission’s ITMP framework, the onus falls to Rogers now to justify its ITMP by providing the Commission with further information and testing on what other games are captured, under what conditions, and why.  Rogers has already agreed to do so, and has also stated to the Commission it is doing testing on an ongoing basis.  We would however ask the commission to further direct Rogers as to the conditions of such tests based on Rogers current disclosures.

Regardless, as there is already direct evidence on the record from Rogers that its existing ITMP captures World of Warcraft and “other games” under certain circumstances, this should be sufficient to assess Rogers’ solution to its overbroad rate-limiting practices. Whitelisting one game – World of Warcraft – as Rogers has reportedly done - will do little to alleviate the impact of Rogers’ ITMP on other games. Rogers has admitted unspecified ‘other games’ are captured and rate-limited whenever their use coincides with P2P usage in aggregate excess of 80 Kbps. Whitelisting is not an effective solution, but rather comparable to a game of ‘whack-a-mole’. While CGO is aware of testing indicative of problems with many other games, including: Rift, Heroes of Newarth, League of Legends, and Minecraft, it is not well placed to identify each and every gaming or other application that Rogers’ overbroad ITMP captures, currently and in the future. Other consumers are even less well-placed to collect such evidence, as it is challenging to distinguish the effects of Rogers ITMP from normal slowdowns in its best-efforts network. Such assessments can be accomplished most easily and effectively by examining Rogers’ DPI, not from the ends of the network. 

Further, CGO is aware of consumer complaints relating to at least six other games that have been made to Rogers. There has been no indication that Rogers has added these games to its whitelist on its own accord. It is neither feasible, nor does it seem desirable to institute a proceeding for each and every game potentially captured by Rogers ITMP. Instead, a more effective solution would be to cause Rogers to develop an ITMP that does not impact on time sensitive applications at all.

The objective of our original complaint was to bring light on policy inconsistencies and enforcement failures to the Commission directly, and also to get Rogers to further disclose the ongoing affects Rogers ITMP is having on consumers connections to the commission.  In a sense this complaint was a consumer audit of the CRTC’s current complaints procedures. A successful audit, that has uncovered more policy inconsistencies, and lack of enforcement now further documented. 

This will be our final communication and reply on this current file.

Warm Regards,
Jason Koblovsky + Teresa Murphy (Founders of the Canadian Gamers Organization)