Making Traffic Public

Where to put all this data?

Posted by Fenwick McKelvey on Wednesday, December 7, 2011 - 18:40

Photograph of Navy Archives Personnel Bess Glenn,1942

In my last post, I discussed the various categories of tests to publicly research the Internet in Canada. These tools beg the questions: where does the data go? Results must collect somewhere if we have any hope to rendering the state of the Internet in Canada. Read more »

What Kinds of Broadband Tests?

Posted by Fenwick McKelvey on Monday, November 28, 2011 - 16:47

I’d like to elaborate on the type of web-based test for the public to measure their broadband connection. We can break these categories down into network performance, network configuration, and what I might call interference. Most testing suites offer a mix of different testing solutions. Read more »

How to evaluate tools that crowdsource Internet research?

Posted by Fenwick McKelvey on Monday, November 21, 2011 - 17:29

Experiments with Light by Ishan Manjrekar

What makes a good public research tool? For the past few posts, I have discussed the need for research into the Internet in Canada. An answer lies in a public research project that provides Canadians the tools to study their home connection and to pool their results to create a national public data set. A number of tools already exist doing precisely this task. While I have mentioned a few, right now I want to discuss some of the values behind any evaluation of a tool. If I'm committed to doing public research, then what should I be looking for in a tool? In the next posts, I will use these evaluation criteria to discuss a few options. For now, I offer the key values and, as always, I appreciate your feedback on the criteria.

Any tool should provide neutral and viable data about the state of the Internet.

This should be a given since any 'good' piece of software means a working piece of software. Beyond just running, a broadband test has three key components: what does the tool measure, and how does it conduct this measurement, and how does a tool store results? These components respectively concern scope, method, and storage.

To question scope, we must ask what aspects of Internet usage does a tool seek to measure. Is it testing upload speed, or jitter and latency? Maybe both. Different tools measure broadband in different ways, usually in a combination of different ways, so evaluating any suite of broadband testing must index what it actually measures, and, in comparison, what it misses that other tools provide.
Scope matters since not all answers have equal value. A speed test, for example, appeals to consumers who learn whether they get what they pay for. Evaluating scope must then understand what a tool tests and what aspects of the Internet would be revealed by collected data.
Even a question as simple as 'how fast is your download speed' can be answered in multiple ways. Steve Bauer, David Clark, and William Lehr in their report Understanding Broadband Speed Measurements discuss how Ookla's SpeedTest aggregates a test into slices, discards the fastest 10% and slowest 30% slices, and then creates an average number, where M-Lab's NDT “the sender tries to send as much data as possible during the ten second test period” (2010, p. 31]. These differences have a real impact on the usability of a tool. The NDT test, for example, might provide different results depending of a home user's desktop configuration of the receiver window (rwnd) (2010, p. 33).
While the testing algorithm is important, the location of the test is just as important. Where does a test take place? Does it occur from the home to a dedicated site or a spare computer in a lab? M-Lab has strict requirements for testing hubs leading to there being only a few locations where SpeedTest has broader requirements leading to more testing locations. Conventional wisdom holds that the less 'hops' – the less networks between a home user and a testing server – the more accurate the test. Thus, evaluating method implies both the way to conduct a test and the infrastructure behind a test.

Finally, we must also remember that a SpeedTest lasts only a few seconds. Not only do we risk forgetting the results, but one test only studies the network for a short moment. Any tool, then, most prolong the study by recording and aggregating the data. Each test might be understood a pixel in a picture of the Internet. Thus, evaluating a tool must consider how data is recorded and stored. Does the data allow for simple generalizations and easy access, or will it become a chore to access the results?

A public broadband testing tool must be open and transparent.

Open source tools and open data, in my mind, are central to any public research project. Yochai Benkler, in his popular and seminal book The Wealth of Networks, suggests modularity allows the greatest number of people to contribute to a project because they can choose the contribution that best suits their needs (2006, pp. 101-104). Studying the Internet requires coders to program, statisticians to analyze results, writers to publicize the information, and artists to represent complex data.

Evaluating tools with a commitment to openness must consider where the code is open source, or at least, are its methods transparent? Openness certainly has degrees, especially when considering releasing the data. Are results available in raw data for downloading, or are they aggregated and accessible through an online archive. Who has access to data? An open license might simply be in the public domain, or a more restricted license only for non-commercial usage?

Finally, any project requires support from organizations developing the tool to ensure public participation in these areas. While organizations like OpenMedia must play a vital role in keeping the public involved and motivated, they need the support of developers to ensure the project supports public engagement. As an interdisciplinary researcher, this aspect of the research excites me because it requires collaboration between the social sciences and the computer sciences to ensure tests answer technical questions about broadband usage, as well as social questions about how to conduct public research.

Any solution must be able to adapt to a changing Internet by allowing for new tests that answer new questions.

The Internet is continually changing, and any tool needs to have a strategy to adapt to change. For example, Glasnost, a traffic-shaping detection tool that has been making headlines, designed its code to allow for more tested to be added, and even to aid in developing these tests. As new applications rise in popularity, perhaps P2P video streaming, then Glasnost can develop and deploy new tests without a complete re-write. Further, if people do contribute, and want to develop new tests, how does a testing suite accommodate their contributions. The challenge, in short, is to future proof the test.

In Conclusion

Viability, openness, and adaptability are my three major areas of my evaluation criteria. A strong score in these areas would ensure that a tool studies pertinent aspects of the Internet in open ways, and that the tool would release usable data to the public and adapt to changing research questions. I am interested in hearing your feedback about this list. Do these values represent the prerequisites of good public research? What other values should be on this list? Read more »

Internet Usage & Crowdsourcing

Posted by Fenwick McKelvey on Friday, October 28, 2011 - 22:56

In my last post, I mentioned crowdsourcing Internet usage. Usage differs from related questions of Internet access. How many Canadians have access to the internet? How much does access cost and is it affordable? Does internet service provision in Canada exclude, for example, First Nations, the elderly, the poor, or rural and remote communities? The answers reveal many of the digital divides that exist in our society. Internet usage, on the other hand, asks what we do with access to the internet. Answers reveal another set of digital divides – technical barriers, bandwidth issues, traffic shaping, and access filtering. Also, Internet usage data could explain popular online activities. What are the most popular sites or ways of communicating online? Since the internet is still developing as a medium, usage describes how we have come to use the web and guide its future direction.

Talking about crowd sourcing assumes that how we use and how our usage has limits is important to understanding the Internet. Tests, such as Speed Tests, Traffic Shaping Detection, or even the number of provider in an area, all paint a better picture of the limitations and opportunities that effect us whenever we use the Net. The Netalyzr tool, for example, conducts over 38 tests to study the connection from their servers to the home computer that started the test. Since these tools begin with users, they offer an understanding of the Internet that puts people-first. Any data collected also adds to some of the great surveys of the Canadian public and how they use the Internet from the Canadian Internet Project and Statistics Canada.

Without beginning with the public, and our usage, then knowledge about the Internet will always be limited. I feel crowdsourcing further adds to survey research because it begins first and foremost with the public participating in the research project. The exciting part of crowdsourcing is that it recognizes that the home users are the best people to conduct research about the Internet since they are the one's most affected by its conditions. Instead of assuming Internet users must just deal with their Internet connections, crowdsourced tool engages the public in understand how the Internet works, and through that process hopefully we can make our Internet even better. Read more »

Crowdsourcing Internet Research? An Introduction

Posted by Fenwick McKelvey on Thursday, October 20, 2011 - 21:12

Your Internet connection may act mysterious when its slowdowns or crashes, but its just as mysterious to researchers and policy makers. We often hear about congestion, cost, and speed without having much evidence. I believe the solution to this lack of evidence rests in developing tools to publicly study our Internet traffic. Such tools, I believe, would make traffic public.

What do I mean by tools to publicly research the Internet? Well, I am suggesting crowdsourcing the study of the Internet in Canada. We often think of crowdsourcing when citizens report the news, but crowdsourcing also applies to figuring out the Internet. Home users can study their traffic, volunteer the data, and help analyze their internet connection. A number of projects already help citizens study the Internet, so this blog will be talking about some of the possible tools to crowdsource the study of the Internet in Canada. More data would help better understand broadband speeds nationwide, the usage of traffic management practices, or the roll out of IPv6.

Over the next month, I will be posting a few ideas and suggestions on how to implement public testing in Canada. I look forward to hearing your opinions. Public feedback is why I am blogging first and foremost. I hope this becomes a place to discuss options and ideas to crowdsourcing Internet research in Canada.

Until the next post, I encourage you to check visit a few others on the subject matter:

Here is PDF copy of a chapter on this subject for a new book by the Canadian Centre for Policy Alternatives. The book entitled the The Internet Tree: The State of Telecom Policy in Canada 3.0 edited by Marita Moll and Leslie Regan Shade. Its a great book if you want to learn more about Internet policy in Canada.
Christopher Parsons, active contributor to OpenMedia and PhD candidate at the University of Victoria, has an excellent recent article on ISP audits in Canada. Chris Parson’s blog is an excellent resource for Internet policy and political matters in general.
Steve Bauer at MIT and a few others have been discussing the various methods of broadband testing tools.
Finally, the IXMaps project is a great example of another public research project. It hopes to locate points of Internet surveillance by the US National Security Association and when our Internet traffic passes through these surveillance points.

Read more »