A year and a bit of technical research

This is a guest post from Jay Daley, CEO of NZRS. It follows on from my blog post:

Post-Council: Business development, technical research, NZRS and InternetNZ


There's been a bit of confusion in the last couple of weeks about technical research within InternetNZ and the role InternetNZ has asked NZRS to take.

Right from the outset InternetNZ was set up to carry out technical research with this object from the constitution:

"To promote and conduct education and research related to the Internet and inter-networking."

Last year Council decided that it wanted to create a technical research function to add to the grants that it gives for research.  Council asked NZRS to deliver this because we have the people, skills, data and the infrastructure to support a technical research function.  We were already doing some research to support our operation of .nz.

Another benefit that Council saw was the opportunity for the research function to provide InternetNZ with the raw data needed to support the development of policy positions and then make that data available to all comers to validate the conclusions reached. 

In June 2013 we established a small team with Sebastian Castro, who had previously been our DNS specialist, leading it.  For the first six months he actually spent more time on DNSSEC than research with the team only properly getting up and running in December 2013.  At about that time we took on a data specialist within that team.  This new team was then able to embark on the following:

  • Development and publication of research tools
  • Capacity building for public benefit research (creating datasets)
  • Data analysis and results publication.
  • Building relationships with stakeholders to enable the above.

In just over a year the team has initiated a number of projects.  On the .nz side these include detecting botnets from DNS traffic, assessing website popularity through DNS traffic and industry classification of domain names.  On the more general inter-networking side our projects include a couple of NZ Internet Topology Maps and a public RPKI validator.

A large project we are working on is trying to build the first comprehensive dataset of 'last mile' infrastructure coverage that can then be used for further research.  So far we have over 100 datasets from a wide range of providers but only a handful have agreed to that data being made available to third parties on our NZRS data service (https://data.nzrs.net.nz).  We encourage data to be licensed openly and made available for reuse.

Our goal in this project is to persuade the infrastructure providers that their data can be safely published as open data for anyone to use for research and that this approach will benefit them.  This is going to be tricky as some providers are concerned that their data should not be misused or misrepresented, but we remain confident we can achieve this over time, even if it takes a few years. 

The information on these projects is admittedly a bit fragmented and in some cases still being written.  Some of the .nz research remains confidential for policy or privacy reasons.  For now these are the existing sources of information:

  • The topology maps are at bgp.topology.net.nz and ip.topology.net.nz
  • Our GitHub page at https://github.com/NZRS is where we publish all code developed by our research team as open source. (The topology map code will up there soon).

The additional information sources in development are:

  • A new section is being written for our main NZRS web site that provides full details of all of our research projects.
  • We are working on an open data portal to publish as much of our research data as is feasible.

The other area of confusion is how this team is funded.  This technical research function is a straight cost to NZRS and the team is not expected to fund its way by commercialisation of its work.   This enables that team to concentrate without funding pressure and without considering any commercial use from that work.

Quite separately from that team it's part of my job as CE to find opportunities for new services (see Jordan's recent blog post for more details) and as part of that I look to see if I can build a commercial service on top of the work of the research team.  A good example of that is the National Broadband Map that we are developing.  This is built on top of the data that we are collecting, it will be a public benefit service and it has the opportunity for us to recover our costs and fund future development by charging for bulk access.  Of course if we succeed in our goal of this data being open to all then anyone will be able to build their own competing service.

If you have any questions on this then please feel free to contact me or Sebastian.

Jay Daley - jay@nzrs.net.nz
Sebastian Castro - sebastian@nzrs.net.nz