A new toolkit for digital inclusion initiatives

In early 2020, as our team of 5 million hunkered down ready to take on the lockdown, we were beginning a journey with a cohort of five changemakers determined to progress digital inclusion in Aotearoa.

Previously, we shared our desire and goals around supporting community groups to learn how to evaluate their work. 

This stemmed from the observation that the information around digital inclusion was patchy and that this was hampering not only efforts by the government and others to invest in this kaupapa but also efforts by community groups to do the mahi on the ground. 

A strange sort of silver lining from the national level 4 lockdown in 2020, was that we no longer needed the data to argue the importance of digital inclusion — most people are now on board with the fact that having access, skills, trust, and motivation to harness the power of the Internet, is a good thing.

Nevertheless, having better data around what works well, is really useful for all those people working towards digital inclusion. By overlaying our community funding activity with our goal of advancing digital inclusion evaluation in Aotearoa New Zealand, we wanted to help fill this evidence gap.

Over the course of three grant rounds, charities and community groups with their sights set on helping people across the motu were funded to implement 14 different initiatives. At the same time, they received support from Standard of Proof to learn how they could put ideas of evaluation into practice. Their various initiatives worked to support people with disabilities, youth, seniors, Māori, and Pasifika.

What we learned along the way

For the team here at InternetNZ, we learned a lot along the way. For one thing, talking about evaluation in a way that resonates with people is hard. Plus, there are so many different ways to do it and different approaches will suit some kinds of initiatives more than others. Often people lean heavily on surveys to do the heavy lifting when it comes to evaluation, but they are not the only tool available and they are not always the most effective tool to use. 

Michelle Whiting, Chairperson of Te Mana O Kupe said that due to the nature of their project, using a narrative approach was a better fit rather than collecting quantitative data.

Another thing we found is that doing this well can take way more time and effort than you might expect. Lots of the ideas groups had about what data they would like to collect, didn’t match up with the reality of what they could collect given the resources available.

We learned that sitting down with your full crew to understand the value of evaluation at the beginning, really matters. If you only try to measure success at the end, you may realise you are missing some important data sets, or that the information you’ve spent all that time collecting, was not actually meaningful. By working through these steps before kicking off their projects, many of our grantees group were able to develop a shared vision for the change they were embarking on. They developed measurements to track the progress that was appropriate to the problem they were trying to solve.

Laurence Zwimpfer, Operations Director and Trustee of the Digital Inclusion Alliance Aotearoa told us that the pandemic delays on the ‘doing’ forced them to spend more time on the ‘thinking’ elements of the evaluation and they felt better prepared when implementation was possible. 

The evaluation toolkit

Now, as we head towards another grant round focused on digital inclusion, we are really excited to share what we’re calling the ‘evaluation toolkit.’

The support and resources provided to previous grantees have allowed us, with the amazing help of Standard of Proof and our awesome Evaluation Advisory Group, to design, test, and iterate this toolkit. After all the hard mahi of these different grantees and all the people they have helped, we hope this set of tools can be useful to others working in this space. 

The toolkit has some templates and exemplars that you can work through with your team as you set out on a project, so you can feel confident that your energy and resources will go to the right things on your to-do list. It's all geared towards helping you create positive change. Planning at the start helps you work more efficiently along the way, and when things go awry, you’ll be able to adjust quickly and easily. Plus, you’ll have all that great data at the end, to demonstrate to other funders the impact you’ve had so that you can get further support.

Our friends at Standard of Proof have provided a report detailing this journey we’ve been on with our grantees. It highlights the challenges, key findings and recommendations moving forward, both for us and for other funders.

InternetNZ

Each year, InternetNZ uses profits from the sale of .nz domain names to give out community funding. It provides funding for community-led initiatives that extend the availability, use, and benefit of the Internet in Aotearoa.