[Note from Dave: The following is a cross-post from Tobias Denskus's excellent blog, Aidnography. Given my critiques of the Global Journal's Top 100 NGO list, I thought it only fair to host an alternate take. Many thanks to Tobias for sharing this.]
It is probably fair to say that The Global Journal’s Top 100 NGO ranking had a bit of a bumpy start. When it launched the first edition in 2012, Dave Algoso’s critical post and editor Jean-Christophe Nothias’ harsh critique quickly dominated the virtual perception in the aid blogosphere. So when the second edition was published in January 2013, vaguely hinting at “innovation, impact and sustainability” as key new criteria to assess NGOs, I was sceptical and mentally preparing for more critical comments. Luckily, the researcher in me won over the potentially ranting aid blogger and I sent out some messages to a variety of organizations featured in the ranking as well as the editorial team asking for more details on process and methodology. I received open and positive feedback all around and one 20 page methodology paper, a couple of email exchanges and a 25 minute phone conversation with one NGO later, a much more nuanced picture had emerged about the ranking, learning processes and the space for discussions the ranking could facilitate further.
The Global Journal did its homework
The background paper (“Evaluating non-governmental organisations – An overview of The Global Journal’s Top 100 NGOs methodology in 2013″ by researcher Cecilia Cannon, currently based at the Graduate Institute of International and Development Studies in Geneva) basically consists of two parts. The first part is a more conventional and broader political science analysis about conceptualizing NGOs and evaluation of NGO work. In my academic opinion, this part focuses a bit too much on “traditional” organizations and well-known global civil society debates which does not take the range of innovative organizations into consideration that the actual ranking features. Acumen Fund, Wikimedia Foundation or Root Capital do not really fit into traditional NGO categories and it would be interesting to discuss some of the broader philanthropical changes further.
The second part engages more specifically with the ranking and its criteria and sets the tone on the first page:
My main question for this post was how well the methodology corresponds with the aims and objectives of the ranking to “showcase diversity”, “evaluate NGOs comparatively”, “stimulate constructive debate” and “present a range of good NGO practice”. All in all, I believe that the ranking addresses these issues well and with the right mixture of “hard facts” and the necessary openness when addressing contested or hard to define terms like “impact”. I will comment more in detail on the process in the next paragraph, taking NGOs perceptions into consideration as well.
Every theme comes with clearly defined sub-categories and a scale which all contribute to a weighted final score. To me, this looks like good practice for establishing a ranking and Cecilia Cannon also identified some ways to improve them further:
Most importantly at this point is that The Global Journal clearly asked for external advice, is open for discussions and seems willing to fine-tune its methodology further, given that there will never be such a thing as a “perfect ranking”.
The view from the NGOs: “It felt like a grant application.”
I selected a random sample of 11 organizations from across the ranking, controlling for size, “brand recognition” and location.
Until today, I have received 5 responses (2 large international, 1 medium-sized & 2 small NGOs). Again, these were open and friendly exchanges and a picture seems to emerge as to the amount of information that those organizations turned over to The Global Journal:
Balancing the needs for thorough information with the time commitments of organization is certainly an issue (“it was a week worth of work” (small NGO B) and one of the biggest challenges seems to be to think about how to treat large global players and small local organizations fairly – even if that means that they may be treated slightly differently with regards to the requirements for information. But more importantly at this point, there were no questions about the rigor of the process and everybody noticed the changes compared to the first ranking process.
“The ranking really helped us in gaining international legitimacy and more global exposure.”
In my long conversation with one small organization I really felt that this is a key aspect of the exercise. For well-known global organizations the inclusion in the ranking is probably just one small add-on to their communications strategy and the impact on funding will probably be small at best. But smaller, innovative organizations really seem to get something valuable out of the exercise which for them was the equivalent of a “grant application” or “external evaluation” in terms of time and staff commitment. After last year’s inclusion, the organization was invited to an international conference and introduced as an “expert” in their area and further networking and publishing opportunities arose as well. Future rankings and research may show how smaller organizations also benefit in more tangible ways if they continue to be among the “world’s best” NGOs.
Personal learning: Looking behind the headline pays off as many things are indeed more complicated in development
I am really glad that I made the effort to reach out to The Global Journal and some of the organizations featured in the ranking. Judged against its self-proclaimed aims, the ranking is delivering on many accounts. If the process continues to be treated seriously and at the same time as an opportunity for discussion and learning for the aid community, Cecilia Cannon’s conclusion that:
may help to establish the “Top 100 NGO ranking” as a household resource with an approach that many in the aid industry could agree on after its bumpy start.
Tobias Denskus is a post-doctoral development anthropologist interested in peacebuilding, the ritualisation of aid professionalism & the impact of social media on policy-making and reflective writing. Follow him on twitter or find more of his writing at Aidnography.