Measuring the Efficiency of Consultation, Participation and Engagement

Around the world citizens like you and me are being consulted about all sorts of things. What kind of digital future do we want? How do we spend our leisure time?  Where should new homes be built? How much local tax are we prepared to pay? What should future provision for the Arts be like? Where should a new waste incinerator be located? Of course it’s not only towns and cities who want information from us; it’s health providers, transport undertakings, central governments and a lot more besides.

One thing is certain: there is a lot of consultation, participation and engagement going on.  And they all seem to use different processes to get our views as citizens, service users or consumers. There are traditional questionnaire surveys, focus groups, e-participation exercises, citizens’ juries, deliberative conferences, world cafés, town hall meetings – the list goes on and on.

Wouldn’t it be good if we knew how efficient these different processes were, and how well they perform their common task of communicating our views as citizens? And what if we could compare the efficiency of all these different processes in different circumstances?  That would be great, wouldn’t it?

I have developed a precise measure of the efficiency of consultation, participation and engagement processes which will enable us to do just that. Drawing on established ideas from ecology and communications theory, the key proposition is that the efficiency of a consultation should be assessed in terms of the information flow it generates.

I have created a new tool for measuring this information flow. It is called the Effective Number of Issues (ENI) and adapts a technique ecologists use to measure the biodiversity of a community of plants or animals. If we think of the information flow as giving rise to a community of issues we can use the ENI to calculate the diversity of this community and thus the efficiency of the process.

To calculate the ENI you need a list of all the different issues that are used in, or emerge from, the consultation process, and a count of the number of times each one appears. You also need the ENI formula, which is ENI = exp(H’), where H´= - ∑px ln(px). In this formula ‘p’ is the relative frequency of each issue and ‘ln’ is the natural logarithm. The calculations are quite straightforward using Excel and there is a step by step guide to the procedure here: https://www.discuto.io/en/info-page/eni-calculation.

Armed with the ENI we can now rate the efficiency of different consultation, participation and engagement processes, and explore what makes one process more efficient than another. As a start, Alfred Taudes, Hannes Leo and I have used the new tool to measure the efficiency of the idea generation stage of the Vienna Digital Agenda e-participation. The ENI for this process (145.9) is the highest ever recorded. Unless, of course, you know of a consultation process that can challenge Vienna for the top position? Do tell us!

About the author :

John May

John May is a community engagement practitioner and social researcher. He is currently drawing on his 40+ years' experience to develop theory to explain and underpin the practice of engagement, consultation and participation with communities and stakeholders. In addition to practical experience of engagement in a wide variety of settings, from hospitals to oil platforms, John has a PhD in marketing (obtained in his fifties) and is the author of a number of papers in peer-reviewed journals.