Examination of Witnesses (Questions 140-159)|
WEDNESDAY 24 APRIL 2002
140. I will give you another example. One of
my colleagues was in one country when he said to an elder leader,
"Don't you think this person should be defeated at the next
election?" and they said, "No, no, no we cannot defeat
them." He said, "Why not?" "Because he has
now got to a position where he has stolen millions off us, he
has got a lot of money in his overseas' bank account and now he
has stopped stealing off us. If we elect a new one, the new one
will start stealing off us to build up their overseas bank account
so we must keep him there." It causes a little bit of despondency
until you read the report and I think it is page 39 where they
imported some instruments in for future leaders. How much of your
programme is committed to influencing future leaders in getting
a better level of governance and less corruption in these countries?
(Mr Chakrabarti) I will ask Mr Lowcock to give some
examples from his own experience in East Africa but from what
I have seen on the ground quite a lot actually. Increasingly say
in Kenya, I know that our people we work with on governance issues
spend a lot of time with other members of civil society, not the
Government necessarily, opposition groups, NGOs and so on, churches,
discussing the way forward to get a more pluralistic system in
Kenya. I think that has paid off. I have not been to Kenya for
ten years. Clearly a lot of the policies of Government are not
good enough but undoubtedly it is a freer society with a freer
press than there was ten years ago. One does have some impact,
it may take a long, long time when you do it collectively but
I think you can attribute some of that success to the efforts
that Mr Lowcock and others of his colleagues have had.
(Mr Lowcock) Just on Kenya. We do spend time discussing
these kinds of issues with the people we work with in the countries
where we work. We try also through dialogue with Government to
persuade them, for example, to put more money in to the primary
health care budget or primary education budget and less into something
else. This question of influence is one that we are very alive
141. In the report, if I have read it correctly,
it is not undertaken as a routine part of your country planning
to ensure the level of governance has achieved a standard which
you are comfortable with, is that right?
(Mr Chakrabarti) No, we take governance very, very
seriously indeed. We are always trying to improve it. I do not
want to leave you with the impression that we would only operate
when governance was at Westminster standards.
142. No, no.
(Mr Chakrabarti) Indeed not. The question is are we
making progress in the right direction or are we travelling backwards?
Is that Government committed to good standards, getting there,
putting in anti-corruption bureaux, what have you, getting a more
free press, allowing the opposition to speak freely. Where that
is happening we are much more comfortable operating with that
143. You do an assessment of the quality of
governance, albeit it might not be at the level you would like
but it is moving forward, you have put this assessment of it in
as a routine part of your country planning agreements?
(Mr Chakrabarti) Yes. Each country strategy paper
that we have that our teams draw up will address governance issues
in the way you describe.
144. Excellent. It says in the report it is
not a routine part of it.
(Mr Chakrabarti) We have just put some new guidance
out and it is absolutely crystal clear.
145. I accept this is a difficult situation
because if you are going to put a resource in you have an expected
outcome so you sit down and develop this plan and say "If
we put this resource in, the outcome given in, for instance, poverty
reduction will be X over one or two years." Is that part
of your country planning?
(Mr Chakrabarti) It has not consistently been set
out in that way. I think the NAO team looked at a number of country
strategy papers and saw some had them in and some did not. With
the new guidance what we are doing is insisting the country teams
specify exactly what impact they will have with their efforts.
(Mr Lowcock) One point the report makes is that we
have been much better at doing exactly what you say when we are
looking at investment in particular activities. So in the example
on page 37 it lists in Bangladesh all the things we thought we
were going to get from this £55 million investment. The criticism
in the report, which we accept, is that we have not been very
good at aggregating, to look at the country as a whole, to forecast
what from all our activities the country as a whole will be able
to do. We have been better at doing it when we are looking at
individual projects and, as I say, we accept that criticism.
146. You have answered my next question because
my next question was about the risk assessment. I know how difficult
it can be but do you automatically do a risk assessment on a possible
outcome and the reasons why you might not achieve either level
A, level B or level C?
(Mr Chakrabarti) I think you are absolutely right
at project level we always have done. We have models which our
project teams think through of where the risks are to not achieving
their objective. In fact those models have been passed on to domestic
policy makers in the aftermath of BSE and foot and mouth to try
and think through risk assessment in the UK. Where we have not
been so good is at country level where we have not thought through
the shocks that could divert us from hitting our targets. Again
the new guidance makes it very clear that country managers will
have to do this. Also we have director's statements now where
we are going to insist that directors responsible for a PSA objective
will have to set out the risk of not achieving them.
147. The last question. I realise how difficult
it is to work with other countries which have different strategies.
One example I was looking at, this country went in and built a
lovely hospital and it had the national flag on top, not the flag
of the country the hospital was in but the country which was providing
the hospital, the only difficulty was there were no doctors, there
was no doctors' training programme, there was no nurse training
programme and this building was an empty shell. The equipment
had been supplied by the host country. This was an example of
a lot of money which could have been much, much better spent.
How do we make progress and bring countries on board and in the
European context is it the time to start maybe naming and shaming
some of these countries with their policies which are in part
(Mr Chakrabarti) I think peer pressure is the key
thing, whether it is naming or shaming but being much more open
and explicit about the sort of projects you spoke of, the appalling
waste of money which is clearly not helping the poor in any way.
There is a process at international level which is run by the
OECD which reviews each country's aid programmes to see whether
they are any good or not. That process, that peer review process
does bring to light some of these appalling failures. That requires
them I think, parliamentarians in those countries and elsewhere
but also in public to speak up and put pressure on the governments
of those countries to do better, frankly.
148. Mr Chakrabarti, I wonder if I could start
by asking a very general question. As a student I shared a flat
with a development economist and it was very obvious to me watching
him for many years, from being an under-graduate going through
to doing a post-doc, how his own attitude to development economics
changed. You have at one extreme the Peter Bauer view of the world
which says basically it is a waste of money, I am not advocating
that at all. Could you characterise, briefly, how the debate among
international development economists has changed over the last
20 years or so from how it was to where it is now and where you
stand in that debate? Just two minutes.
(Mr Chakrabarti) Okay. Twenty years ago I would have
characterised the development economics profession as being on
the cusp, I suppose, of change in terms of its thinking. At that
point a lot of people still believed in protectionist economies,
very strong state dominated approaches to development. Then there
was a shift in the 1980s as we saw the debt crisis take hold,
many of these countries unable to pay back their debt, realising
that actually they had to move to perhaps more market orientated
economies. That was very much a theme of the 1980s I think. I
would say in the 1990s and since there has been a recognition
that some of that was perhaps a little bit of an over-reaction.
You do need a viable state involved to run effective services
for the poor in particular so there has been a slight reaction
against that. The key biggest change I have seen in the development
profession in the last ten years, it has been an evolving thing,
has been the recognition that it is really country's strategies
that we must buy in to rather than donor strategies. I think for
many of us, we were designing programmes 20 years ago, 15 years
ago, which were very much focused on what we could deliver as
opposed to what that country maybe required. We are moving much
more to trying to support their policies, their strategies rather
149. That is very helpful. It helps to lead
into my next question which is really about the whole environment
of measurement. You have these Public Service Agreements and International
Development Targets, you have these Millennium Development Goals,
this laudable list of things which are achievable or should be
achievable. Mr Steinberg earlier mentioned the notion on Page
26 in Paragraph 2.13: "...with the majority of measures it
is not possible to determine the extent to which any achievement
is the result of DFID's efforts..." Later on in the Report
at Paragraph 3.39 the NAO says that its analysis of country planning
documents shows that there were no quantified links between development
activities proposed, resources committed, and progress against
PSAs ir IDTs. Then on Page 43, Paragraph 3.43 it says: "Country
Strategy Papers explicitly identify planned development expenditure
over the strategy period. But there is little discussion or analysis
of the relation between resources committed and impacts or outcomes
resulting." Page 32, to go back to the point you were making
about the central importance that has been recognised in the last
few years of the overall country plans, flags this up as an even
more important factor. Paragraph 3.6: "None of the Country
Strategy Papers chart the projected trend in poverty status indicators
over the plan period." It does not sound to me like there
is much of a culture of measurement or performance evaluation
in your Department at all. I do not think anyone doubts that you
are doing good work but it does not sound to methis is
all or nothing languagethat any or many of your staff seem
to take the business of performance evaluation or measurement
at all seriously.
(Mr Chakrabarti) That is a slightly unfair characterisation.
They take it very seriously at project level and we are world
renowned for it at project level. The problem has been to try
and get an aggregation at country level which the NAO Report quite
rightly picks up on.
150. This is the point you made earlier?
(Mr Chakrabarti) Absolutely. There is no doubt about
that. Having read the papers I can see that. The new guidance
which we have just put out tries to change all that. We need to
work it through with country teams. It gets them to try and identify
the outputs and the impacts from what we are doing.
151. Why do you think they do not do it at the
(Mr Chakrabarti) Partly it is a culture change. It
has taken some time to shift from the project level up to the
152. On Page 48, and this is referring to the
country level, there is a quote: "... it's a very interesting
debate which I think is going on internally is how we use our
Evaluation Department, it's very unclear what its role is at the
moment..." "... I've always been very sceptical about
the worth of evaluation in DFID,"a startling statement"not
because it's a bad thing to do, it's not but because as an organisation
we rarely take much notice of it."
(Mr Chakrabarti) I think that is absolutely accurate
in terms of the recent output of our Evaluation Department, which
had been much more focused, I am afraid, on lesson learning too
late almost, so the programme managers who are quoted there are
saying, "It is all very useful but I am not being taught
lessons when I do my job."
153. It says in Paragraph 4.12: "But there
are indications that formal evaluation reports are not read by
country teams. This raises the risk that evaluation findings are
not receiving the attention they need to influence development
practice ..." They are saying, are they, Page 53, that sometimes
you do not get the information until it is of academic interest
rather than when you are implementing policy? They are saying
it is all very well, it is glossy, but it is a complete waste
of time because it is too late?
(Mr Chakrabarti) That is right. That has certainly
been the case with much of it. There have been more generic lessons
which we have learned and which have affected us in real time,
lessons about aid effectiveness more broadly and how projects
should be done over a longer timeframe and so on, but the lessons
around some of the sector work are too late for them to be really
useful on the day to a programme manager. What programme managers
have done, quite rightly, is to use personal networks and retreats
to learn lessons in real time.
154. Are you saying the formal evaluation you
undertake is of so little value that their attitude would beand
I do not want to unfairly characterise it but crudely they might
be saying"We know we are doing good work, why don't
you just let us get on with it and stop bothering us with all
this evaluation and measurement?"
(Mr Chakrabarti) That is not the reaction on the ground.
The reaction is it would be more helpful to have a proper evaluation
done which really did impact in real time as opposed to two or
three years late.
155. How do you go about getting what you call
real time evaluation?
(Mr Chakrabarti) You have to have the evaluation teams
working much more closely with the country teams than you have
had in the past. We have had an unhappy separation of functions
and I would like them to be much closer together to do more work
in terms of monitoring the implementation of programmes than we
have done in the past.
156. How much money is spent on evaluation?
(Mr Lowcock) The budget of the Evaluation Department
157. That was not my question. Maybe they are
the same thing.
(Mr Lowcock) They are not the same thing unfortunately.
That budget is £2 million. There is quite a lot of evaluation
work that is done by people who are running country programmes.
For example, every year for all our major activities we carry
out a review of whether they are achieving their objectives. We
do 500 of those a year and they are done by the people running
the projects. When all our major projects finish the people who
have been responsible for them commission a project completion
report. Those things contribute to the overall evaluation effort
and, of course, that costs us money, having commissioned that
activity. We do not have a number on that unfortunately because
it is done by people who do lots of other things as well. The
bit of the organisation which just does ex post impact
assessment independently costs us £2 million per annum.
158. You mention the case of Bangladesh on Page
37 and talk about improved life expectancy from 57 to 62 and a
list of things for a specific amount of money. They all sound
extremely measurable, including the number of children who get
full-time primary education. It sounds like there could be a very
high degree of specicivity. What is so difficult about taking
that in these various pockets where you have got these specific
projects and adding it all together?
(Mr Chakrabarti) I hope very much that the next generation
strategy papers will do that and that they will have targets of
primary enrolment at the level of the country. In Bangladesh that
should be possible. The Government of Bangladesh should be able
to come up with those targets in its poverty paper and then we
should just adopt these targets if we are happy with this new
paper they are writing.
159. On Page 29 is says that the EU receives
55 per cent of the money that goes from DFID to multi-lateral
institutions, £709 million. I think you said earlierand
correct me if I am wrongthat the result of giving some
money through the EU is that it makes it less easy to achieve
your targets. Is that right?
(Mr Chakrabarti) It is a less effective programme
than many others.