Skip to content

Frequently asked questions

What happened to the old Guidebook?

The original EIF Guidebook, published in 2014, provided information about 50 programmes available in the UK. Like the current Guidebook, the old Guidebook provided evidence ratings to indicate programmes with stronger or weaker evidence of positive outcomes for children. Unlike the current Guidebook, these evidence ratings were based on a summary of assessments by 15 other international evidence clearinghouses.

Since 2016, EIF has conducted its own assessments of evidence. The current Guidebook was launched in 2017, and provides strength-of-evidence information based solely on EIF assessments.

  • Any programmes that were included in the old Guidebook and have been reassessed by EIF are included in the new Guidebook, with updated ratings and other information (if they meet the requirements for the level 2 threshold).
  • Some programmes included in the old Guidebook will be reassessed by EIF. For information about programmes that are currently going through the assessment process, see: Work in progress
  • Any programmes in the old Guidebook which have not been reassessed by EIF are not included. Information about these programmes is available on request.
Read more Read less

Why do the evidence ratings start at 2?

The EIF Guidebook includes programmes that have at least preliminary evidence of achieving positive outcomes for children – which is equivalent to level 2 on our standards of evidence. Programmes that are ‘not level 2’ (or rated NL2) are not included in the Guidebook.

Previous versions of the EIF evidence standards included lower ratings of 0 and 1, where 1 indicated that a programme has a logic model and testable features of the kind that might be used to evaluate its impact at some point in the future. In our view, this was not a helpful distinction in terms of helping commissioners and others to understand what the evidence was saying about likely or possible impact. To reflect this, we have adapted our evidence standards to exclude specific conclusions about programmes that do not meet the level 2 threshold.

In its current form, the Guidebook provides information based on evidence of impact gathered through a formal evaluation process, and does not make judgments about the adequacy or correctness of logic models and programme designs. Programmes that do not have evidence are rated NL2 (‘not level 2’), rather an 0 or 1.

For a list of NL2 programmes, see: Other programmes

Unlike the cost rating, there is no level 5.

Read more Read less

Should I only commission programmes with evidence at level 4?

There are several advantages to implementing interventions with established evidence.

  • Level 4 or 4+ programmes have evidence showing consistent benefits in multiple places with multiple populations. But this is no guarantee that an intervention will provide these benefits again in a new setting or authority, particularly if it is not delivered as intended. Established evidence does, however, increase the likelihood that it will.
  • Interventions with established evidence also tend to be more developed, meaning that the original providers have worked through various issues that could hamper effective implementation. In particular, interventions with established evidence are more likely to be able to provide assistance to set up and ‘install’ the intervention to maximise its effectiveness, for example through recruitment guidance or training materials.

However, you should never commission an intervention on the basis of its evidence rating only. Using evidence is one of three necessary parts of the good commissioning, alongside effective implementation and careful consideration of costs and benefits. More information about effective commissioning is available on our website.

Read more Read less

Should I decommission programmes with evidence at levels 2 or 3?

It would be unwise to automatically decommission a programme on the basis of disappointing evaluation findings. Demonstrating impact is a journey: many well-evidenced programmes have had evaluation setbacks in the past, and have used the lessons to strengthen their programme model.

Commissioners have a responsibility to encourage and support ongoing evaluation of early intervention activities. This is especially true for programmes rated below level 4: all may be suited to particular local needs and circumstances, and lower rated programmes can be an excellent source of innovation and experimentation, but it is important that commissioners make a commitment to monitor, test and adapt. More information about effective commissioning is available on our website.

For more information on what the evidence ratings mean for commissioning, see: How to read the Guidebook

Read more Read less

Are all Guidebook programmes available in the UK?

No. The Guidebook includes programmes that have not been implemented in the UK, so far as we’re aware. It also includes programmes that are available in the UK but for which their best evidence is based on evaluations conducted in other countries, often the US or Australia. However, all of the interventions included in the Guidebook may be feasibly implemented in the UK and have evidence that is relevant in the UK context.

To limit your search to programmes that have been implemented in the UK, tick the ‘Show only…’ option in the filter options.

Read more Read less

I can’t find the programme I’m looking for – does that mean it doesn’t work?

The Guidebook is not an exhaustive list of all programmes available in the UK. If you can’t find a particular programme in the Guidebook:

  • The programme may have been assessed by EIF and found not to meet the threshold for a level 2 rating. These programmes are listed on: Other programmes
  • The programme may not have been assessed by EIF. For information about programmes that are currently going through the assessment process: Work in progress

Other programmes may be included in future assessment reviews. For more information, see: Getting your programme assessed

Read more Read less

Why don’t you provide information about effect sizes?

Effect size is a useful way of understanding how big a change a programme has been shown to achieve. It is not the same as strength of evidence, which helps us to understand how confident we can be that the change is caused by the programme.

However, individual effect sizes are not easily comparable, in part because of the diversity and multiplicity of outcomes and in part because of the diversity of methods used. Where possible, the Guidebook provides information about effect sizes on a study-by-study basis only. For many programmes, effect size information is not available.

Read more Read less

Why is your cost rating nothing like the service price that I’m being charged?

Our cost rating is not the same as the market price of an intervention, which will be negotiated and agreed commercially between providers and commissioners. The cost rating is an assessment by EIF of the relative input costs of early intervention programmes, such as practitioners’ and supervisors’ time, qualifications or training requirements. This assessment is based on information that programme providers have supplied about the components and requirements of their programme, and not on any price list or advertisement.

Read more Read less

Do you only assess programmes?

The Guidebook provides information only about programmes – manualised, repeatable activities and models of early intervention delivery. We know that early intervention works in other ways too – from a teacher, police officer or other frontline professional spotting the first signs of risk in a child or family, through to wholesale systemic changes across the many agencies and services at work in a local area. Information and guidance about these other forms of early intervention can be found on our website.

Read more Read less

Do you evaluate programmes?

EIF does not conduct evaluations. Evaluations may be carried out by a range of organisations, including universities, professional research companies and major national auditors. Where possible, the Guidebook provides links to key evaluation studies, which include details about who conducted the evaluations and their methods.

Read more Read less

How do you select programmes to be included in the Guidebook?

EIF conducts programme assessments as part of our ‘What works’ reviews of evidence related to early intervention in specific issues, age groups or other populations. We do not conduct assessments on a one-off or on-request basis, outside of these review periods.

For more information, see: Getting your programme assessed

Read more Read less

How do I get my programme included in the Guidebook?

For new, upcoming evidence reviews, EIF issues a call for expressions of interest for programmes that fit within the scope of the review and wish to be considered for assessment. These opportunities are notified via our website, Twitter and newsletter.

We do not conduct assessments on a one-off or on-request basis, outside of our ‘What works’ reviews.

For more information, see: Getting your programme assessed

Read more Read less

How often will you be adding programmes to the Guidebook?

Programmes that are assessed as part of current or future reviews, and which are found to have at least preliminary evidence of achieving positive outcomes for children, will be added to the Guidebook at the conclusion of each review, as and when individual programme reports are complete.

For more information on current assessment reviews, see: Work in progress

Read more Read less

Do you assist local authorities and commissioners to implement or evaluate programmes?

EIF is not generally able to provide bespoke guidance to local areas about using evidence, implementing programmes or evaluating the effect of decisions and changes. We do work with local areas who commission EIF to help them to understand the research and evidence assessments that we have previously conducted and published.

We also work more closely with members of our Places Network to support early intervention in their areas. 

In addition to the guidance resources available on our website, we also run a series of seminars and masterclasses around the country: keep an eye on our website, Twitter and newsletter for more information.

Read more Read less

Submit a new FAQ

If you do not find the answer you are looking for, please submit your question using the form below. If it is asked frequently, we will add it to this page.

Submit a new Guidebook FAQ

Published March 2017   |   Last updated April 2017