At Epilepsy Research UK, we are determined to identify the best research for funding. We look for innovative research ideas, of the highest scientific merit, that have the greatest potential for making a real difference to patients' lives. We also want value for money, and work completed on time.

In order to do this, we run an annual funding competition using a two-stage process to evaluate proposals. Our Scientific Advisory Committee (SAC), which consists of eight epilepsy researchers from a variety of scientific backgrounds and two lay people, make the final decision about which applications to fund. Using a committee of scientists to evaluate other scientists' work is called peer review. As part of our peer review process we also borrow the expertise of independent specialist researchers around the world.

Our selection process
First, all applicants send in a preliminary application. These are short, only two to three pages. They include a lay description of the planned work and a 500-word technical proposal.

At this stage the SAC are only interested in the quality of the science to be undertaken:

• Is the question being asked worth answering?
• Are the investigations proposed the best way to find an answer to the question?

Each member of the SAC independently assigns a score to each application. At a meeting, all applications and their average scores are discussed. The most promising proposals are shortlisted, and a full application is requested for each. We normally ask for full applications for 2 to 3 times as much research as we're able to fund.

Full applications require much more detail about just how the question posed in the preliminary application will be answered, e.g. how many patients will be tested? Which equipment will be used and why? How will the results be analysed? The application also requires a short CV from each applicant and a detailed costing.

For each of these applications, the SAC selects two independent reviewers. These are scientists working anywhere in the world in the same scientific field as that of the applicant. They must not have worked with the applicant before, but they are likely to be aware of them and their work. The external reviewers are sent the proposal confidentially, and return a critique of it to the SAC. In the 2006-2007 grant round, 34 experts in the UK, USA, Canada, Germany, France, Italy, Australia and New Zealand provided reviews.

At this stage of the process, the SAC have to consider factors other than pure science:

• Do the applicants have the necessary expertise to successfully complete the project?
• Is the proposal value for money?
• Which proposals have the greatest potential for patient benefit?

Taking into account the expert reviews, the SAC again independently assign scores to each application. At their final meeting, they decide which projects will be selected for funding. Copies of the external reviews (anonymised) are sent to all applicants, whether successful or not.

Why use peer review?
The peer review system is almost universally used in medical science. It has flaws but it is the best system available. Epilepsy Research UK's procedures aim to ensure that our process is accountable, balanced, independent and impartial.

Using peer review to select research is a badge of quality for medical research charities. It is a criterion for membership of the
Association of Medical Research Charities. It's also used by the Higher Education Funding Council of England to prioritise which charity-funded research at universities will be eligible for top-up funding.

We aim to make the process as constructive as possible to all applicants by feeding back comments from external reviewers. This means that even if a project is not funded, the applicant should have gained some ideas about how to improve their next proposal.

Problems with peer review
Peer review has obvious flaws. It can be difficult for people in one scientific discipline to appreciate the importance of another, leading to bias. Peer review relies at all stages on scientists' behaving honourably, not borrowing ideas, being overly enthusiastic about friends' projects or overly negative about rivals'. It can be very incestuous in small research communities, where everybody is commenting on each other's work. It's quite a secretive process. It tends to focus on the negative points of studies rather than the positive ones. The quality of review and exact standards being applied can very hugely between reviewers.

However there is no obviously better system. Bibliometrics - a method of analysing researchers' publication records for quantity and quality - is not as widely accepted. Open review, where a proposal is published and anyone can comment, is also not favoured by researchers, who don't want anyone to steal their ideas.

At Epilepsy Research UK, the members of our SAC are carefully selected to be representative of the wide range of scientific disciplines which contribute to epilepsy research, and also of the geographical distribution of research centres in the UK. We also have two lay members on our SAC. This is to counter any possible bias.

We ask members to declare any conflicts of interest they may have about any project. For example if an application has been sent in by a colleague at the same institution, they do not score it and do not contribute to any discussion about it.

We don't require members of our SAC to refrain from applying for funding themselves while they're on the committee, as the UK epilepsy research community is too small to allow this. We don't want to be responsible for taking really good researchers off the market for 3 years while they sit on our committee!

We do feed back reviews to applicants, though anonymously. (If reviewers know their name will be sent back to the applicant, the reviews tend to be less critical. We need the most honest opinions we can get.)

We never use external reviews in isolation, but always "filter" them through the eyes of the SAC. This way we maintain a quality standard as far as we can.

It's in everyone's interests that we fund the best research projects we can. More information about peer review can be found on Sense About Science - a good read, highly recommended.

Epilepsy Research UK, PO Box 3004, London W4 4XT Tel: (+44) (0) 20 8995 4781 • Fax: (+44) (0) 870 838 1069 Registered Charity No. 1100394