Introduction

Each year, the federal government gives out hundreds of billions of dollars through about 1,500 grant programs. To help agencies decide which groups will receive funds, how much they will get, and what they can do with the money, the government asks citizens to participate on expert grant review panels. It is an honor to be asked to participate in one of these panels. It takes just a few days of your time (for which you will be compensated), but it affords the opportunity to have a tremendous impact on a wide variety of federal programs.

This guidebook is designed to help teach you all about the federal grant review process so that you can help the government make those funding decisions. You will learn how the process works, what all the terms mean, tips on how to be an effective reviewer, and, finally, how you can be considered to be a grant reviewer.

First, a few fast facts about federal grants:

• The number of grant programs changes slightly each year. Some new programs are created each year, while some older programs might end. The average growth over the past five years has been four grants per year.
• 889 of all grants are project grants. These provide funds for specific projects and for specific time periods. They also are the ones that most often require the use of grant review panels.
• 173 are formula grants. These are funds going to states or their subdivisions in accordance with distribution formulas that have been prescribed by law or administrative regulation. The activities they fund are generally of a continuing nature and are not confined to a specific project.
• The use of outside experts in the peer review process is about 65 years old.
• Once awarded, individual grants can be "continued" for three-five years, often in increasing amounts each year.

Why You Want to be a Grant Reviewer

The opportunity to serve as a reviewer will give you tremendous insight into each federal grant program. You will learn more about the activities conducted under that program, who runs it, who gets the money, and how much they receive, as well as what kinds of projects are conducted. It is also your opportunity to play a very significant role in a process whereby the government decides specifically how to spend billions of taxpayer dollars. This is probably one of the best opportunities you will ever have to make sure those dollars are spent wisely and effectively.

For those reviewers whose organizations themselves are seeking or planning to seek federal or even private funds, it will be a useful way to gain insight into the review process and will help make you better prepared to develop successful grant applications for other programs. You will learn the do's and don'ts of quality grant writing as well. All of this practical knowledge will make you a greater asset to your organization.

There is also another important reason why you should participate in the federal grant review process: Your government needs you! It needs your insight, your wisdom, and the benefit of your experience in the field. Think of it as being like jury duty: It's your opportunity to serve.

Who Can Be a Grant Reviewer?

The following general qualifications are desired in grant reviewers no matter which department is asking you to conduct the review. Some grants might also call for individuals who have expertise in a particular area, depending on the specific grant. Generally desired qualifications include the ability to:

• Evaluate and apply criteria related to program requirements.
• Read and analyze applications (e.g., for strengths and weaknesses).
• Write clearly, accurately, concisely, and effectively.
• Communicate effectively.
• Contribute to an effective group process by being cooperative, constructive, and flexible.
• Maintain strict confidentiality.

What is Involved in the Grant Review Process?

• Reviews usually last from three-five days.
• They are often held at a hotel in Washington, D.C., or nearby.
• The grant review "season" is usually spring-summer because all grants decisions need to be made before the end of the federal fiscal year (September 30 of each year).
• Grant review panels include a chairman (nonvoting) and about three-12 other panelists (grant reviewers). In addition to individuals from the private sector, federal staff from other offices or departments may serve as reviewers on a panel.
• The time that will be required of you varies.

-- Five-day grant reviews will take only that amount of time, with no additional outside work required. Keep in mind that sometimes the days can run pretty long, depending on how quickly you work alone and as a group.

-- Shorter on-site (in the hotel) reviews of two-three days will involve some prior time on your part--about one-two days, depending on how fast you can read and score each assigned application. Generally, it will take you about three-five hours to read, make comments, and score each of your assigned grant applications.

• Τhe pay ranges from $100-$300 per day, plus travel and expenses, for a total of $500-$1,500. Sometimes you get paid on Friday when you turn in all your signed forms; sometimes your pay is mailed to you or deposited directly into your bank account up to about six weeks later. You will have to pay taxes for the amounts considered income.

Types of Grant Reviews

Federal departments use three main types of grant reviews. Each department handles some details differently, but the basic process is generally the same in each case. The differences lie mainly in how long the review lasts, how large each review panel is, whether or not you have to develop a consensus as a group, and when you receive the applications for review. During all types of reviews, one or more other panels are almost always operating concurrently.

Grant review methods that are widely used include:

On-site Review (Consensus Method). The on-site review method lasts about five days, and all work is done on-site at the hotel. You will not see the applications you are to review until the first day the review process begins at the hotel.

• Panels are smaller, with only three-four members, plus a nonvoting chairman.
• Each reviewer reads and scores all of the 10-12 applications (on average) being considered by their assigned panel.
• Usually, all panelists initially review the same applications at the same time, working alone, and then meet together daily for a few hours to discuss and give final scores to each application, one application at a time.
• At panel meetings, each reviewer first announces his or her score for each scoring section. Then, section by section, each person explains in detail how he or she developed that score. The group must adjust their individual initial scores for each section so that they are more closely aligned. This is done through discussion and negotiation.
• Final scores are then averaged, and the group prepares a consensus review form for that application that includes written comments (developed and agreed upon by the entire group) describing the strengths and weaknesses of the application section-by-section.
• The applicant will eventually receive a copy of this panel summary report (although your name will not be attached to it).
• This is a very rigorous process because you have to review many applications, your group must reach a consensus, and you do not see applications in advance.

Field Review (Primary/Secondary/General Reader Method). A field review is one in which each panelist is supposed to read, score, and prepare comments for all assigned applications before he or she comes to the hotel to begin the reviews with the entire panel.

• This features a larger panel, with an average of 10-12 persons, plus a nonvoting chairman.
• Each panelist handles an average of three applications as primary reader, two-three as secondary reviewer, and two-three as reader.
• The primary and secondary reviewers make their comments (in that order) and report their scores to the whole group.
• These two main reviewers can also recommend that the application be approved or disapproved. Approval at this stage does not mean that the applicant will receive funds. It merely means that the entire panel will then proceed to discuss and score the application. However, if the panel votes to disapprove the application, the entire panel does not give individual scores for that application.
• The reader makes comments on the application but does not have to provide a score.
• Then all panel members discuss the application and give it a score based on the discussion (since they have not read it themselves).
• No consensus is required. Each person is free to give any score he or she thinks is merited.
• The panel as a group first decides whether or not the application itself should be approved or rejected for funding. If the panel votes to reject the application, no further discussion is necessary and no scoring takes place.
• Approval by the panel means that the application can move forward for discussion and scoring, but it does not mean that it will automatically receive funding.
• After the discussion and scoring is completed, the primary and secondary reviewers must turn in their written comments on the strengths and weaknesses of the applications they reviewed.
• Panels work together in a small conference room for the entire time.
• Because you do a lot of the work in advance before coming to the hotel, this panel review lasts only about two-three days. While you might be at the hotel for only two days, you will get paid for an additional day to cover the time you spent doing the advance work--but expect the work to take longer than one day.

Telereview Session. In a telereview session, panel members are asked to read and evaluate applications based on stated criteria, and then score their assigned applications, at their own homes or offices. Later, they participate in a three-hour teleconference call with all of their fellow panelists to determine the main strengths and weaknesses of each assigned application. This session is a variation of the field review.

The Role of Federal Staff

Usually, a number of federal staff will be on-site with the panels at the hotel to help with the review process. They have different assigned roles, but all are there to help the panelists complete their task on a timely basis. Among their assigned tasks are the following:

• Develop the RFP.
• Select reviewers and panel chairmen.
• Screen applications according to the published eligibility criteria.
• Manage the review session.
• Oversee the work of any outside contractor who is helping them with hotel and travel arrangements, expense reimbursements, etc.
• Train reviewers and panel chairmen.
• Attend panel meetings (occasionally) to make sure they are making progress.
• Resolve problems and clarify issues or concerns during the process.
• Reassign or remove reviewers whose behavior or performance warrants such action.
• Critique panel summary reports daily.
• Approve payment vouchers.
• Review and edit the printed compilations of reviewers' comments for accuracy and consistency.
• Using the reviewers' comments and scores, along with project abstracts, prepare funding recommendations for their superiors.

The Role of the Panel Chairman

The chairman is selected by the federal staff from among the list of grant reviewers. Staff generally look for someone who has prior experience as a reviewer and/or panel chairman. Experience in a program area might be helpful but is not required. As a "first among equals," the chairman generally works to keep the panel moving ahead so that it completes its work by the end of the week. The chairman's duties include the following:

• Lead daily panel discussions. Chairmen generally do not read any of the applications themselves (although they can if they want to) and do not offer their own scores. Their objective is not to dictate to their fellow panelists, but to keep the process moving forward to a timely and satisfactory conclusion.
• Serve as liaison with the federal program staff.
• Monitor the progress of individual reviewers.
• Offer guidance to individual panelists who need additional assistance.
• When differences of opinion about an application exist among reviewers, direct the discussion to determine the reasons for these differences and help the group resolve their differences.
• Answer questions or get answers to questions from federal staff.
• Check reviewer scoring forms to make sure they are completed properly.
• Work with the panel to complete panel summary reports.
• Make sure that all forms are correctly completed and that panelists have everything they need to do the work.
• Notify federal staff of any problems concerning confidentiality, conflict of interest, or difficulties encountered by reviewers in completing assignments.

After all materials have been turned in to the federal staff, chairmen are the last persons on their panels to be allowed to leave. They also are frequently paid a bit more than the other panelists.

How to Read the RFP

• Read the RFP and all materials sent to you in advance--very carefully. Isolate the various important criteria that the RFP lists. Then focus on finding the information within each application that relates to each of the criteria listed in the RFP. Does the application include all the information it is supposed to include?
• Consider breaking down the criteria listed in the RFP into separate line items and then breaking down the total score allocated to that section into even smaller pieces, assigning values to each piece that will add up to the total allotted. You can make your own "template" of your ad hoc scoring system and use it for all the applications you have to review. This can help you assign "weight" to the different criteria based on your judgment. It will also help you move through each application much more quickly. Finally, it helps you explain and justify your score to your fellow panelists during later discussions of that application.

How to Read the Applications

• Don't try to memorize. Skim nonessential information in the applications that doesn't relate to published criteria from the RFP.
• Read the entire grant application at least two times. The first time, try to get a good understanding of the proposed services, program, and strategies. The second time, look for the specific elements that the RFP says they are supposed to include. It might help to highlight them with a marker or note in the margins where the required elements appear (if you are allowed to mark on the applications) in case you need to find them again later.
• If you are sent the grant applications in advance, make sure you complete all the reading, scoring, and comment writing before coming to the hotel. Some reviewers place themselves at a disadvantage and delay progress by the entire panel by not having their work done on time. Your fellow panelists will appreciate it, and if one of your colleagues is not ready to present but you are, you will be the one to present the coherent picture of the application to the entire review panel for them to score.

Participating in Panel Presentations and Discussions

• During field reviews, begin your oral comments to the full panel with a brief overview of the application itself, including a description of the organization and what its proposed program is all about. This will greatly help your fellow panelists, who have not read the grant application but do have to give it a score. After giving your summary of the organization and its approach, you can then give your strengths and weaknesses comments and your score.
• Reviewers who serve as "reader" during field reviews are not required to provide a numerical score for those applications. But you should because it gives your opinions more weight in a way that can be understood, and thus makes your views more likely to help and influence other panelists who have not read that application.
• The better prepared you are to justify your score, the more your point of view is likely to prevail (and the faster you will get through the process).
• When you want to justify the score you gave on a particular section, make a note of the page number so that you will be ready in the event you are challenged by other reviewers.
• No matter which kind of review you are doing, be prepared to listen carefully to your fellow panelists, take their insights and comments into account, and adjust your tentative scores accordingly. Their backgrounds and experiences will often prove very useful in giving full and due consideration to the applications before you.
• Try at all times to maintain a professional and courteous manner during the group's review sessions. Sometimes, discussions can turn heated, but if you "keep your cool," you will gain the confidence of other reviewers and help bring the point to a resolution.
• Generally, reviewers are not permitted to use additional criteria or consider any information that is not in the application. However, you can use knowledge you already have about the organization, its capacity, etc., since that is part of the expertise you bring to the review process.

Miscellaneous Do's and Don'ts

• Do not collaborate with other panel members during the independent part of the review process.
• Avoid conflicts of interest with the organizations whose applications you are reviewing.

-- This includes serving as an officer, director, trustee, partner, or employee of the applicant organization, or having a close family member in such a position, as well as any past, present, or anticipated financial relationship between the organization and the reviewer or a family member. It does not include knowing the organization or the staff.

-- If you or your panel has been assigned an application with which you have a conflict of interest, instruct the federal staff before evaluating the application so they can assign that application to a different panel.

• Do not evaluate one application against another. Just compare each application to the criteria published in the RFP.
• Never contact applicants before, during, or after the review process.
• Do not discuss the applications your panel is reviewing with other panels that might be working at the same time.
• Maintain confidentiality before, during, and after the review session.

-- While a list of all reviewers who participated in the review process might be released under the Freedom of Information Act (FOIA), the list will not link individual reviewers' names with the applications they scored.

Helpful Words for the Comments Section

In order to give constructive criticism to all applicants, reviewers also help prepare strength and weakness summaries for each application reviewed. The following list of words has been compiled to help you write these evaluative comments. These words will help you convey your thoughts more accurately and convincingly. They are also less likely to be sent back to you for change or improvement by the federal program staff who are at the hotel with you.

STRENGTHS WEAKNESSES

Comprehensive Partial

Justified Unjustified

Reasonable Unreasonable

Documented Undocumented

Appropriate Inappropriate

Details Broad

Exhaustive Limited

Adequate Inadequate

Thorough Careless

Focused Unfocused

Complete Incomplete

Unique Common

Extensive Limited; narrow

Provides evidence Lacks evidence

Innovative Common

Sound Unsound

Qualified Unqualified

Strong Weak

Experienced Inexperienced

Demonstrates Conceals

Describes Fails to describe

Feasible Impossible

Presents Fails to present

Specify Does not specify

Convincing Unconvincing

Examples of "More" and "Less" Useful Comments

Strengths

Less Useful
The applicant plans to use an outside evaluator.

Discussion
The comment does not explain why this is important.

More Useful
The applicant's plan to utilize the services of a highly experienced outside evaluator will help to ensure that the evaluation is conducted appropriately and is both valid and useful (p. 43).

Less Useful
The application includes a dissemination plan.

Discussion
The comment does not explain how or why this is good or bad.

More Useful
The application has a well-developed dissemination plan that includes a good understanding of the various products' usefulness to the different audiences (p. 25).

Weaknesses

Less Useful
Parent involvement in an advisory capacity should be strengthened to ensure success.

Discussion
The comment is vague and does not explain why parent involvement is important.

More Useful
The essence of this project is parent involvement and volunteers. By including them on the committees, it is possible to achieve a more collaborative effort (pp. 4, 11).

Less Useful
The timetable for developing a database of confidential client information by the end of the project period is unrealistic.

Discussion
The comment does not give the basis for this conclusion.

More Useful
The development of a client information database in one year is unrealistic because of the level of effort required to develop software (pp. 7, 41).

Less Useful
The budget is missing some key items.

Discussion
The comment is not supported with details.

More Useful
The applicant fails to provide funds for reimbursement of health centers for training expenses, as required by the RFP (pp. 22-23).

The proposal fails to provide any supporting data on fiscal control and accounting procedures (pp. 7, 41).

The proposed budget lacks sufficient detail under the various categories to determine its reliability (pp. 41).

Common Weaknesses and Errors to Look for in Grant Applications

It is not unusual for experienced grant reviewers to see common errors and weaknesses in the grants they review. As a new reviewer, you need to know what these errors and weaknesses are and what to do when you find them.

1. Ideally, grant applicants should always seek grant opportunities that match their program's goals and objectives. Watch out for applicants who appear to be doing that the other way around. Too often applicants will try to make their "round" program fit into a "square" RFP.
2. Note in your comments areas where the applicant has not provided the information specifically asked for in the RFP. Example: "The applicant has failed to include funds in their budget for two trips to Washington for staff to participate in the required training sessions."
3. Note where the applicant proposes to do something outside the scope of the RFP, or in opposition to the letter or spirit of the RFP (and its authorizing legislation, a copy of which will be provided to you).
4. Evaluate the data cited by the applicant very carefully to make sure the information is not outdated and is relevant to the point the applicant is trying to make. Also make sure that any literature cited by the applicant is not outdated or irrelevant. You will have to base that judgment on your own knowledge of the literature in that field.
5. If the applicants include a section on how they plan to evaluate their program, make sure that their plan will give them the kinds of data and feedback you think should be gathered in order to judge the program's effectiveness.
6. Carefully review the educational and professional credentials of the staff in key positions. Can they perform the work they are being asked to perform? Often, applicants will include older resumes that are not updated or do not highlight skills that are specifically needed to perform the tasks required. If you can't judge them based on what they have given you in the application, note that in your review.
7. Make sure the applicants' proposed budget includes everything that they are supposed to request. Are the budget items allowable? Are they necessary for the success of the project? Are they reasonable? Are they costs that would be incurred by a "prudent" person? Does the information in the budget form match the proposed program as described in the narrative?
8. Look carefully at the proposed budgets as if the money were coming out of your own pocket. Don't spend a lot of time quibbling with fellow reviewers over specific costs; but if, for example, you think the applicants are asking to compensate someone at too high an amount or they really don't need those 10 new computers to conduct this project, put that down in the budget weaknesses section and reduce the score accordingly.
9. Make sure the applicants show evidence that they understand and have worked with the target population they are supposed to serve.
10. Think about the sustainability of this program after the funds run out. Have the applicants addressed that issue?
11. Don't try to read between the lines of an application. If the applicants haven't made it clear, you can't assume that it's true.
12. If you feel applicants should not have been included for review because they did not fit the eligibility guidelines, be prepared to cite specifics in order to explain your assertion to the rest of the panel. Occasionally, this does happen. In that case, the group can simply give the application a very low score, which effectively knocks it out of the running. Or, in a field review, they can recommend that the application be rejected, which ends the consideration of that grant by reviewers.

Evaluations: Encouraging Accountability

Many RFPs ask applicants to include some kind of evaluation process in their grant proposal. It is usually significantly more useful to Congress and the public to have grant recipient organizations gather data regarding the outcome of their federally funded programs, rather than merely go through a process form of evaluation. For example, a process evaluation proposal is focused more on programs and activities than on the results (e.g., increased knowledge or changed behaviors). Process evaluations would help applicants measure such things as what they are going to do, how many brochures advocating abstinence they will print and mail out, or how many teens will be exposed to their media campaign against drinking and driving.

On the other hand, a proposed outcome evaluation proposal will describe how applicants plan to measure what happened after their program is completed as opposed to not having the proposed program at all. It asks: What results are they achieving? What impact have their program activities produced? In what ways is their community a different and better place as a result of their efforts?

Thus, you should be encouraged to look for applicants who use an outcome evaluation as part of their application. If the RFP calls for some kind of program evaluation but doesn't tell applicants which kind they should use, be prepared to explain the higher value of an outcome evaluation to the other grant reviewers on your panel and to comment positively on those applications that are willing to measure program effectiveness in this way.

What Happens When Your Work Is Done: Who Gets the Money?

After the review panels complete their work, the grant team conducts an internal review to assure that the reviewers' scoring sheets are completed correctly and that the application meets all the program's requirements. These two review processes become the basis for a final listing of applicants that the grant team considers for funding.

Does A High Score Guarantee Funding?

No. Even if a proposal ranks high and the grant team considers it for funding, the application still might not be funded because of one or more of the following reasons:

1. A large number of high-quality applications may have been received under the competition, which means that a number of applicants scored very close together.
2. Congress and the department can allocate only a limited amount of money to that program, so only a certain number of applicants can receive funds. If there are several high-scoring applicants, it is probably hard to distinguish among that group of applicants when their scores are separated by just a few points.
3. The federal staff determines that they want to spread the grants around geographically, so they first take scores into account and then consider the region represented by the grantees.

Be assured, however, that both career staff and Administration appointees who make the final funding decisions give tremendous weight to the scores they receive from their grant review panels.

How To Become A Grant Reviewer

Now that you understand the process and how important it is, how can you participate in it? There are several ways you can get your name in consideration for a grant review panel.

• A few agencies have Web sites where they allow members of the public to submit their resumes for consideration and express their interest in several specific grant programs. Your name would then go into a database of potential grant reviewers to be considered for use when needed.
• You can also write to the program officer of a particular grant, include your resume in the letter, and express your interest in being considered to be a reviewer for the next grant cycle of their program. The names and addresses of these individuals can be found in the Catalog of Federal Domestic Assistance (see Web address below), listed with each grant program in which you might have an interest.
• In addition, you can follow grant announcements as they are published in the Federal Register. When you see a program in which you have an interest and you think you might be able to offer some expertise, write to the program officer whose name and address is included in the announcement, expressing your interest, and include your resume. You can have the Federal Register Table of Contents e-mailed to you each day it is published so that you can follow this more easily.

Federal Department or Agency Grants Web Sites

Department of Agriculture
http://www.usda.gov/nonprofi.htm

Department of Commerce
http://home.doc.gov/Assistance/Grants/

Corporation for National and Community Service
http://www.nationalservice.org/egrants/index.html

Department of Defense
http://aspe.hhs.gov/cfda/ideptdod.htm

Department of Education
http://ed.gov/topics/topics.jsp?&top=Grants+%26+Contracts

Department of Energy
http://www.sc.doe.gov/production/grants/grants.html

Environmental Protection Agency
http://www.epa.gov/epahome/grants.htm

Federal Emergency Management Agency
http://www.fema.gov/ofm/grants1.shtm

Department of Health and Human Services
http://www.hhs.gov/agencies/grants.html
http://www.hhs.gov/grantsnet/

Department of Housing and Urban Development
http://www.hud.gov/grants/index.cfm

Department of the Interior
http://www.doi.gov/non-profit/fax.html

Department of Justice
http://www.usdoj.gov/10grants/index.html

Department of Labor
http://www.dol.gov/oasam/grants/prgms.htm

National Endowment for the Arts
http://www.arts.gov/guide/index.html

National Endowment for the Humanities
http://www.neh.fed.us/grants/index.html
http://www.neh.fed.us/manage/index.html

Small Business Administration
http://www.sba.gov/expanding/grants.html

Department of State
http://www.statebuy.gov/busops.htm
http://aspe.hhs.gov/cfda/ideptdos.htm

Department of Transportation
http://www.dot.gov/ost/m60/grant/
http://aspe.hhs.gov/cfda/ideptdot.htm

Department of the Treasury
http://aspe.hhs.gov/cfda/idepttre.htm

Department of Veterans Affairs
http://aspe.hhs.gov/cfda/ideptdva.htm