|
|
National Council of State Directors of Adult Education |
|
|
|
Last Updated: 03/11/05 RESPONSE TO THE OMB “PROGRAM ASSESSMENT RATING TOOL” (PART) ADULT EDUCATION STATE GRANTS (Title II Workforce Investment Act 1998) Information in the President’s 2007-2008 budget documents erroneously reports that adult education does not have the data to document that it is a successful program. Beginning in 1998, adult education state programs have vigorously collected data to provide valid, reliable documentation that reflects impressive levels of performance Section I. The Administration Picture: Attached (Section III) is a detailed analysis of the PART rating for adult education. In summary, three points are important. 1. Numeric vs. Percentage Targets One part of the administration, OMB, requires numeric targets for its Key Performance Measures. Another part of the administration, ED, established percentage targets for adult educations PART Key Performance Measures. Adult education is rated low because we have not set numeric targets. HOW CAN CONGRESS HELP? Convince OMB to accept percentage targets as the better measure for adult education effectiveness. Rationale: Because adult education responds to a number of adult learner needs (e.g., improved basic skills, GED or high school diploma, transition to post secondary, maintaining a job, preparing for a job, transition to a training program, parents preparing to help their children with homework), we never know in advance how many students will cite employment as their reason for attending. Therefore we cannot assign a target number to be successful in their quest for employment, better employment, or any of the indicators. Agreeing to a percentage of adult students that will reach their goals is the only sensible procedure. 2. Common Measures OMB rates programs based on “common measures for job training programs.” Those common measures include “increased earnings.” Adult education is an “education” program and does not currently collect all the data elements. Adult education is rated low because we do not collect all the common measures data. HOW CAN CONGRESS HELP? Convince OMB that adult education is an “education” program not a “job training program.” Rationale:
3. “Ambitious Targets” ED set performance targets that may be out of line with actual program performance. In fact, the PART budget analysis characterizes them as ambitious targets” and further questions “whether theses ambitious targets are attainable.” (page 107 of http://www.whitehouse.gov/omb/budget/fy2006/pdf/ap_cd_rom/part.pdf) In short, even though the states have set and met or exceeded their negotiated targets, ED may have set unrealistic “unattainable” levels for itself. As a result, the effects of the program were questioned because possibly “unattainable” levels were not attained. Even so, progress on four of the five performance indicators has steadily increased and one has remained relative constant over the three years of data collection. Three of the five possibly “unattainable” ED levels were met or exceeded. These data certainly do not represent a program of questionable success. HOW CAN CONGRESS HELP? Help the Administration set realistic targets based on performance data. Rationale: OMB has reported that ED has set targets that may not be attainable. Section II. The State Picture: Ø The National Reporting System (NRS) that is mandated by Congress does provide data that proves adult learners are attaining high school credentials; are going on to post-secondary education; are getting and keeping jobs and are increasing their ability to read, write and do mathematics. That is what these programs are mandated to accomplish and that is what they are doing for approximately three million adult learners annually. Ø All states, with very few exceptions, have met or exceeded their U.S.D.E. approved annual performance goals every year since 2002. Ø A great deal of federal and state funding and human resources has been tied to creating, implementing and fine tuning a reporting system that does work. Ø Fifty years of government effort has been tied to creating, implementing and fine tuning a system of adult basic education and literacy. While the system is consistently under funded to meet its national goal, data does clearly show adults are benefiting in large numbers relative to the funding level. Ø The PART report is based on a study done when the NRS was in its infancy and not yet fully implemented. According to the PART itself, “There are extensive populations of adults with low literacy skills in the U.S. Further, the non-English speaking population is growing. The literacy skills of these populations are too low to be effective members of the workforce and to participate as citizens in our democratic society”. Ø AEFLA is the only legislation and funding that provides federal oversight and fiscal support to adult literacy programming in this country. Nothing similar will exist to meet this fundamental economic and social need. Ø The PART is based on 2001 data that do not reflect the very rigorous accountability system in place at the federal, state and local levels and outlined here: o The NRS clearly outlines a set of Performance Indicators that directly support the national measure of reducing illiteracy in the US. o Each state annually negotiates ever-increasing performance levels for each of the five student Performance Indicators and is held accountable for meeting each one through annual reporting. o The states fund local programs that are, in turn, accountable for INDIVIDUAL STUDENT OUTCOMES based on the five core measures. o State level, inter-agency computerized data matching systems are in place to track outcome achievement. o The U.S.D.E. approves State Plans that must include systems for funding or sanctioning local programs based on their performance on the Performance Indicators. Again, these data certainly do not represent a program of questionable success. HOW CAN CONGRESS HELP? Help the administration understand that adult education collects and reports rigorous data regarding adult learner success and that with a very few exceptions, state adult education meet or exceed their performance standards. Section III. ATTACHMENTDetailed Analysis of the OMB PART rating. Section I: Program Purpose & Design—100% out of a possible 100% Section II: Strategic Planning—29% out of a possible 100% OMB has established certain principles for job training programs. Because adult education is an “education” program which supports job training, the principles are not always applicable. 1. Common measures: OMB’s common measures for job training programs include “increased earnings” for participants. As an education program, adult education does not collect these data on its students. As a result, adult education is rated “0” on PART questions related to having specific goals. 2. Numeric vs. percentage targets: In PART questions related to performance, OMB requires “numeric” targets (the number of participants who attain goals). The Department of Education (ED) has set “percentage” targets. As a result, adult education is rated “0” for questions related to reporting performance. According to OMB, they are not “aligned with budget policy.” Section III: Program Management—67% of 100% 1. Again, rating of Program Management is tied to the “strategic goal framework” and the “common measures.” As a result, adult education is rated low because it “does not provide sufficient valid and reliable performance information.” Section IV. Program Results—0% out of 100% 1. Again, rating of results are tied to “established targets” (numerical) and “common measures” and adult education is deemed to have poor results without numerical data on the key performance indicators. SUMMARY. OMB, one branch of the administration, requires “numeric targets” to document performance. ED, another branch of the administration, requires “percentage targets” as evidence of results. OMB rates programs based on “common measures for job training programs.” Adult education is an “education” program and does not currently collect all the data elements. As a result, adult education is 1) being rated by PART on criteria on which it does not collect data and 2) is penalized because the two departments of the administration do not agree on evidence (i.e., numeric or percentage) to document results. OPINION: 1. The “common measures for job training programs” do not apply to an education program like adult education. The cause and effect relationship between an adult pursuing an education and increase in earnings is spurious at best and impacted greatly by many non-education related variables. Adult education should not be evaluated on increased earnings of adult learners. 2. Adult education has documented continuous improvement of adult performance by way of the National Reporting System which reports the percentage of learners who are successful. Due to the ever changing adult student populations, “numeric” targets are impractical and percentage of students who enroll for specific purposes serves the program evaluation process well. The percentage targets should be maintained and accepted by OMB. Because adult education responds to a number of adult learner needs (e.g., improved basic skills, GED or high school diploma, transition to post secondary, maintaining a job, preparing for a job, transition to a training program, parents preparing to help their children with homework), we never know in advance how many students will cite employment as their reason for attending. Therefore we cannot assign a target number to be successful in their quest for employment, better employment, or any of the indicators. Agreeing to a percentage of adult students that will reach their goals is the only sensible procedure.
|
|
|
Contact us: Dr. Lennox
McLendon; 444 North Capitol Street, NW; Suite 422; Washington, DC 20001 |
|
