My path to the point where I am now is probably very different from the path many of you took to get to this point. There were transitions happening at my workplace, and opportunities were coming available. The Learn and Serve Indiana grant was one of these.
My knowledge of the grant was very little. I worked in close proximity to the previous grant manager, and through cubicle eavesdropping – which is unintended at times – I knew that this grant was capable of many great things that others were not. So, when it opened up, I jumped at the opportunity.
The first year of the job flew. Undeniably, it was one of the most challenging points in my life. Almost everything was new to me. What were these strange things: Semester of Service; mini-grants; LASSIE – isn’t that an old show my grandparents used to make me watch? It was kind of like Mars to me – it was foreign soil and I was the astronaut sent to discover it.
I was quickly introduced to our Service-Learning Technical Advisors and we dove in right away. Our SLTAs – who have become the face of LSI in the field – are and continue to be my rock with LSI. I cannot and will not ever be able to thank them enough for the work that they’ve helped me accomplish while I’ve been in charge of this grant. In many ways, they are the ones that make the grant what it is, and they deserve much more credit than I do at times.
Soon, the time came for me to go to D.C. – er, Crystal City – and receive my mandated LSA training. Being new to this, I was in instant shock. There was a whole world of service out there that I had never known about. I have to admit, I volunteered in high school and college, but I wasn’t the everyday go-getter. However, the people I met at the conference helped me further realize that service is more than just going overseas and fighting for your country. There is service that needs to be done right here in our country – in our own backyards – and we are the ones tabbed for this responsibility. And this realization is what brought me back to Indianapolis ready to serve.
Throughout my first year, I met many different administrators, teachers, students, and community members who thought the same. I have been knocked off of my feet on numerous occasions after meeting 8- and 9-year olds who can recite – from memory – how pollution affects their community to a roomful of adults in a hot, hotel meeting room. I have seen people brought to tears because of their demonstrations.
Over my first year, I saw many great projects that have motivated communities to make changes; that have motivated business to change their policies in regards to what’s best for the environment; that have brought a rich history back into a community that has provided for so many. I have met many great, well-rounded students who will someday be the face of this country. There is absolutely no doubt in my mind that being a part of a service-learning project had something to do it.
Although my first year was shockingly new at times to me, I welcomed the test. Being able to see – in person – the affects of a good service-learning project, both in and out of the classroom, has offered me a much different and brighter outlook on the world than I had just a year ago. Although at times the paperwork can bog you down a bit, I make it a point to go out into the field as much as possible to see these projects, and to be a part of it all. It’s a good feeling to know that you are a part of something positive, and it’s now a part of your job.
Andrew Conway, Indiana Department of Education
Monday, January 31, 2011
Monday, January 24, 2011
Evaluation
In this age of accountability, as program directors and coordinators we know that Evaluation is a cornerstone of our continuing ability to prove our outcomes, maintain continuous improvement, secure broad-based support from all stakeholders and, ultimately, maintain stable levels of funding for our initiatives. The question is…how do we do this? Evaluation is difficult to implement, manage and accomplish and is most of all…very costly and time consuming. Typically, large amounts of both time and funding are required to form an evaluation process that has any sort of validity in the heavily-scrutinized education research field. This usually means hiring external evaluators and getting set to deal with logistically complex data management and retrieval procedures and with various levels of requirements as to securing access to the student numbers. This is simply the reality of meeting the “burden of proof” which to this day still hangs somewhat heavily over the service-learning field. We have made great gains in the directions of standardizing our practice based on indicators which should lead to resultant increases in student academic outcomes, but we still do not, as a field, have a mass of well-designed studies that show the beneficial outcomes as a result of high-quality practice. As a field and to be accepted as a viable school-based educational methodology we must grow to maturity together in this area. But how is this accomplished? It is no simple task and minimizing the effort required to implement professional evaluation can be equally counterproductive; improperly designed evaluation tools are more or less the evaluative equivalent to “a shot in the dark,” until proper design procedures are considered at the front-end, then there will be no verifiable result.
Javier Betancourt
FL&S Program Officer
In Florida we have made some strides in the area of evaluation, but we are by no means at a point where conclusive reporting procedures have been developed. Florida has structured its evaluation and reporting requirements as follows: since 1992, all sub-grantees have had to identify at least one need of participating students that would be addressed through the proposed service-learning activities. They are required to project a quantifiable impact that the service-learning activities will have on the identified need, and describe how that impact will be measured. The need can be academic (grades, test scores, etc.), behavioral (i.e., conduct or attendance), or related to skills acquisition or civic/social responsibility, as measured via teacher-designed or –identified observations, rubrics, grading systems, or surveys. Renewal sub-grantees also have to project an impact their activities will have on the community/ties they will serve, and describe how that impact will be measured.
From 2003-2006, student participants completed pre- and post-surveys focused on civic engagement. The surveys were derived from sample instruments provided by the Corporation for National and Community Service. No elementary-age survey was available, so program staff modified the middle school survey to be developmentally appropriate for students in grades 3-6. From 2006-2009, Florida Learn & Serve contracted with a professional evaluator to develop instruments to measure civic engagement and achievement motivation in grades 4-12. There were pre- and post-surveys for civic-engagement, and a post-survey for achievement/motivation. The surveys were developed, tested, evaluated, revised, re-tested, revised again, and validated using rigorous standards. Instruments were approved by an Institutional Review Board, and active consent was received for all participants, including parents of students under age 18.
In addition, sub-grantees must complete data tables at the end of their grant period that report on participating students’ attendance, conduct, and performance on the statewide exam. For comparison, the sub-grantees provide these same data for non-participating students in the same grades at their schools (see forms and instructions at http://learnandserve.hhp.ufl.edu/projects/evaluation/evaluation1011.html).
Participating school districts collect the data and consent forms. Once collected, the data, which contain no student-identifying information, are entered into databases from which reports may be generated. Since the 2009-2010 school year, sub-grantees have used these instruments and data forms. The goal of the surveys and data tables is to collect data that can be aggregated statewide; these data will complement the unique student impact data that each participating school is also collecting.
As the 2009-10 data have not yet been studied, it is not known what impacts the sub-grants have had. Previous final reports by sub-grantees have, in hundreds of individual cases, indicated positive academic, social, civic, or skills acquisition gains of participating students. But, for all the aforementioned reasons related to validity and reliability—data that are self-reported, from small samples, non-longitudinal, idiosyncratic, wholly quantitative or qualitative, involving small populations, etc.—the Florida results are not complete, comprehensive, replicable or of sufficient rigor to draw conclusions on the degree to which the treatment of service-learning was responsible for gains. Hence there is much room for growth to reach a level of sufficiency in evaluation.
In terms of what you want to measure, as a director you must ask yourself the question of whether you are hoping to get data on servers, the served, or both? We found it was a handful just to get good data on students who served. The goal of the Florida Learn & Serve program is to engage K-12 students in service-learning that is directly tied to the students’ curricula, grades, performance, and acquisition of related KSAs that the schools and teachers want the students to learn. Therefore, we’re primarily trying to measure the degree to which engagement in service learning leads to improvements and gains in these areas. Every project has unique and individualized needs and related goals. Some hope to improve performance in reading, math, science, social studies, or other core subjects (by having students apply those subjects via service). Others seek to modify and improve behavior. Others focus on the development of civic competencies or job/career skills through the service. These individual projects yield individual results that cannot be aggregated across the state, school district, schools, classrooms, or even (sometimes) from one student to another.
Because such data are so individualized (and we do not have the time, money, expertise, or staff to conduct n = 1 studies), we also use the aforementioned surveys focusing on civic engagement and achievement motivation and collect basic comparative data on participating vs. non-participating students related to attendance, grades, and conduct—see the above link. This is a realistic view of the state of evaluation in our state program hence it shows both strengths and weaknesses. At our stage of development as a methodology there really is no greater impact to be made than through validating the effectiveness of our model. Until we can prove quantitatively what we already know to be true qualitatively we will continue to be relegated to the periphery of the educational practice. Bu there is everything to be gained by meeting the “burden of proof of concept” that has been placed before us.
Joe Follman
FL&S Director
Javier Betancourt
FL&S Program Officer
In Florida we have made some strides in the area of evaluation, but we are by no means at a point where conclusive reporting procedures have been developed. Florida has structured its evaluation and reporting requirements as follows: since 1992, all sub-grantees have had to identify at least one need of participating students that would be addressed through the proposed service-learning activities. They are required to project a quantifiable impact that the service-learning activities will have on the identified need, and describe how that impact will be measured. The need can be academic (grades, test scores, etc.), behavioral (i.e., conduct or attendance), or related to skills acquisition or civic/social responsibility, as measured via teacher-designed or –identified observations, rubrics, grading systems, or surveys. Renewal sub-grantees also have to project an impact their activities will have on the community/ties they will serve, and describe how that impact will be measured.
From 2003-2006, student participants completed pre- and post-surveys focused on civic engagement. The surveys were derived from sample instruments provided by the Corporation for National and Community Service. No elementary-age survey was available, so program staff modified the middle school survey to be developmentally appropriate for students in grades 3-6. From 2006-2009, Florida Learn & Serve contracted with a professional evaluator to develop instruments to measure civic engagement and achievement motivation in grades 4-12. There were pre- and post-surveys for civic-engagement, and a post-survey for achievement/motivation. The surveys were developed, tested, evaluated, revised, re-tested, revised again, and validated using rigorous standards. Instruments were approved by an Institutional Review Board, and active consent was received for all participants, including parents of students under age 18.
In addition, sub-grantees must complete data tables at the end of their grant period that report on participating students’ attendance, conduct, and performance on the statewide exam. For comparison, the sub-grantees provide these same data for non-participating students in the same grades at their schools (see forms and instructions at http://learnandserve.hhp.ufl.edu/projects/evaluation/evaluation1011.html).
Participating school districts collect the data and consent forms. Once collected, the data, which contain no student-identifying information, are entered into databases from which reports may be generated. Since the 2009-2010 school year, sub-grantees have used these instruments and data forms. The goal of the surveys and data tables is to collect data that can be aggregated statewide; these data will complement the unique student impact data that each participating school is also collecting.
As the 2009-10 data have not yet been studied, it is not known what impacts the sub-grants have had. Previous final reports by sub-grantees have, in hundreds of individual cases, indicated positive academic, social, civic, or skills acquisition gains of participating students. But, for all the aforementioned reasons related to validity and reliability—data that are self-reported, from small samples, non-longitudinal, idiosyncratic, wholly quantitative or qualitative, involving small populations, etc.—the Florida results are not complete, comprehensive, replicable or of sufficient rigor to draw conclusions on the degree to which the treatment of service-learning was responsible for gains. Hence there is much room for growth to reach a level of sufficiency in evaluation.
In terms of what you want to measure, as a director you must ask yourself the question of whether you are hoping to get data on servers, the served, or both? We found it was a handful just to get good data on students who served. The goal of the Florida Learn & Serve program is to engage K-12 students in service-learning that is directly tied to the students’ curricula, grades, performance, and acquisition of related KSAs that the schools and teachers want the students to learn. Therefore, we’re primarily trying to measure the degree to which engagement in service learning leads to improvements and gains in these areas. Every project has unique and individualized needs and related goals. Some hope to improve performance in reading, math, science, social studies, or other core subjects (by having students apply those subjects via service). Others seek to modify and improve behavior. Others focus on the development of civic competencies or job/career skills through the service. These individual projects yield individual results that cannot be aggregated across the state, school district, schools, classrooms, or even (sometimes) from one student to another.
Because such data are so individualized (and we do not have the time, money, expertise, or staff to conduct n = 1 studies), we also use the aforementioned surveys focusing on civic engagement and achievement motivation and collect basic comparative data on participating vs. non-participating students related to attendance, grades, and conduct—see the above link. This is a realistic view of the state of evaluation in our state program hence it shows both strengths and weaknesses. At our stage of development as a methodology there really is no greater impact to be made than through validating the effectiveness of our model. Until we can prove quantitatively what we already know to be true qualitatively we will continue to be relegated to the periphery of the educational practice. Bu there is everything to be gained by meeting the “burden of proof of concept” that has been placed before us.
Joe Follman
FL&S Director
Friday, January 21, 2011
Linking Learning to Environmental Action
“Service-learning’s pedagogical power lies in its ability to integrate academic content with hands-on learning, authentic problem solving, and community action to enhance students’ academic achievement and civic development.”
Our schools have been understandably preoccupied with student achievement in language arts and mathematics for several years. These subjects are at the foundation of learning. Included with reading and math, however, students need knowledge of science and history, cultures and languages, art, music, health and physical activity. Without a full understanding of the integration of all disciplines, humans cannot understand themselves in the context of our environment. Our pattern identification, reasoning, and systems understanding abilities require that we think carefully and act responsibly to safeguard the planet for future generations.
Most K-12 students today are aware of environmental issues. From the second grade through high school they have studied and can speak of climate change, rain forest loss, endangered plants and animals. Many of them discuss these issues at home with parents, grandparents, and siblings. But do they know where trash goes when it leaves their house, why we recycle, or how the runoff from our roofs and parking lots affects streams, rivers and the Coastal Bays? They struggle to understand the interdependency of natural systems or the complexity of adaptations that enable survival.
During the 2009-2010 academic year, CalServe (California Department of Education) funded twenty one watershed projects with the expectation that there would be a measurable benefit to the community. Each project was unique, recognizing local needs and student interest. Attention in the design of each project was given to the connections among subject matter proficiency, the age of the students, the characteristics of the communities in which the schools are located, and the benefits that would result to the communities and the state. Projects ranged from native xeriscopic plantings in the California desert to salt marsh restoration in Coastal Bays. Students were asked to study, plan, do, evaluate, reflect and report. In each instance high quality Service-Learning was the guideline and the goal. The projects did not only provided an opportunity for youth to understand the environment in which they live, they have also provided these youth a way of better understanding how their actions benefit their community. And, the projects provided the community with an active way to understand its youth.
A full report on the 2009-10 Community Benefits of Twenty-One Watershed Related Service-Learning Projects will be released this spring.
Michael Brugh, CalServe Program Director
CalServe Initiative
California Department of Education,
1430 N Street, Suite 6408
Sacramento, California 95814-5901
Ph. 916-319-0543
Fx. 916-323-6061
mbrugh@cde.ca.gov
http://www.cde.ca.gov/ci/cr/sl
Our schools have been understandably preoccupied with student achievement in language arts and mathematics for several years. These subjects are at the foundation of learning. Included with reading and math, however, students need knowledge of science and history, cultures and languages, art, music, health and physical activity. Without a full understanding of the integration of all disciplines, humans cannot understand themselves in the context of our environment. Our pattern identification, reasoning, and systems understanding abilities require that we think carefully and act responsibly to safeguard the planet for future generations.
Most K-12 students today are aware of environmental issues. From the second grade through high school they have studied and can speak of climate change, rain forest loss, endangered plants and animals. Many of them discuss these issues at home with parents, grandparents, and siblings. But do they know where trash goes when it leaves their house, why we recycle, or how the runoff from our roofs and parking lots affects streams, rivers and the Coastal Bays? They struggle to understand the interdependency of natural systems or the complexity of adaptations that enable survival.
During the 2009-2010 academic year, CalServe (California Department of Education) funded twenty one watershed projects with the expectation that there would be a measurable benefit to the community. Each project was unique, recognizing local needs and student interest. Attention in the design of each project was given to the connections among subject matter proficiency, the age of the students, the characteristics of the communities in which the schools are located, and the benefits that would result to the communities and the state. Projects ranged from native xeriscopic plantings in the California desert to salt marsh restoration in Coastal Bays. Students were asked to study, plan, do, evaluate, reflect and report. In each instance high quality Service-Learning was the guideline and the goal. The projects did not only provided an opportunity for youth to understand the environment in which they live, they have also provided these youth a way of better understanding how their actions benefit their community. And, the projects provided the community with an active way to understand its youth.
A full report on the 2009-10 Community Benefits of Twenty-One Watershed Related Service-Learning Projects will be released this spring.
Michael Brugh, CalServe Program Director
CalServe Initiative
California Department of Education,
1430 N Street, Suite 6408
Sacramento, California 95814-5901
Ph. 916-319-0543
Fx. 916-323-6061
mbrugh@cde.ca.gov
http://www.cde.ca.gov/ci/cr/sl
Monday, January 10, 2011
Success Story: Norwich Public Schools
Kids In Action
During the ASPIRE Summer Service Academy, 60 Norwich middle school students participated in over 60 hours of service planning, implementation, and demonstration. Students were split up to work on three different projects for the duration of July 2010.
One project investigated the mission and needs of downtown Norwich social service agencies that provide for the city’s most impoverished individuals and families. This group investigated what the most pressing social issues are for some of Norwich’s residents and what they and others can do to improve the living conditions of others. The students decided that they wanted to learn how people in Norwich volunteer and recognize them for their actions through a video. They interviewed staff at St. Vincent de Paul’s Place, toured the facility, and helped serve a meal. Before the end of the program, they began to edit the footage that they had from their project and used their reflection time to start writing a poem dedicated to the people in Norwich that serve others in need. This project has continued during the school year.
The second group planned and developed a fruitful student and community garden at Teacher’s Memorial Middle School to provide fresh produce for families in need. Students designed the layout for raised beds and a patch for squash and pumpkins. They also built a fence, a gate, and concrete stepping stones to guide visitors through the garden and protect their work and harvest from animals and other unwanted visitors. About 50 pounds of vegetables were harvested and distributed among the project participants and patrons at St. Vincent de Paul’s Place. They concluded the 4 weeks by visiting St. Vincent de Paul’s Place with shovels, compost, seeds, and straw to rehabilitate 3 raised beds at the soup kitchen that had been left untended for a few seasons. The students planted a late season round of beets, cucumbers, beans, and spinach for the volunteers and patrons to tend and harvest.
The last group focused on the elders in their own families, investigating their family lineage and the topic of diversity in their own lives. They researched the cultural and ethnic diversity present in Norwich, CT and the origins of their own family. They interviewed their parents and grandparents to design a family tree that included symbols and objects representing the lives of their grandparents and great grandparents, many of whom originated in countries outside the United States. The group presented their work to their families and the other participants at the Summer Service Academy.
Submitted by Agnes Quinones, Connecticut
During the ASPIRE Summer Service Academy, 60 Norwich middle school students participated in over 60 hours of service planning, implementation, and demonstration. Students were split up to work on three different projects for the duration of July 2010.
One project investigated the mission and needs of downtown Norwich social service agencies that provide for the city’s most impoverished individuals and families. This group investigated what the most pressing social issues are for some of Norwich’s residents and what they and others can do to improve the living conditions of others. The students decided that they wanted to learn how people in Norwich volunteer and recognize them for their actions through a video. They interviewed staff at St. Vincent de Paul’s Place, toured the facility, and helped serve a meal. Before the end of the program, they began to edit the footage that they had from their project and used their reflection time to start writing a poem dedicated to the people in Norwich that serve others in need. This project has continued during the school year.
The second group planned and developed a fruitful student and community garden at Teacher’s Memorial Middle School to provide fresh produce for families in need. Students designed the layout for raised beds and a patch for squash and pumpkins. They also built a fence, a gate, and concrete stepping stones to guide visitors through the garden and protect their work and harvest from animals and other unwanted visitors. About 50 pounds of vegetables were harvested and distributed among the project participants and patrons at St. Vincent de Paul’s Place. They concluded the 4 weeks by visiting St. Vincent de Paul’s Place with shovels, compost, seeds, and straw to rehabilitate 3 raised beds at the soup kitchen that had been left untended for a few seasons. The students planted a late season round of beets, cucumbers, beans, and spinach for the volunteers and patrons to tend and harvest.
The last group focused on the elders in their own families, investigating their family lineage and the topic of diversity in their own lives. They researched the cultural and ethnic diversity present in Norwich, CT and the origins of their own family. They interviewed their parents and grandparents to design a family tree that included symbols and objects representing the lives of their grandparents and great grandparents, many of whom originated in countries outside the United States. The group presented their work to their families and the other participants at the Summer Service Academy.
Submitted by Agnes Quinones, Connecticut
Subscribe to:
Posts (Atom)
