Data & Research EVALUATION OF PROGRAMS
The Secretary's Conference on Educational Technology-1999
Perspectives on Technology and Education Research: Lessons from the Past and Present

Margaret Honey, EDC/Center for Children and Technology2
Katherine McMillan Culp, EDC/Center for Children and Technology
Fred Carrigg, Union City New Jersey Board of Education


This paper1 offers a perspective that grows out of what we, at EDC's Center for Children and Technology, have learned from nearly three decades of research on educational technology. Rather than providing a detailed account of what we now know about the impact of technology on learning, we discuss where the research field is heading and review what we think of as the most promising directions for technology's role in education (President's Committee of Advisors on Science and Technology, 1997; Bransford, Brown, Cocking, 1999; Coley, Cradler, & Engel, 1997).

Several factors are prompting us to think differently about research. Each one is based in our observations of steady growth and change on other fronts.

  • First, there are changes in the nature of the technological elements involved in the research;
  • Second, there are changes in the kinds of research questions being asked.
  • And third, there are changes in how research is being done and the methods being used.

Technological change. Throughout the 1970s and '80s, technical innovation brought increasingly diverse and more powerful technological tools into schools. Early studies sought to demonstrate the impact of technologies or software on student learning and they were tied very specifically to the particular technologies used by the subjects of the study. These technologies were typically text-based, locally networked or stand-alone computer-assisted instruction applications. As these technologies have become outdated and replaced by graphics-rich and networked environments, the studies that looked at impact on students have been outdated themselves. Additionally, because these studies tended to focus so specifically on particular technologies and their impact, they contributed little to the larger and more challenging project of learning about the generalizable roles that technologies can play in addressing the key challenges of teaching and learning, and about optimal designs for such technologies.

The pace of both technological development and the introduction of new technologies into educational settings has dramatically accelerated during the past decade. The combination of computation, connectivity, visual and multimedia capacities, miniaturization, and speed have radically changed the potential for technologies in schooling. These developments are now making it possible for technologies to be designed and deployed to produce powerful and linked technologies that can substantially address some of the core problems of education. For example, there are currently existing technologies that are ideally suited to ameliorating problems like teacher isolation, access to rich and substantial resources, and parent involvement in schooling (Glennan, 1998; Hawkins, 1996; Koschmann, 1996; Pea, Tinker, Linn, Means, Bransford, Roschelle, Hsi, Brophy, & Songer, in press).

Changes in the questions being asked. As the technologies themselves have changed, so have our research questions. We began, in the 1970s, by asking questions about whether certain kinds of computer-based activities improved student learning. Studies did find improvements in student scores on tests closely related to material covered in computer-assisted instructional packages (Kulik & Kulik, 1991). But these studies did nothing to help us understand how technologies might, or might not, help to support the kinds of sustained and substantial inquiry and analysis that we all want our children to achieve. Of course these studies did not help us learn about these more complicated issues because they did not ask the more complicated questions that would be required in order to begin to learn about these issues. More specifically, a key problem with these studies is that they did not acknowledge that effective technology use needs to be embedded in a larger process of school change--that understanding the impact of technology integration requires understanding technology use in a social context--and instead tended to treat technology as a discrete and isolated, yet, it was hoped, overwhelmingly powerful input.

Implicit in these initial strands of research was an assumption that schooling is a &black box& (Tally, 1998). Research attempting to answer the question &Does technology improve student learning?& had to eliminate from consideration everything other than the computer itself and the evidence of student learning. Teacher practices, student experiences, pedagogical contexts, and even what was actually being done with the computers - all of these factors were bracketed out. This was done so that the researcher could make powerful, definitive statements about effects - statements unqualified by all the complicated, gritty details of actual schooling.

The problem was that all the studies conducted in this way - and there were hundreds - told educators clearly that specific kinds of technology applications, such as integrated learning systems, could improve students' scores on tests of discrete information and skills, such as spelling, basic mathematics, geographical place names, and so on. But these studies were not able to tell educators very much that helped them address the larger challenge of using technology to support students in developing capacities to think creatively and critically, and to learn to use their minds well and deeply.

It has become clear through past research on the impact of technology on education that technologies by themselves have little scaleable or sustained impact on learning in schools. In order to be effective, innovative and robust technological resources must be used to support systematic changes in educational environments that take into account simultaneous changes in administrative procedures, curriculum, time and space constraints, school-community relationships, and a range of other logistical and social factors (Chang, Honey, Light, Moeller & Ross, 1997; Fisher, Dwyer, & Yocam, 1996; Hawkins, Spielvogel & Panush, 1996; Means, 1994; Sabelli & Dede, 1998; Sandholtz, Ringstaff & Dwyer, 1997).

While the pressure continues to develop answers about how technologies may contribute to student learning, there has been increasing recognition that technology is a crucial player in a more complex process of change that cannot be accomplished by technological fixes alone. As a result, researchers are increasingly asking questions about how technology is integrated into educational settings; how new electronic resources are interpreted and adapted by their users; how best to match technological capacities with students' learning needs; and how technological change can interact with and support changes in many other parts of the educational process, such as assessment, administration, communication, and curriculum development.

Changes in methods. Answering these kinds of questions also requires the expansion or improvement of a whole range of interconnected resources--including technologies, teachers, and social services--that cannot be isolated for study the way a single software program can be. Further, the kinds of outcomes associated with changing and improving the circumstances of teaching and learning are much more holistic than those measured by most standard assessment practices, and they require more sophisticated strategies of the researcher who is attempting to capture and analyze them. To explore how best to use technology in the service of these goals requires looking at technology use in context, and gaining an understanding of how technology use is mediated by factors such as the organization of the classroom, the pedagogical methods of the teacher, and the socio-cultural setting of the school.

Researchers are now emphasizing questions about the intersections of design, learning, school culture and practices, and other factors that shape the impact technologies can have in schools. A key recommendation growing out of the President's Committee of Advisors on Science and Technology is the need for large-scale, longitudinal studies that examine the consequences of technology use in school settings in concert with a broad range of factors.

To illustrate how technologies can be used to support and extend a broad-based program of education change, we would like to use the example of a comprehensive program of reform that has taken place in the Union City New Jersey schools.


The Union City Story

Union City, New Jersey, is located in Hudson County, directly across the Hudson River from Manhattan. With 60,000 residents in 1.4 square miles, it is the most densely populated city in the United States. The predominant ethnic makeup of Union City is Cuban, though recent arrivals from the Caribbean, Central and South America, as well as long-time Italian residents, add to the diversity of the city's population. Of the 9,803 students in the District's eleven schools, 93% are Latino, 68% of whom do not speak English at home. Thirty two percent of the students are enrolled in the District's bilingual/ESL program. The Brookings Institute classified Union City as one of the 92 most impoverished communities in the United States; 27.5% of all children live below the poverty line and 84% receive free or reduced lunches.

The Center for Children and Technology first began to work with the Union City schools in 1992. We were brought into the district by Bell Atlantic to assist with an initiative known locally as Project Explore. Back in 1992, Project Explore represented an innovative home-school networking initiative. It supplied 135 seventh-grade students and 20 teachers with networked computers at home and at school. While Project Explore has been the focus of our research, our work with Union City extends beyond this effort. In 1995, in collaboration with the Union City Board of Education and Bell Atlantic, we were awarded a grant from the National Science Foundation to conduct a project called Union City Online: An Architecture for Networking and Reform. This effort built upon the work of Project Explore, and extended the networking infrastructure to the District's remaining ten schools to help launch a number of other projects to help develop the human infrastructure - the people resources that it takes to make a complex project like this succeed and remain successful overtime. Another core goal of Union City Online was to take a substantial and sustained look at the relationship between networked technology and education reform (Honey, Carrigg Hawkins, 1998).

What is critical in this story is understanding what has happened in Union City during the past 10 years. In 1989, the Union City schools failed in 44 out of the 52 categories that the State of New Jersey uses to determine the effectiveness of their school districts. They were failing in areas such as student attendance, drop-out rates, and scores on standardized tests, and as a result they were facing state takeover. Like many urban districts, Union City was also facing many obstacles to correcting these deficiencies, including language barriers, parents with limited formal education, and students with little incentive to stay in school.

Rather than lose local control of the school district, however, Union City decided to face these challenges head on and drastically reform the entire educational system. The District formulated and implemented a five-year Corrective Action Plan calling for systemic changes in the educational system. Using their own version of a whole-language approach to learning -- which put literacy front and center in their reform efforts -- the District focused on creating a curriculum which would support students in moving away from rote learning and toward the development of thinking, reasoning and collaboration skills. In order to facilitate these goals, the district did a number of things, including the following:

  • Classes were extended in most subject areas to 111-minute periods in the elementary and middle schools, and 80-minute periods in the high schools.
  • In-service training for teachers was increased from 8 hours a year to 40 hours.
  • Buildings were refurbished, windows were replaced, and classrooms and hallways were painted.
  • Individual student desks were replaced by cooperative learning tables.
  • Textbooks for individual students were replaced with class libraries.

Union City chose to implement the reforms first in the elementary classrooms, then add classes year by year until reform reached every grade level. This decision meant that no student schooled in a reformed learning environment entered a new grade only to face the former method of instruction. Furthermore, the District did not have to face on an unmanageable scale the inevitable headaches that arise during renovations and the first years of new curricula. It also meant the District was able to take the lessons learned from each successive implementation and apply them toward easing the transition in subsequent years.

In addition to curriculum reforms, substantial increases in the District's operating budget played a critical role in Union City's efforts. Over the past eight years, the budget for the Union City School District increased from $37.8 million in 1989 to $100 million in 1997 as a direct result of equitable school funding legislation, known in New Jersey as the Quality Education Act (QEA).

Beginning in 1993, Union City also made a deliberate decision to invest substantially in technology resources. They did this largely out of equity considerations, believing that urban students would once again risk falling drastically behind suburban students if they did not have access to state-of-the-art technological resources. The District built fiber backbones in each of its eleven schools. Approximately 85% of the 2,200 instructional computers -- those in classrooms, media centers, and computer labs -- are part of a district-wide network that connects the schools, two public libraries, city hall, and the local daycare center through T-1 lines back to the central office servers. With a ratio of four students per computer, Union City is now one of the most, if not the most, wired urban school district in the United States.

The Center for Children and Technology has been conducting research in relation to both the NSF-funded work in Union City and the Project Explore initiative. Our most recent examination of the impact of the district reforms and the impact of technology on student learning resulted in three important findings:

  • The educational reforms have had a substantial impact on students' standardized-test performance, particularly at the K-8 level, where the reforms have been in place the longest.
  • The Explore students (those with home as well as school access to technology) gained a substantial &leg up& during the first year of the project, scoring significantly better than their district peers in writing and mathematics. This increase, however, is not due to technology alone, but to increased expectations and to the dedication of teachers and administrators in ensuring that this group of students would excel.
  • Writing is the one area where deep and sustained access to technology has made a difference. At the 7th-, 8th-, and 9th-grade levels, Explore students do significantly better than their non-Explore peers on the writing portion of state tests.

Our research suggests that deep and sustained access to technology has the potential to have a positive impact on both students' learning and on the school community's views of their students' capabilities. But our research also suggests that technology in and of itself, in the absence of other components of school reform, would not produce these kinds of changes. We have identified eight key reform strategies integral to the Union City school district's success. These are:

  • Instructional leadership at the building level
  • Effective school improvement teams
  • Extensive professional development in whole-language teaching approaches and cooperative learning
  • A strong emphasis on student creativity and the expression of ideas in multiple formats
  • An emphasis on providing different points of entry into a task for children working at different ability levels
  • A de-emphasis on remediation and an emphasis on learning for all
  • Establishment of classroom libraries and media-rich classroom environments
  • Multi-text approach to learning that includes the integration of technology into instruction.

Union City has taught us a great deal about how research can focus on improving circumstances of learning, and on determining how technology can help make that happen. This requires viewing technology not a solution in isolation, but as a key component in making it possible for schools to address core educational challenges. A consensus is emerging that the larger issue that needs to be addressed across a wide range of iterative, collaborative research projects is gaining an understanding of the qualities of successful technological innovations as they unfold and begin to have an impact within local, district, regional, and national contexts.

As researchers have come to focus on these issues, a number of common characteristics have emerged in the design and methods involved in this type of research.

Key assumptions of this kind of research include:

  • Recognizing that technologies in and of themselves rarely bring about substantial change in teaching and learning.
  • Understanding that the impact of technology on specific aspects of teaching and learning can be usefully understood only in context. Technologies matter only when harnessed for particular ends within the social contexts of schools. We are not suggesting that this eliminates the need for careful formative research with users in experimental or laboratory settings. But it does mean that the research agenda is not completed when a robust application has been developed for use in learning settings. It means that a key phase of research must involve looking at how new technological applications can be integrated into school contexts and how they fit into the complex process of school change.

Methodological features of this kind of research include the following:

  • It is largely process-oriented. The researchers' goal is to understand how innovation occurs in schools, not just what the outcomes correlated with the innovation are.
  • It is oriented toward change rather than doing better within the old framework. Tools and programs that are interesting to study are those that support or act as catalysts for change in the organization of teaching and learning.
  • Teachers and researchers play an active role in interpreting technologies as tools for reforming schools and in supporting and sometimes guiding the change process.
  • It is multidisciplinary, combining elements of different fields, including:
    • anthropological lenses on the culture of schools and classrooms and kids' lives inside and outside them
    • developmental and cognitive psychology lenses on learning
    • sociological lenses on school institutions and school change.

There are also important design elements that this type of research entails:

  • Long-term collaborations with educators. Teachers must be partners and co-constructors of the innovations and of the research process, rather than being viewed as subjects or passive recipients of the innovation.
  • Systemic integration and research on the impact of innovations across multiple levels of the school system. Isolated classroom experiments are being replaced by broad examinations of the roles technological innovations can play in the whole system of schooling, at the classroom, individual school, district, state, and national levels. This type of research includes &test-bed& studies that track long-term school changes that are technology-enhanced.

What we have learned so far? Several broadly supported conclusions have emerged from this type of research.

  • We have begun to learn about the roles that specific technologies can play in helping to reorganize the education workplace.
  • We have become accustomed to defining our strategies and research questions from the point of view of education problems or challenges, rather than beginning from the technologies' capabilities.
  • We have come to appreciate the powerful role technology can play in creating new links between schools and the world outside the schools, connecting individuals, providing resources, and broadening the cultural and political contexts available to students and teachers for exploration and examination.
  • Most importantly, we have learned that research that is focused on change cannot be done at a distance, nor can it proceed from the assumption that the answers lie outside of the school community.

Our work in Union City has taught us a great deal about the value of working in collaboration where all parties are learning together and privileging the knowledge, expertise, and limitations that everyone brings to the task at hand.


References

Chang, H., Honey, M., Light, D., Moeller, B., & Ross, N. (1998). The Union City story: Education reform and technology: Students' performance on standardized tests. New York: Education Development Center/Center for Children & Technology.

Coley, R.J., Cradler, J., & Engel, P.K. (1997). Computers and classrooms: The status of technology in U.S. schools. Princeton, NJ: Educational Testing Service

Bransford, J., Brown, A., Cocking, R. (1999). How People Learn: Brain, Mind, Experience and School. National Research Council. National Academy Press: Washington, D.C. 1999.

Fisher, C., Dwyer, D.C., & Yocam, K. (1996). Education and technology: Reflections on computing in classrooms. U.S.: Jossey-Bass.

Hawkins, J. (1996). Technology in education: Transitions. New York: Education Development Center/Center for Children & Technology.

Hawkins, J., Spielvogel, R., & Panush, E. (1996). National study tour of district technology integration: Summary report. New York: Education Development Center/Center for Children & Technology.

Honey, M., Carrigg, F., Hawkins, J. (1998). Union City Online: An Architecture for Networking and Reform. Learning with technology: The 1998 yearbook of the Association for Supervision and Curriculum Development. Alexandria, VA: ASCD.

Koschmann, T. (1996) (Ed.). Computer-supported collaborative learning: Theory and practice of an emerging paradigm. Mahwah, NJ: Lawrence Erlbaum Associates.

Kulik, C., & Kulik, J.A. (1991). Effectiveness of computer-based instruction: An updated analysis. Computers and human behavior, 7, 75-94.

Means, B. (1994) (Ed.). Technology and education reform. San Francisco: Jossey-Bass.

Pea, R.D., Tinker, R., Linn, M., Means, B., Bransford, J., Roschelle, J., Hsi, S., Brophy, S., & Songer, N., (in press). Toward a learning technologies knowledge network. Educational Technology and Research.

President's Committee of Advisors on Science and Technology, Panel on Educational Technology. (1997). Report to the president on the use of technology to strengthen K-12 education in the United States. Washington, DC: USGPO.

Sabelli, N., & Dede, C. (in press). Integrating educational research and practice: Reconceptualizing the goals and process of research to improve educational practice.

Sandholtz, J.H., Ringstaff, C., Dwyer, D.C. (1997). Teaching with technology: Creating student-centered classrooms. New York: Teachers College Press.

Tally, B. (1998). State of Research on New Media and Children. Talk given at the Center for Media Education's conference on Ensuring a Quality Media Culture for Children in the Digital Age, October 22-23, 1998.


Footnotes

1. An earlier version of this paper was prepared for the Intel Corporation by Katherine McMillan Culp, Jan Hawkins, and Margaret Honey.

2. Margaret Honey is Director of EDC's Center for Children and Technology. Katherine McMillan Culp is the Assistant Director for Research at EDC's Center for Chidlren and Technology, Fred Carrigg is the Executive Director of Academic Programs for the Union City New Jersey public schools and has worked in Union City for 27 years.

  Table of Contents Next 

 
Print this page Printable view Send this page Share this page
Last Modified: 11/09/2006