Major Activities including:
Quality Review | Project Management | DLESE Annual Report | Project Review | Strategic Planning | Partnership Building | DPO Web site
Major Findings | Training and Outreach | Publications and Products | Contributions
The purpose of the DLESE Project Office (DPO) is to lead community discussions surrounding the growth of the DLESE digital collection, coordinate the development of the services that will support the creation of exemplary resources and their use in enhancing geoscience education, and lead the community-building activities of the Core Service (CS) groups.
Major activities of the past year months focused on completing the quality review and beginning to implement the community recommendations; implementing the first phase of project management; leading the CS groups in writing an integrated annual report; performing an internal project review and preparing for the NSF Visiting Committee review; performing a SWOT analysis in anticipation of developing a new strategic plan; developing a project office web site; conducting a pilot study of peer-review; and carrying out a range of partnership building activities. In addition, we are developing a project office web site. Each of these activities is discussed below.
Much of the work completed in the quality review and developing recommendations for Steering Committee consideration was done under award EAR- 0434212 and discussed in the most recent annual report for that project. As Executive Director, I have been directing the efforts of the CS groups to develop and initiate an implementation plan. In late December and January the CS groups reviewed the quality recommendations and made adjustments as needed to develop a consensus to move forward. At our January 20-21 All PI meeting, we focused exclusively on moving from the recommendations to an implementation plan. About 75% of the effort required to implement the quality recommendations falls upon the DLESE Program Center (DPC), as it controls 80% of the resources in DLESE. The process of developing the implementation plan was hampered by two major issues: a lack of commitment to modify the current work plans of the key group and a lack of knowledge of what work will be postponed or not completed by working on the quality recommendations. Thus, there was no agreement on how to move forward on any activities beyond scientific review (to be carried out by the DPO) after two days of discussion. The DPO has developed a first draft of an implementation plan for all components of the quality plan including scheduling of activities but it awaits refinement once the project management plan is complete and the Management Council sets new priorities.
We moved forward on three activities that required minimal DPC contributions in the short term as we negotiate the project plan and new priorities. These activities are a pilot study of scientific review, developing a process and criteria for providing a DLESE Service Award and Best Resource Award, and developing mechanisms for improving recognition and access to DLESE collections.
The pilot study of scientific review using 120 resources began in February. Community Service PIs Ed Geary and Bryan Aivazian contributed the DWEL review system as a framework for a new DLESE science review system. Evaluation PIs Susan Buhr and Lecia Barker, as well as LuAnn Dahlman from Data Services and Kim Kastens from Collections Services all contributed to developing the science review criteria and rubric, as well. The pilot study of the review questions and process included gathering reviews from our CS team as well as participants in a workshop on peer review at the 2005 Annual Meeting. These two groups provided information on the usefulness of our review questions in assessing the scientific accuracy of a resource and determining actions needed for accessioning into the new DLESE reviewed collection. We have selected a suite of resources focusing on earthquakes, oceanography, water resources, and climate. The resources span all audiences and all years of DLESE operation. As we move forward for more expanded reviews, we hope this initial cross section of the library collection will provide indicators of whether older resources or those developed for particular audiences are likely to be more faulty in their science content.
In July, NSF decided that the upcoming funding year would be the final year for the current suite of DLESE projects and requested new work plans for 2005. In light of this change and in consultation with NSF, we received basic funding to develop and implement full-scale review. We began developing pedagogical review criteria based on lessons learned in the science review. In addition, we have contracted with Isovera (formerly e-guana.net) for development of a review-tracking tool to support full-scale review. The review tool will be modeled after a tool in use by AAAS and the BEN library and will be in full operation by November 2005. We have also assembled names of over 100 reviewers and have begun recruiting Associate Editors to oversee reviews of small sets of resources in their discipline.
Acting on recommendations of the community, the DPO organized a working group led by Sharon Locke to develop criteria for giving awards to persons contributing the best resource to DLESE and providing outstanding service to the community. This activity is an important step towards fulfilling the early community desire for providing academic/professional recognition for outstanding contributions to DLESE.
The working group has completed its work and made recommendations to the SC for action in February 2005. The SC disbanded in July 2005 and no further action was taken, however, the report contains a strong base of work to move forward quickly on this issue, should DLESE be continued in a new form after 2006.
A third effort was organized to resolve a long-standing issue of appropriate access and recognition of collections within DLESE. For several years, collection builders have expressed dissatisfaction with the ways that their collections are accessed in the browse and search structures. A group of collection builders and community members led by Anne Egger worked with the DPC to identify simple to complex solutions to a number of problems. The simplest solution is acceptable to all as a starting point, but is not sufficient. A final report with recommendations was delivered to the DPO and DPC for action in August. The DPC is refining its work plans for the upcoming year and will share them after October 1.
In January 2005, John Pawlak was hired as project manager for the DLESE project office. He has 20 years experience in technical project management for AT&T Laboratories. His duties include working with the five CS to develop mechanisms for planning and management of all project resources (time, people and money). The process of developing a project plan was foreign to everyone in the project and requires a great deal of trust. In January 2005, the only document guiding the work plans of the different teams was the Integrated Work Plan. It was developed in spring 2003 as a high level guide to the work of each Core Service. Only minor updates had been made since then, the last being in spring 2004. In the All PI meeting in January, the PIs indicated the process of developing the IWP had been quite beneficial for building a sense of team, but once completed there had been little done to keep it current and updated as needs and plans changed. Sharing of detailed information about staffing, work plans and schedules, project goals and outcomes has not been the culture in DLESE.
After a difficult start in which PIs had to be educated about the need and value of project management, John was able to successfully gather sufficient information from Data, Evaluation, Community and Collections services. The level of data gathered from the DLESE Program Center lacked the detail to be useful for planning. Given that the DPC controls the bulk of the personnel and resources, the overall project plan is not useful without their full cooperation. We are hoping that the DPC will be more trusting of the process in the future, as effective project management depends on data at a level sufficient to be able to make decisions about priorities based on resources available. Currently, the lack of a functional project management plan is upholding further implementation of the quality recommendations as well as most every other decision facing the library. The Management Council had agreed to talk about this issue again once NSF has shared its expectations for DLESE based on the Visiting Committee review. With the NSF decision to terminate DLESE next year, these discussions are no longer relevant. As such John Pawlak left the project in August 2005.
For the first time, the DLESE Core Services wrote a joint annual report covering their work for the period of 2003-2004. The process also included the first sharing of financial information among the groups. The entire process was fruitful in providing each CS an updated understanding of the major activities of the other groups. Due to the part-time effort of most of the PIs, they have little time to keep up with the activities of the other CS groups. In discussions at the December 2004 MC meeting, we found that without exception no MC member had a working understanding of the goals and major activities of any other CS. Writing the report stimulated some discussion of areas of overlap and possible collaboration. This report writing effort prompted groups to more completely document their work in anticipation of the March 2005 NSF Visiting Committee review.
John Pawlak completed an internal project review for the DPO upon joining the project. It was used as a baseline for the work of the DPO as it identifies areas that need additional project planning and management to move DLESE from an experiment to a production quality facility. John had performed these types of reviews inside AT&T on a routine basis. This review was used by the DPO to prepare for the NSF Visiting Committee. We developed a number of strategies for addressing the deficiencies in the organizational structure, process management, quality management and project management.
One of the strongest concerns raised by the Visiting Committee was over the state of the DLESE Strategic Plan. The plan was developed in 2001 and has not been reviewed or revised since then. Formally, the SC controls the Strategic Plan. However, the Management Council embarked on an exercise to begin discussions of a new strategic plan in May. We completed a list of Strengths and Weaknesses of DLESE Ð the SW of the SWOT analysis. Then, the MC decided to wait until after NSF has shared the Visiting Committee report and NSFÕs future expectations for DLESE. Independently, the DPO continued the effort on our own. We identified opportunities and threats facing DLESE and then developed a matrix showing how strengths and weaknesses might be used to overcome threats and take advantage of opportunities. The document serves as a guide in helping us set priorities. It may also serve as a starting point for a future planning effort by the community.
Below is a brief list of opportunities awaiting action by DLESE. The list of hypothetical or potential opportunities is much greater. Many have been on the back burner as the partner matured or because the current DLESE structure prevents action. Yet, the potential is there because of the degree of success that DLESE has had in developing core infrastructure for a dynamic library and organizing the geoscience education community to provide support for community needs. These opportunities are listed to encourage the community to pursue them within a reincarnated DLESE or some other project.
DLESE lacks the capability to formalize project-wide partnerships or commit to pursuing a new strategic activity due to a lack of centralized decision-making and financial controls. Currently, each CS initiates minor partnerships that advance their own goals, but there is no mechanism for formalizing partnerships that will impact the entire project. Attempts to develop a partnership strategy to create major partnerships benefiting the DLESE community overall have failed due to a lack of trust and willingness to work as a team among some PIs.
Merlot Digital Library (www.merlot.org) is a model for developing a broad base of membership and alliances for sustainability. Having 16 State University systems (e.g all public 4 yr institutions in CA, MN, NC, FL, etc.) as sponsoring members, it provides access to peer-reviewed resources for undergraduate education in science, arts and humanities. Merlot seeks DLESE as a partner to provide the Earth science collection. DLESE would immediately get access to the faculty and students in these states.
COSEE leaders have indicated strong interest in DLESE providing technical infrastructure to support their educational efforts. The opportunities range from providing cataloging and search capabilities of regional collections developed by the COSEE sites to providing collaboratory or intellectual commons type technologies to allow online communities to create new resources and knowledge. In the upcoming year, the DPO will work with COSEE to develop concept maps for ocean-earth interactions. This effort will provide a mechanism to explore how these concept maps might be developed for the entire Earth system and serve as a mechanism for more effective browsing or searching of the collections. DLESE can provide COSEE a technological framework that further integrates the efforts of the COSEE sites and increases their effectiveness.
National Science Teachers Association provides digital links (SciLinks) to reference and instructional resources that supplement the articles published in their journals. DLESE could play a role in providing the Earth science links for the high school and middle school journals. With over 50,000 members, this is a strategic partnership that would give DLESE access to teachers filling the pipeline with future scientists.
PLATO Learning (http://www.plato.com) is providing online courses for students in over 200 school districts across the US. PLATO is interested in a partnership in which DLESE provides the digital resources for new courses they are developing in Earth science for middle and high school students. This partnership would provide income to DLESE to offset the existing costs of cataloging and reviewing resources.
Outreach to Research Facilities led by DLESE Data ServicesÕ to transfer the data and knowledge of the research community to the classroom is the most promising development in DLESE over the past 18 months. Two workshops in the past 13 months have resulted in 20 teams of scientists, data providers, curriculum developers and educators creating cutting edge instructional materials that are rich in data analysis and inquiry. IRIS, UNAVCO, LEAD, Palmer LTER, EarthScope, CUAHSI, SIOExplorer, ERESE a just a few of the participants in 2005. Each team will develop a classroom investigation with the support of Data Services over the next year.
Broader Impacts of GEO Research can be supported by DLESEÕs well-established technical and social infrastructure. Although much of this infrastructure is untapped/unknown to the scientific research community, the potential of this infrastructure and the DLESE community to support stronger Criterion 2 efforts is significant. DLESE could play a key role in promoting community education about Criterion 2 and the mechanisms to satisfy it. Support structures to assist researchers in their Criterion 2 efforts might include: a referral / brokering service for obtaining evaluation, curriculum development, professional development or other educational skills; expansion of the Data Services outreach effort; cataloging services; and technical or communication services. DLESE can also play a critical role in developing an integrated network of organizations and institutions supporting Criterion 2 efforts and geoscience education efforts.
Federation of Earth Science Information Providers (ESIP) has over 70 Earth Science members including NASA, USGS and NOAA, as well as Unidata, GLOBE, Terrafly and DLESE. The federation has approached DLESE to co-develop an educational portal to resources the federation members provide for educational audiences. The DPO will pursue this opportunity over the coming year. The portal would provide access to a diversified array of data, analysis and visualization tools, and educational resources that integrate the two. The portal would give DLESE exposure to a broad swath of the research community while providing ESIP with exposure to educational audiences currently out of their reach.
NSDL and other Cyber Infrastructure Programs represent untold opportunities for technical and educational collaborations. DLESE has long been recognized as a leader in the NSDL, being one of the most comprehensive digital library projects. This has made DLESE the library of choice for testing new capabilities such as automatic assignment of metadata to resources, development of personalization services, and integration of curriculum design tools. Recently, ITR and other cyber infrastructure projects have made significant advances that might transfer well to the digital library effort including especially in portal technology and ontologies. Partnerships under discussion with ESIP and JPL will build upon an existing partnership with GEON to create an earth science ontology that also incorporates education concepts and relations. Having an ontology useful to many other earth science cyber infrastructure projects such as CHRONOS and CUAHSI will allow greater integration of these projects over the next decade.
Creating a DLESE Membership is critical for helping DLESE identify and prioritize its efforts to meet the needs of the GEO community. While DLESE is open to all, it is challenged to identify who it serves. Membership with clear benefits and obligations is key to defining strategic efforts, establishing ownership of DLESEÕs success, and responsibility for its performance.
Members as Ambassadors is a concept overdue in DLESE. Currently, we recruit users one by one through presentations at conferences and in Ambassador workshops. Turning each member and their institution into a recruitment agent has the potential to significantly increase and awareness of DLESE at relatively low cost. Members could include professional societies such as the Association of Educators of Teachers of Science an organization that includes higher education faculty preparing the next generation of teachers, federal agencies such as NSF and Department of Education, to school districts, colleges and universities.
The DPO has established a new domain name for the project office site (dlese-project.org) and has begun developing it. The concept behind the DPO site is to separate the information about the operations of DLESE from the services and resources provided to the educator/user of DLESE. Presently, the DLESE.org site contains more information for the developers of DLESE than for educators. By separating operations from the portal for educators, we can better serve the needs of our users. Ideally, the dlese.org site would be a one-stop shop for educators to learn about the latest scientific discoveries, gather instructional resources, participate in professional development, gather assessment information and instruments etc. To do this, we must separate the operations content from the education content to free up valuable space on the home page and to make browsing menus more efficient.
A single link from the dlese.org site will provide access to the resources that the CS, partners and collection builders would need to conduct their work. This is not a simple task and will considerable time to outline a structure to meet the needs of the operations side of DLESE. Once a proto-type is developed, discussions can begin about redesign of the dlese.org site, per the quality plan recommendations.
The DLESE community and Core Services have accomplished much in the past five years and is poised to make considerable impact on the teaching and learning about the Earth system. However, the major finding of the past nine months has been that DLESE requires a significant reorganization to achieve this impact in a cost-effective and timely manner. Reorganization of its governance and management structure will allow the library to move forward as an integrated project with a single set of goals, plans, measures of success, quality assurances, and resource management. Currently, we operate as 16 independent PIs at 11 institutions, each with their own work plan as outlined in the individual proposals submitted 2-5 years ago. The organizational structure promotes competition within the CS rather than trust and collaboration. The SC and ED have responsibilities but no authority. All authority and decision making within DLESE is done on an individual basis by the PIs. We have documented the specific weaknesses of the current organizational structure and developed potential strategies for short and long term adjustments that could make operation of DLESE more efficient and effective.
No specific training or professional development activities were proposed or conducted by the DPO. However, the DPO office conducted two workshops for high school teachers to learn about DLESE. We also worked with the CS to operate booths at national meetings. Indirectly training was provided as the DPO staff and CS staff have learned about project management, project planning, SWOT analysis and other organizational management approaches through the ongoing activities. Working with an experienced project manager, the ED also learned the fine details of establishing a well managed, production quality facility. As part of the peer review process, we are involving K-12 teachers as reviewers and hopefully recruiting them as regular users of the library.
Additionally, attendance at several digital library conferences broadened the PIs understanding of future opportunities in partnerships with non-digital library projects to more fully meet the needs of the DLESE community.
We have created the DPO Web site at http://www.dlese-project.org to share information about the operations of DLESE. In addition, we have a peer review system under development. This system will be transferable and modifiable to work with a range of metadata formats, review criteria, and resources, thereby creating opportunities to provide a service to other libraries or facilities in the future.
To the principal discipline The project has provided opportunities to learn about project management, quality assurance programs such as ISO 9000, Total Quality Management, Malcolm Baldridge and many others. If implemented in DLESE, they will provide a model for other NSDL and cyber infrastructure projects. A number of other organizations are interested in the implementation of the quality recommendations, especially those related to resource quality and expanding the earth system vocabulary. The products from the overall quality review and resource review efforts are transferable to other Earth science cyber infrastructure projects.
Other disciplines The lessons learned in project management and transforming a project from a research experiment to a production quality facility to support the research of the community are valuable and transferable to other disciplines as well as the primary discipline.
Development of human resources The scientific and pedagogic review of resources ongoing in DLESE will have impact on the quality of teaching in K-16 classrooms across the country. Students will be engaged with learning experiences that contain current and accurate science content and use effective pedagogy.
Physical, institutional or informational resources that form the infrastructure for research and education Implementing the quality recommendations has been identified as a critical step in getting other research groups to partner and contribute to DLESE. DLESE is positioned to be a critical node in the geoscience education efforts of a number of research groups including the Earth Science Information Providers in the Earth Science Foundation.
Other aspects of public welfare beyond science and engineering The educational value DLESE offers to the public and K-16 teachers has broad impact on the public understanding of our world and human impacts on our world. DLESE resources and services provide current information critical for good policy-making and day-to-day living.
DLESE Visiting Committee Report
DLESE Peer Review Opportunities
DLESE Quality Review