Technology Plan Evaluation Strategies

Throughout my research to find effective strategies to evaluate Technology Plans, I was interested to find that I could not locate a strategy that my school uses to monitor the effectiveness of their plan. Therefore, I thought that I would conduct my own research to find a strategy that I would use if I were in charge of evaluating the technology plan at my school.

I found a website www.ncwiseowl.org/it/TechPlan0509/Templates/EvalofLEATechPlan2005.doc that asks questions for each section of the technology plan. The evaluation is very simple: put a checkmark next to each question of the item is evident in the plan.  For each section, if there is an objective that is left unchecked then the evaluators and memebers of the technology teacm have to come up with a plan or suggestion to complete that task in the future.

Technology plan evaluation strategies

 

 

To start, it would be helpful if a TUP were published and accessible to the staff.  I am a bit embarrassed that, as a cyber school, a technology dependent school, I was unable to find a document that outlined the school’s technology plan.  My search explains the tardiness of this post and my search is not yet over.  There is a published “Acceptable Use” type policy and general description of how technology is and should be used posted in the employee and student handbooks.  The tech guy in my office and the executive assistant looked at me as if I were asking for something foreign.  I did look at a few official documents filed with our state department of education but they did not contain necessary information.  So I am on a quest with our corporate office.  If there is no such document, I intend to review the rubric with someone in our technology division.

 

So, in my researching, beyond having a document that was available and accessible, maybe as an addition to the employee handbook, an organization’s plan should be able to answer the following questions:

 

• Does the plan/policy/document outline and describe the type, level and availability of training and support?
• Is there a process of analyzing need and providing timely updates of software and hardware?
• Is the plan anticipating new technology through budgeting, support and a desire to maintain reasonable currency?
• Is there some sort of description or directions for outlining or suggesting integrating technology into the curriculum and its use in the classroom?
• Are the classrooms configured that using the tools are practical and not awkward, uncomfortable or cumbersome?
• Is the infrastructure sufficient and secure enough to handle existing connections and tools and expandable enough for future materials?
• How is outdated equipment handled? Is there a responsible way to dispose of old equipment through donation, recycling, or reselling?

 

If a plan can manage to answer these questions, then, in my opinion, it is on its way to being a useful tool for guiding the purchase, use and upkeep of some expensive equipment.  As many of the planning articles suggested, the administrative and general staff must be open to frequently referring and updating the plan.  Once it’s done, it’s just the beginning.

Evaluating Tech Plans

Technology plans are vital for a district to have in place if they want to be successful in the technology field. Students and teachers need to stay up to date with the latest education technology to stay at par with future competition. Technology plans are in place for big districts to be on the same page with technology, and implement a plan to move their school in the technological age. To look at an education technology plan and evaluate it, a person must study the current plan in detail. They need to identify the key components of the plan, decide if the plan properly states how they plan to achieve their technological goals, and form an opinion to see if the technological goals are appropriate and in the best interest of the students. Sun Associates evaluate their plan with these criteria:

I. Level of student engagement A. Students use technology to support higher levels of learning B. Achievement of school expectations for student technology literacy II. Level of teacher engagement A. Teachers use technology to design improved opportunities for learning/assessment III. Availability and accessibility of appropriate resources

• A. A curriculum that fosters meaningful learning
• B. Alignment of the curriculum with learning and assessment strategies that make use of appropriate technology tools
• C. Allocation of technology tools and resources that supports their constructive use in the teaching and learning environment
• D. Equitable access to technology by the school/learning community

IV. Organizational support A. Organizational supports crosscut all aspects of technology integration. Specifically, a strong organizational capacity for integrating and implementing technology to improve teaching and learning would be indicated by:

• • Transformations in school leadership to support technology and change
  • Staff-development programs that support capacity-building for improving teaching and learning
  • Use of technology as an information management tool by teachers and administrators

V. Community involvement A. The school community supports, through participation in and contributions to, the school’s integration of technology in its teaching and learning environment

http://www.sun-associates.com/eval/indicators.html

Technology Evaluation Strategy

I found this strategy that Iowa uses…

• Is your plan a real tool for powerful teaching and learning?
• Does it have the support of the entire school community?
• Are the goals you established broad and comprehensive?
• Do they truly address the learning needs of your school?
• Have you looked at the “whole picture” in your district? Are the technology plan goals closely related to the broader goals of the district?
• Does your technology plan identify how technology enhances curriculum?
• Have you developed a plan for student assessment of technology use?
• Have you specifically addressed the issue of staff development? Is your staff development plan for technology comprehensive?
• Have you accomplished the goals you set for this year? If not, why not? If so, what is the next step?

…and I realized how impossible it is to ever stay on top of technology.  I understand that usage plans are important, but it is apparent in my situation why this plan is never going to be truly attainable.  I understand requirements of usage plans, but it is easy to see why under-served populations get taken advantage of due to lack of personnel or qualified personnel.  I’ve condensed the evaluation to the following headings due to insanity:

1. Does your plan address the greatest needs in the school and are the needs measurable?
2. Does your plan include staff development and is there an adequate measurement for that?
3. How far does the plan think ahead?
4. What flexibility is there for revision in your plan?

It seems that in this day and age a rationale or a drawn out list has to exist for everything we do.  Teachers have to attach a state standard, objective, activity, procedural method, and alternative assessment for every lesson.  I’m a firm believer in progress, but how much is too much?  At what point do we step back and say, “I think we overthought this one.”  Too much devotion to lesser significant concepts like the support of the “entire” school community (which will never happen) versus what’s most important, “Is this the best option we have to help children succeed,” is taking away from real quality education.

TUP Evaluation Strategies

Districts across the country have spent thousands and thousands of dollars on technology for schools, but have they reached their technology goals? Many have not and there are several different reasons why. According to SEIR*TEC, these are the reasons why:

 

1. Not enough district professionals with evaluation expertise, particularly true in rural districts.
2. Traditional measures of student learning seldom measure the benefits students gain through technology use.
3. Policymakers and funding agencies often have unrealistic expectations about the improvements in learning that will result from technology initiatives – they want to see immediate changes when it takes years to fully incorporate technology effectively.

http://www.seirtec.org/P2P.html

 

The following are sample questions associated with effectively evaluating a technology use plan:

1. What technology personnel are in place to support both instructional and administrative technology utilization by students, teachers and staff?
2. What federal, state, and local budgets are currently available for funding technology needs?
3. Is the network environment secure for student use?
4. Are suitable policies in place to guide appropriate actions?
5. Does the current bandwidth of your school LANS and district WAN adequately support the usage of the networks?
6. Is sufficient hardware available in your schools for ongoing use of computers for instruction?

http://209.85.173.132/searchq=cache:6_R05cI_5SIJ:www.ncwiseowl.org/It/TechPlan0509/Templates/EvalofLEATechPlan2005.doc+Technology+plan+evaluation+strategies&hl=en&ct=clnk&cd=1&gl=us&client=firefox-a

 

Overall, an effective TUP evaluation will not only evaluate the technology use within a school, but how the technology positively benefits student learning as well as teacher learning. If the teachers aren’t learning new strategies, uses, and programs, then students aren’t learning either. Evaluations need to measure all that is incorporated in the TUP on a regular basis, giving suggestions on improvement.

 

Tech Use Planning

Throughout the process of studying Tech Use Plans, I’ve noticed, at least in my own institution, that there are a lot of good ideas.  I have also noticed that there is a tendency to be very general about the objectives.  And, I have also noticed that implementation is usually a work in progress or it has not started.  Tech Plans are like the rest of the formal-education world–lots of good ideas and intentions but not necessarily getting things done.  In my conversations with the technology director she was very open to suggestions for the Tech Plan.  As teachers we need to be proactive in learning what our institutional tech plans involve and giving input.  It may not always be actively sought out but it will likely be appreciated.

I feel that the best ways to know the effectiveness of technology being used is to hear from the people who are using it. Teachers and students will be the ones to notice if the way technology is being used is helping or not. A survey of students and what they think about specific technologies in use in their classes can give close up perspective of its affects on the students’ learning and interest in the class.

Evaluating Technology Use

This week I came across an article entitled “Core Elements of a Technology Plan” on the website for the Idaho Commission for Libraries.  This article was clear and concise and helped me to understand what a tech plan really is and what the most important components are. The first four were Clear Goals/Realistic Strategy, Professional Development, Needs Assessment, and Budget. These elements described what should be included in a Technology Use Plan. The fifth, and final, element was Evaluation. The article included some questions that should be answered in an evaluation process. They were

1. How frequently the tech plan will be evaluated
2. The person(s) responsible for updating the plan
3. How progress toward the goals and objectives will be measured or monitored
4. If and how goals were met
5. Any unexpected outcomes
6. Any new needs that emerged
7. Any goals that are no longer relevant.

I think that, regardless of a which particular strategy is used, if these questions are answered, the evaluation is on the right track.

The organizations of the Sun Associates and SIER*TEC include specific strategies including surveys, observations, and focus group interviews. Using these strategies are a more authentic way of evaluating technology use, specifically in a classroom, because they include user feedback and do not just rely on numbers and statistics.

Evaluating a Technology Plan

<!– /* Font Definitions */ @font-face {font-family:Wingdings; panose-1:5 0 0 0 0 0 0 0 0 0; mso-font-charset:2; mso-generic-font-family:auto; mso-font-pitch:variable; mso-font-signature:0 268435456 0 0 -2147483648 0;} @font-face {font-family:”Cambria Math”; panose-1:2 4 5 3 5 4 6 3 2 4; mso-font-charset:0; mso-generic-font-family:roman; mso-font-pitch:variable; mso-font-signature:-1610611985 1107304683 0 0 159 0;} @font-face {font-family:Calibri; panose-1:2 15 5 2 2 2 4 3 2 4; mso-font-charset:0; mso-generic-font-family:swiss; mso-font-pitch:variable; mso-font-signature:-1610611985 1073750139 0 0 159 0;} /* Style Definitions */ p.MsoNormal, li.MsoNormal, div.MsoNormal {mso-style-unhide:no; mso-style-qformat:yes; mso-style-parent:”"; margin-top:0in; margin-right:0in; margin-bottom:10.0pt; margin-left:0in; line-height:115%; mso-pagination:widow-orphan; font-size:12.0pt; font-family:”Arial”,”sans-serif”; mso-fareast-font-family:Calibri; mso-fareast-theme-font:minor-latin;} p.MsoNoSpacing, li.MsoNoSpacing, div.MsoNoSpacing {mso-style-priority:1; mso-style-unhide:no; mso-style-qformat:yes; mso-style-parent:”"; margin:0in; margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:12.0pt; font-family:”Arial”,”sans-serif”; mso-fareast-font-family:Calibri; mso-fareast-theme-font:minor-latin;} .MsoChpDefault {mso-style-type:export-only; mso-default-props:yes; font-size:12.0pt; mso-ansi-font-size:12.0pt; mso-bidi-font-size:12.0pt; mso-ascii-font-family:Arial; mso-fareast-font-family:Calibri; mso-fareast-theme-font:minor-latin; mso-hansi-font-family:Arial; mso-bidi-font-family:Arial;} .MsoPapDefault {mso-style-type:export-only; margin-bottom:10.0pt; line-height:115%;} @page Section1 {size:8.5in 11.0in; margin:1.0in 1.0in 1.0in 1.0in; mso-header-margin:.5in; mso-footer-margin:.5in; mso-paper-source:0;} div.Section1 {page:Section1;} /* List Definitions */ @list l0 {mso-list-id:122509172; mso-list-type:hybrid; mso-list-template-ids:-1364184660 67698689 67698691 67698693 67698689 67698691 67698693 67698689 67698691 67698693;} @list l0:level1 {mso-level-number-format:bullet; mso-level-text:; mso-level-tab-stop:none; mso-level-number-position:left; margin-left:.25in; text-indent:-.25in; font-family:Symbol;} @list l1 {mso-list-id:321127287; mso-list-type:hybrid; mso-list-template-ids:460466710 67698689 67698691 67698693 67698689 67698691 67698693 67698689 67698691 67698693;} @list l1:level1 {mso-level-number-format:bullet; mso-level-text:; mso-level-tab-stop:none; mso-level-number-position:left; text-indent:-.25in; font-family:Symbol;} ol {margin-bottom:0in;} ul {margin-bottom:0in;} –>

/* Style Definitions */
table.MsoNormalTable
{mso-style-name:”Table Normal”;
mso-tstyle-rowband-size:0;
mso-tstyle-colband-size:0;
mso-style-noshow:yes;
mso-style-priority:99;
mso-style-qformat:yes;
mso-style-parent:”";
mso-padding-alt:0in 5.4pt 0in 5.4pt;
mso-para-margin-top:0in;
mso-para-margin-right:0in;
mso-para-margin-bottom:10.0pt;
mso-para-margin-left:0in;
line-height:115%;
mso-pagination:widow-orphan;
font-size:12.0pt;
font-family:”Arial”,”sans-serif”;}

Evaluating the Technology Plan

Phil Fite

Technology performance is a continuous process that adapts to the school’s changing situations and includes ongoing evaluation. Continuous evaluation also facilitates making changes if aspects of the plan are not working.

Evaluating the technology plan can be conducted by various means.

· Simple observations, both negative and positive, made by students, parents and teachers.

· Interviews and informal meetings with both instructors and students can be used to discuss what both groups have learned from using the technology.

· A simple written survey can assist in measuring how much of the plan has met its original objectives and outcomes.

Here are some questions that should be dealt with when evaluating your technology plan:

· How and when will you assess the impact your technology plan has on learner performance?

· When and how often will the technology plan be reviewed?

· How will teachers demonstrate they are carrying out of the TUP?

· How will the level of proficiency, gained by students, teachers, and staff, be assessed?

· What is a key indicator of success for each part of the plan?

· How will you evaluate these decisions to accommodate for changes that are the result of new information and technologies?

· How will you create and allows changes in the technology plan?

TUP techniques

The good Technology Use Plan starts with these elements:

• Establish clear goals and realistic policies for using technology to improve education.
• Have a professional development policy to ensure that staff understands how to use these new technologies to improve education or technology services.
• Include an assessment of the technology services, hardware, and software that will be needed to improve education.
• There must be a sufficient budget and plan to acquire and support all aspects of the plan: the hardware, software, professional development, and other services that will be needed.
• There needs to be an evaluation method that enables the school to monitor progress toward the specified goals.
• Be prepared to make any modifications, to the plan, in response to new developments and opportunities as they arise.
  1. What innovations in technology have emerged that you can use to improve education for your school?
  2. How do you identify useful new technologies (e.g., attend conferences, reading journals, and networking with other Educational Technologist)?

Evaluation Strategies-A Continuous Process

As I searched the Internet on the topic of technology plan evaluation strategies I came across the North Central Regional Technology in Education Consortium (which has been disbanded) posting, Evaluating the Implementation of Your Technology Plan. I have copied and pasted their philosophy and questions below. I feel the evaluation of the TUP needs to be an on going process in order for the plan to be effective.  This goes along with my belief that the TUP needs to be viewed as a living document and evaluating the implementation is an important part of the process.

North Central Regional Technology in Education Consortium

Evaluating the Implementation of Your Technology Plan

Technology implementation is a continuous process that adapts to the organization’s changing circumstances and includes ongoing evaluation. Effective evaluation will force planners to rethink and adapt objectives, priorities, and strategies as implementation proceeds. Continuous evaluation also facilitates making changes if aspects of the plan are not working.

Evaluating the implementation of a technology plan can be conducted by various means. Simple observations, both negative and positive, that have been made by students and teachers using the technology are the most helpful. Interviews and informal meetings with both instructors and students can draw out the lessons that both groups have learned from using the technology. A simple written survey can assist in measuring the extent to which the plan has met its original objectives and expected outcomes. The following questions should be addressed when planning the evaluation of the implementation of your technology plan:

• How and when will you evaluate the impact your technology plan implementation has on student performance?
• Who will be responsible for collecting ongoing data to assess the effectiveness of the plan and its implementation?
• What windows of opportunity exist for reviewing the technology plan? (For example, the plan might be reviewed during curriculum review cycles.)
• How will accountability for implementation be assessed?
• How will you assess the level of technological proficiency gained by students, teachers, and staff?
• How will you use technology to evaluate teaching and learning?
• What is the key indicator of success for each component of the plan?
• How will you analyze the effectiveness of disbursement decisions in light of implementation priorities?
• How will you analyze implementation decisions to accommodate for changes as a result of new information and technologies?
• What organizational mechanism will you create that allows changes in the implementation of the technology plan and in the plan itself?