About Me

Anna-Marie Robertson picture Hi! My name is Anna-Marie Robertson. For the past 10 years I have been working as an online Mathematics teacher. I felt that I should pursue this Masters of Educational Technology because I was doing it every day. I felt I should have the credentials to back up what I was doing. I started this journey in the Spring of 2010 when I met a recruiter at the Virtual School Symposium in Austin, Texas. He convinced me it was time to start. And start I did. My first class was taught totally in the virtual world of Second Life on the BSU island. That opened my eyes to a myriad of possibilities for educational technology. I plan to graduate from this program in December of 2011 with my M.EdTech and a certificate for Online Instructor.

I worked for over 20 years as a paraprofessional to an accountant. This because of my love of numbers and their constancy. It wasn’t until my children began to be teenagers that I realized I wanted to teach. I tried substitute teaching and found my passion in life. I earned my BS in Secondary Education in May of 1996 from Idaho State University. I have only taught at two schools since then. My first school was an alternative school I helped to start in my home district. Three years after starting there, my principal introduced me to a man that was starting an online school for at risk students. I applied and was hired. Technology has always been a big part of my life. I know this degree will greatly enhance what I can offer to my students.

Favorite Quote

Education is not the filling of a pail, but the lighting of a fire. William Butler Yeats

Mechanics of Game-Based Learning

In Jim Gee’s Edutopia video, he mentioned that teaching needs to be “reprofessionalized,” and that there are too many entities that are telling teachers how to teach. These include administrators, text-book publishers, politicians, and even standardized tests. Gee said that when we take away the influence of all of this and let teachers teach the way they know how to do professionally, education will begin to change. You know what? He’s right! I have felt it myself. I have felt the pressure to teach certain content and to do it a certain way simply because that’s how the student will see it on a standardized test that they must pass not only for themselves, but also for my job. It’s sad that I can’t teach the way I know how to do because of some test somewhere. He also mentioned that it’s risky to use social networking in schools right now because of the way schools are designed as “test-prep academies.” When will the political arena of education wake up and realize that teachers are ready to teach and to use modern technology to help students learn to collaborate, build, and think? Stop tying our hands by telling us, “Sorry, you can’t do that in this school because those sites are blocked.” Argh!

After I viewed this video, I was asked, “What reflections or thoughts do you have about this notion of the paradigm shift? How might it change your school or institution?”

Currently I teach for an online school. I am in charge of creating my own curriculum. I can create it the way I see fit within the guidelines of the school. I am anticipating that this paradigm shift away from a “test-prep” type of class to a more collaborative and game-based feel will motivate my students. I am willing to take the chance on this and redesign my course so that students are constantly gaining ground and never losing ground with their grade. I am ready provide rewards to be earned along the way and to let the student see his or her progress in concert with the progress of the group. I am ready to experiment with choice in my course.

Blog Post #5 Media Comparison Studies

During this week’s studies, the topic of media comparison evaluation was discussed. Dr. Perkins provided a document that included the following ideas:

A media comparison study is not a study that compares different types of entertainment media such as news or movies. Instead, the word media is used as the plural of medium which is the mode of transmitting information. The media comparison study focuses on students that have learned a certain topic or material using a specific medium, such as computer based learning, and compares that against student learning that occurs without this medium. In this type of study, the media used is the independent variable of the study and the learning or outcomes are the dependent variable.

Significance is a statistical measure done in such a way as to avoid biases. It is a measure that can lend reliability to the outcomes. If a study is shown to have a significant difference in the outcomes, then it can be said that these findings were not a result of some random chance. However, that said, just because the statistics show no significant difference that doesn’t mean there aren’t differences to be found. It just means that in this particular instance, in this narrow area that was the focus of the analysis, there was no statistical difference found. Also, when a study declares that there is no significant difference between two variables, that in no way, means they are equal or as good as one another. Also, even if a significant difference is shown to exist in the learning outcomes being measured, that doesn’t mean these findings have any practical use.

To do this type of comparison study, one must be careful to ask the right questions in the right way so as to avoid bias and to increase reliability. Most assessments created by untrained (as far as evaluation) teachers will inherently be biased. For large scale evaluations where the stakes are much higher, the assessment tools must be both valid and reliable. One main problem with such data collection that looks only at learning outcomes is that it assesses only surface knowledge. In addition to this problem is the fact that the population being evaluated is individuals who are as dynamic and different as possible. Every individual that approaches an evaluation as a participant comes with their own set of life experiences and prior knowledge making it difficult to assess them with any degree of sameness. This problem of individuality is but one of the many variables linked with evaluating individuals. The variables are as wide as the participants themselves.

Comparison studies of media for use in education can be conducted properly if the right questions are asked and the right data collection tools and methods are used. Instead of asking about the learning outcomes, the focus of a comparison study for media should focus on the media itself. The media is simply a tool for delivering design decisions.

From this discussion about media comparison, I have learned that the medium being used to deliver content in education can be evaluated, but just not evaluated based on the learning outcomes of the participants. This is because the media used is only the tool for delivery of the content and this tool can be utilized in any number of ways depending on the instructional strategies being employed. It makes mores sense to evaluate the tool on its own merits, not on the outcomes of learning.

I really don’t have any experience in this area. I am reading and learning about evaluation as I go along. If I had had some experience in this area, I would add it in. It makes sense to me that it’s not the tool (media) that causes learning, but the way the tool is used. I have been using technology for a long time and it has been my experience that when a new tool is introduced, it actually detracts from and slows down the pace of learning. The tool should only be used when it makes sense to use it. A new tool should never be used in an education setting simply because it’s there. There’s a time and a place for teaching and learning technology, but when it comes to regular classroom learning, I believe a student should be allowed to choose the tool that makes sense to him or her to complete the activity.

July 11, 2011

Blog Entry 4

Rubrics and Scoring Guides. What are they are and how can they be both reliable and valid?

Reliability: Tree's growth rings to measure the age of the tree; Validity: Testing each participant using the same equipment.

According to Carol Boston (2002), A rubric has essentially two uses: To evaluate students and to enhance their learning. Using a rubric or scoring guide to evaluate students helps teachers to be more consistent in scoring and also helps the teacher define exactly what it is that will be evaluated and the quality that is being sought, thus weeding out unnecessary information that could cloud the activity. Rubrics can also provide the student with the exact information that their work will be judge on, thus allowing them to focus on and improve their learning of the task at hand without unwarranted guessing.

Boston (2002) suggests using a metarubric or rubric to judge the quality of rubrics being utilized in a current classroom or course. She says a good metarubric include the following four areas: 1) Content/Coverage, 2) Clarity, 3) Practicality, and 4) Technical Quality/Fairness.  Using this metarubric to judge our own rubrics will help tighten them up and clean out any redundancy or ambiguity. Once the rubric is ready to roll, she suggests using an activity with students to familiarize them with how the rubric will function where they use the rubric themselves to judge other work.

Two other areas of concern when developing rubrics for use in evaluating student performance include validity and reliability. The validity of a rubric or other scoring device is determined by the degree to which the device actually assesses what it was meant to assess. Boston (2002) suggests three types of evidence that are commonly used to examine the validity of an assessment tool. These include content, construct, and criterion.

When looking at the content for validity, ask the question, “Are students being assessed on their knowledge of the subject at hand, or are they being assessed on their ability to interpret and understand the question?” If they are being assessed on both their knowledge and grammar skills, then the categories of the rubric should reflect both.

A student’s reasoning processes are the focus of construct validity. This is a very personal process and is internal to every individual. When looking at the construct, both the correct response and the reasons for that response must be viewed together.

Criterion-related evidence is based on and should reflect the outcomes of a current or future event. If a course is preparing a student to work in a certain field, then the assessment rubric should reflect the same criteria as the future workplace.

To determine which of these three types of evidence should be assessed, one should consider what type of material is being assessed. If the material being assessed is knowledge based, then the content type is warranted. If the material involves reasoning, the the construct type should be utilized. And, finally, if the purpose of the material is show how a student will perform outside of the learning environment, then the criterion-related evidence is warranted. However it is possible that all three could be used.

An assessment tool that is reliable is one that will produce the same scores for each student regardless of who does the scoring, when it’s completed, or when the student was assessed.  Two types of reliability include interrater and intrarater. The rater being the person doing the rating. Interrater reliability refers to the variability between the persons doing the scoring and includes the amount of subjectivity allowed by the rater. Intrarater reliability refers to the outside influences on the rater such as fatigue that might affect the rater’s ability to be reliable in scoring.

Boston (2002) asserts that no matter what is included in the rubric or what is being assessed, it is clear that the scoring device should always be shared with the persons being scored prior to the evaluation so as to allow the students the opportunity to prepare their product in a way that shows they have met the criteria.

I found the three types of evidence commonly used to examine the validity of an assessment tool to be very interesting. I have not been exposed to this before, but it makes sense to choose an instrument depending on if you are assessing knowledge, reasoning, or future outcomes. I think I should like to explore creating each of these types of rubrics a bit further.

References:

Boston, C. (2002).Understanding scoring rubrics: a guide for teachers. College Park, MD: ERIC Clearinghouse on Assessment and Evaluation, University of Maryland.

July 1, 2011

This has been a very frustrating week. There seems to be a lot of time-consuming work to

Picture credit: Sids1: http://www.flickr.com/photos/sids1/3231184356/

Ah, Summer! (Picture credit: Sids1:http://bit.ly/kftPu4)

be done in both of my graduate classes. I thought I would get to start working on my Evaluation final project this week. I have created a time line for it in a Google Spreadsheet, but I have yet to accomplish anything except an introductory email. I am now a week behind on my final project. I don’t like being behind. This week I worked on two large projects for my 505 Evaluation class. I created a fictitious Evaluation Proposal and spent 6 1/2 hours working on that (one whole day). Then I worked to complete an Evaluation Test which was created using Google Docs Presentation. I like this program. I’ve used it several times before. The technology was not a problem for me. I’m not sure what happened except that maybe I’m a bit OCD about my work. This project took me 14 hours to complete spread over three days. That was basically my week. This was a fun project. We were asked to find pictures that served as a metaphor for the definitions of particular evaluation terms and then write the definitions for them in our own words on an additional slide. There were fifteen terms, and 33 slides total. I thought it was a bit much. So basically I accomplished everything for the weekly work in this one graduate class, but nothing else got done. Now I’m behind. I felt these two projects were a bit excessive. I am wishing I had taken this course during the regular school year and not in the summer. I don’t think BSU should offer this course during the summer. The projects are good, but there is not enough physical time in the summer to do them justice.

June 24, 2011 I’ve Chosen My Project

This week has been eventful in my Evaluation class. I have chosen my project. I will be conducting an evaluation to take a look at some online math-based graphing tools. This is something I have needed to do for some time. At the online school where I work, we will be going through the process of re-designing the math courses to fit into the new delivery system. As we do that I would like to incorporate more interactive tools from the web. One of the tools we have needed to take a look at is online graphing tools. I have been collecting them for quite awhile but didn’t really know how to decide which one would best fit our needs. Now I have a way to create an evaluation tool to help us do just that. I am hoping that I can create this tool in such a way that it can be scaled to allow us to apply it to other concepts in the math curriculum and possibly to apply it to other subjects throughout the school. I believe I have learned how to apply an evaluation to something other than just a program or project. Dr. Perkins helped me see and understand how to look at choosing online tools from an evaluative standpoint. I needed that.

I think the most interesting part of this evaluation project as I see it now, will be working with other teachers through my Personal Learning Networks to gather data in the form of web tools and to get their feedback as to what should be included in the rubric that will be used to evaluate the tools.

Going forward, I have a few questions. Will I be able to deploy the surveys and get the needed feedback in the time I have allotted? I am wondering what should be included in the rubric to determine what a “high-quality” online graphing tool looks like? I am also wondering if I should start the EPD now with the limited information I have about my project? Basically it’s only an idea right now. Maybe by next week, It will be a bit more solidified.

EDTECH 505 – Evaluation

I have just begun my last core class for the Boise State University Educational Technology

The ABCs of Evaluation - Timeless Techniques for Program and Project Managers

Master’s Degree Program — 505 Evaluation. I think this will prove to be a very interesting class. I am two weeks into the program and have already completely reading the entire textbook (The ABCs of Evaluation), all nine chapters. There was a lot of information in the book, but I liked the way it was presented. It was very concrete-sequential. I think it will be easy to follow as I begin to choose my project for the course. For my project I am to choose a program (of study) that can be evaluated. I have several in mind and I hope as I go through more steps in this course it will become evident which one I should choose. So far I have on my list a summer math program at Sylvan Learning Center, My neighbor’s homeschool program for her child who has special needs, and a Homeschool Support program. All of these programs involve education in a slightly alternative way. I am looking forward to getting more in depth with this information. I believe that evaluation can help keep a program focused and organized, and I like to be focused and organized.

Of everything I have read about in the textbook so far, I am most intrigued by the Evaluator’s Program Description. It seems everything in the evaluation hinges on this one document, and everything that is needed throughout the evaluation should be contained in this one document. I view it as a type of road map for the evaluation. I am excited to begin preparing one.

I am looking forward to working with an agency to take a look at their program for this class. I am sure I will have a lot of questions as this proceeds, but I am anticipating friendly people and possibly a bit of expertise as I pursue this project. I am wondering if we have to prepare a formal, summative evaluation or if a more informal evaluation will suffice, one with only formative evaluation findings. I get a bit intimidated at the thought of doing something for the first time and by myself that usually takes an expert in the field to perform. I am sure I will make mistakes, but I hope I am granted some leeway to learn from those mistakes.

The Independent Project

I watched a video today that gave me hope for the future. I am a teacher, and I am a teacher of Mathematics. I have watched first-hand what our school systems do to our kids. I have watched the kids power-down when they walk into a school building. I have watched them zone out when they enter a classroom. I have personally tried to make the changes that are needed in our education system. I’ve tried to bring excitement and empowerment into the classroom to no avail. Then I watched this video: http://bit.ly/iGWdXP

The opening quote caught my attention: “If you want to build a ship, don’t drum up the men to gather wood, divide the work and give orders. Instead, teach them to yearn for the vast and endless sea.” -Antoine de Saint-Expuery-

This embodies my philosophy of education — of learning. I would like to find a way to help these students grow their project. I want to see more students become empowered and to feel like they are in charge of their own education. How can I help with this?

Follow

Get every new post delivered to your Inbox.