Tag Archives: map

Takeaways from NWEA MAP Training

My partner and I attended two NWEA MAP Foundation workshops over the weekend. This was our first year at a MAP partner school, and at the beginning the amount of data seemed overwhelming. Two testing seasons later we felt we had a better handle on it and were ready to learn more, which is why we jumped at the opportunity to attend the session – especially since the registration costs were covered by the State Department. We joined about twenty other teachers from Niamey, Freetown, Monrovia, and Abuja at the International Community School of Abidjan where presenter Terri Howard led us through the Stepping Stones to Using Data and Climbing the Data Ladder.

Here were our takeaways:

  • How do you know when to differentiate? Double-digit figures in your class’ standard deviation (in the Teacher by RIT report) means you should differentiate your class. The Class by RIT report also gives useful data, especially when you click on the subject row headings (i.e. Math) to show the goal strands within, where students will be grouped according to RIT band. Aim for 2-4 groupings.
  • Is the test a reflection of where they are? Standard Error (also in the Teacher by RIT) report indicates the reliability of your test; a 3 is normal and the lower, the better. If you have students scoring 4.8 or above you should strongly consider retesting, as this indicates they did things like quickly guessing through their questions.
  • How do you get students on board? In addition to the usual goal-setting, schedule student-led conferences after the testing season (this can coincide with your normal parent-teacher conferences) where students can explain their results. This motivates them to try their best – no one likes to explain showing less-than-expected or negative growth. Having a visual reminder, such as a sticky note with their target score, also helps keep them focused according to NWEA researchers.
  • How do you get parents on board? Education is the key here. Hold a parent workshop – at AISB we plan to do one just prior to the start of the testing season – where you explain RIT, growth, and especially DesCartes. Emphasizing DesCartes and avoiding discussion of percentiles helps keep parents focused on how MAP tests measure what students are ready to learn, not mastery. Use sample data at this session, as using their kids’ real data will distract them from learning about how to interpret the results.
  • How do you distribute results to teachers? You can tweak your CRF to create non-existent classes, like one for all ESL students across grades, or one for all HS students, or one combining two small grade levels to see what they’d look like as a combined class (a common situation in small West African schools). In the Spring, you might give access to one grade’s data to their teachers for the following school year so that those teachers can begin to differentiate from the very start. Finally, you can submit Data Repair Requests with updated CRFs if you want to fix anything after the testing season.

On an unrelated note, regional trainings like this are always nice because they bring together a group of schools in a similar situation, and this is especially true in West Africa. The Ed Tech community always talks about creating a global personal learning network (PLN), but it was only by going to a face-to-face training that I could meet other educators who knew what it was like to teach in a school without internet, or one that had been shut down because of a coup, or one that had graduating classes of just half a dozen teachers. People outside of the region just don’t have the experience in the unique set of challenges we work with every day. And I’d argue that face-to-face conversation has a depth and flow that electronic communication misses. One superintendent has decided to eschew email completely:

So we left the conference having had good, old-fashioned conversations and notes, not Hangouts and hashtags, but still ready to use data to inform our instruction.

Cut MAP Testing Setup by 66%

Update on this post: NWEA contacted me to let me know that they offer expanded support times for international schools – 3am GMT Monday to 1am GMT Saturday. There’s also a web form at http://www.nwea.org/about-nwea/contact-us if you prefer not to call. Finally, they’ve got support forums at http://community.nwea.org/ – it looks like a mixture between pedagogical and technical questions. Props to NWEA for being proactive and responsive.

I’ve had a steep learning curve this year: in addition to my responsibility over all things IT, I have three preps as a tech teacher, work with three elementary classes, and am the MAP testing coordinator. When you wear a lot of hats you look for efficiencies, and one of the most frustrating things about MAP testing was how obtuse and labor-intensive it was.

Cutting Per-Test Setup Time by 66% Through Scripting

The previous MAP testing coordinator told me to leave 15 minutes between sessions to set up tests. I quickly discerned that this was to do with the labor-intensive nature of MAP testing. Each station needed to be logged in and have the correct test selected. Between waiting for the login to occur and selecting the right MAP test for our Mac Minis, it would take about a minute per workstation. Luckily, our school had shelled out for an Apple Remote Desktop license. This enabled me to do a batch login for all of our testing stations simultaneously by executing the following Unix command, which just tells the workstation to simulate the keystrokes of an actual physical login:
osascript <<EndOfMyScript
tell application "System Events"
keystroke return
delay 1.5
keystroke "username"
delay 1.5
keystroke return
delay 1.5
keystroke "password"
delay 1.5
keystroke return
end tell
EndOfMyScript

Make sure you replace the credentials with those from your environment, run the commands as root, and save the above as a Template in ARD so you don’t need to type/copy-paste the commands each time:

Screen Shot 2013-10-19 at 11.23.39 PM
Click to zoom.

I also set up a user account that executed an Automator action at login to connect to the NTE share, launch TestTaker and send the login keystrokes to TestTaker:

automator
Click to zoom.

*note that the script to log into the TestTaker problem – the last link in the chain of events in the Automator scrip – is a bit different, as per the code below. The 60-second delay at the beginning gives TestTaker time to launch before inputting proctor credentials:
osascript <<EndOfMyScript
tell application "System Events"
delay 60
keystroke "mapproctorusername"
delay 1.5
keystroke tab
delay 1.5
keystroke "mapproctorpassword"
delay 1.5
keystroke return
end tell
EndOfMyScript

When all was said and done, I simply needed to select the class, student and test after sending the login commands from ARD. It cut our setup time from 15 minutes to 5 minutes.

On the Magic of Resumable Downloads

NWEA is so considerate to offer the MAP for Primary Grades audio files as a 1GB download that is non-resumable because it’s behind a secure login. Under absolutely ideal conditions, that download would complete on our school’s 2Mbit line in just over an hour. Except that our connection doesn’t get 2MBits/sec to NWEA’s servers, so the effective download rate is about 30KB/s. And guess what happens if the download gets interrupted, like it does EVERY TIME you try to download it?

You get to start all over again, from the beginning. That’s what.

Solution to downloading huge MAP for Primary Grades audio files if you’re on a slow, unreliable internet connection in West Africa? Get your parents to download it for you, then have them post it to your personal web hosting service. Then, download it using your favorite resumable HTTP downloader like wget or uget.

Or, if you have access to a diplomatic pouch, order the DVD. Ours arrived three days into our testing season, and the hand-written title etched in Sharpie ink made it clear that not many people go with that option.

Beating Obtuse Software Design

Here’s the funny thing about the NWEA TestTaker client (the software that students use to take MAP tests) for Mac: it’s not actually a native Mac program. If you open Activity Monitor while TestTaker is running, you’ll see a process for wine. No, not the delicious stress reliever we all know and love – quite the opposite. Wine is an open-source compatibility layer that allows you to run Windows programs on Linux systems without actually installing Windows. Apparently, NWEA decided it would be better (easier?) to take the Windows TestTaker and hack it to work with Macs by running it in Wine.

Now, I love open-source – my primary machine at home is a custom-built PC running Linux Mint Olivia. But a lot of open-source is not exactly end-user friendly, and it’s only free if your time is worthless (ie you’ll spend a lot of time setting it up). I’m actually shocked that TestTaker runs as well as it does, but it’s still prone to obtuse errors. If a test session closes down unexpectedly and you get a student set up on another station, there’s a good chance that TestTaker will give you a “resource busy” error until you manually log into the server and terminate the old client’s connection from there. Sometimes you’ll just get helpful errors like the one below that inexplicably disappear when you repeat the exact same action (usually trying to start a student’s test):

Errors like this make it easy to troubleshoot MAP Test problems.

 

And most delightfully, sometimes you’ll launch the Mac client, only to be left with a black screen that you can only get out of by force-quitting the application

Getting Support

You know what’s really convenient? Calling the States at 8pm because you’re on GMT time and the support line is on the US Pacific Coast. That’s how you change admin passwords – there’s no web interface. I didn’t even bother to try, so confirmation emails still go to the old email of the guy I replaced (luckily he’s a stand-up guy and isn’t going to sell our students’ results on for money). I wish NWEA had support forums so they could crowdsource solutions, rather than relying on a less-than-helpful knowledge base. NWEA does have an international support line with expanded hours, as well as a web form to get in touch with people. There are also forums at community.nwea.org. I’ve asked them if they’ve considered doing a web chat support option – I met plenty of people at AISA for whom calling/Skyping is not an attractive option because of the poor telecom infrastructure in Africa.

A Note About Doing MAP Tests on a WiFi Connection

Don’t do it. Ever. It won’t work.