The Master and the Student.

 

Image generated using DALL-E 3
via Bing Image Creator

I've been involved with (or I've been sitting in on) a lot of meetings at the moment about the future of assessment in a post AI world.

How can you ensure the students are actually learning when they can use AI to complete any part of any assessment for which they have advanced warning? If your goal is to test their own, actual knowledge, the options available are limited. They grow even more limited if you don't want every assessment item to be some kind of invigilated exam.

So what can we do to make sure the graduates of our universities actually meet the graduate attributes? In a world where it's almost impossible to survive without hustling, how can we ensure the student who has graduated from your university actually learned anything, apart from how to hustle their degree?

To be honest, I'm not sure you can. The more I hear and read about this conundrum, the more I think about the implications of this Brave New World in which we find ourselves, the more I think our current education system can't continue to work like this.

What's the solution? 

Well, some clever bunny somewhere is going to come up with a better option for assessing large numbers of students with small numbers of teachers and markers, but I think the only "true" solution that will let us ensure the graduates actually know their stuff is to go back to the old "Masters' Apprentices" way of doing things. Small numbers of students attached to teacher who is able to regularly talk to them and watch them work, and glean from those conversations and observations if they know what they are doing.

This is going to fly in the face of budget conscious higher education institutions who want to have hundreds of "bums on seats" while only having scant handfuls of academic faculty members on the payroll.

But I honestly think this is the way we need to go to ensure we aren't completely deskilling ourselves and undermining our future. The Capitalist way of delivering education has just "Capitalist-ed" itself into a corner, and it was barely fit for purpose before it became possible to outsource your every thought and sentence to a machine.

How can you ensure your students actually know their stuff before they go out into the real world? By talking to them. Really, deeply, meaningfully talking to them. Have a real conversation with them about what they know and how they would apply that knowledge. Give them puzzles to solve and be with them, as they solve them – watching how they go about it, asking them about their processes.

This is, I feel, the best way to know if you are happy to release that student out in the world to be a professional in your field.

Tough and Competent, or: Ad Astra per Aspera

For some time, I had a badly drawn image of the Apollo 1 Command Module on my desk. I recently replaced it when I needed a badge design to upskill myself with the library's badge maker. Instead of a poorly drawn image, I know have a home-made badge using the official mission insignia.

I've been slightly obsessed with the Apollo 1 mission ever since I first read about it. You may be more familiar with Apollo 11 (landed on the moon) and Apollo 13 (had a movie with Tom Hanks), and maybe Apollo 17 (last manned mission to the moon), but Apollo 1 was the first manned Apollo mission. Then they had a bunch of unmanned missions and tests before having the second manned mission with Apollo 7.

Why such a long gap between manned missions? Because Apollo 1 was a death trap. All three astronauts were killed in a systems test before they even got close to launch.

The Command Module of Apollo 1 (which I had badly sketched to keep on my desk) was riddled with problems. It had faulty wiring, flammable materials and a badly designed hatch that couldn't be easily opened from the inside if it was under pressure. As it turns out, the hatch couldn't be easily opened from the outside either, if the inside was on fire. 

They pumped the thing with oxygen for the "plugs-out" test, then an exposed bit of wire overheated and the interior of the module went up in flames. Then, thanks to the poor hatch design, they couldn't get out of the module to escape the fire.

They also couldn't call for help, because the comms system was faulty and kept cutting out. The patchy recording of the crew's last words included the phrase "How are we going to get to the Moon if we can't talk between two or three buildings?"

Why did I keep a drawing of this death trap on my desk? Why do I now have the mission patch as a badge?

Because every single one of the faults that lead to the deaths of Gus Grissom, Ed White and Roger B. Chaffee were noticed before the test. They knew each and every one of these problems (and many more besides) existed, individually, prior to trapping these three men in a tiny fire bomb.

No one took the time to fix them, because there were budgets and deadlines to meet. And no one made sure you could get out of the module if something went wrong, because everyone just assumed nothing would go wrong.

This disaster, and the knowledge that it was entirely preventable if people just took the time to fix the little isolated problems when they noticed them, lead to Gene Kranz, the Chief Flight Director of Mission Control, making the following speech:

“Space flight will never tolerate carelessness, incapacity and neglect. Somewhere, somehow we screwed up. It could have been a design in build or in test, but whatever it was, we should have caught it. We were too gung-ho about the schedule, and we locked out all of the problems we saw each day in our work.

“Every element of the program was in trouble and so were we. The simulators were not working, Mission Control was behind in virtually every area, and the flight and test procedures changed daily. Nothing we did had any shelf life. Not one of us stood up and said, 'Damnit. Stop.'

“I don’t know what the Thompson Committee will find as the cause, but I know what I find. We are the cause. We were not ready. We did not do our job. We were rolling the dice, hoping that things would come together by launch day when, in our hearts, we knew it would take a miracle. We were pushing the schedule and betting that the Cape would slip before we did.

“From this day forward, Flight Control will be known by two words, 'tough’ and 'competent.' 

"Tough means we will forever be accountable for what we do or what we fail to do. We will never again compromise our responsibilities. Every time we walk into Mission Control, we will know what we stand for.

“Competent means we will never take anything for granted. We will never be found short in our knowledge and in our skills. Mission Control will be perfect."

 "Tough and Competent."

Tough means we will not compromise when it comes to our responsibilities, competent means we will not fall short in our knowledge and skills.

I have this reminder of Apollo 1 floating around to remind me that I should try to take care of a problem when I see it. I should make sure that, when I do a job, I should make a point of doing a good job. 

I know nothing I do will ever be perfect. I know I'm not capable of doing perfect work – and I'm especially not capable of single handedly changing an organisational culture that is far from "tough and competent"... But...

But insofar as it's within my ability to do so, I should strive to be tough and competent. Insofar as it's in my ability to do so, I should encourage the people I work with and work for to be tough and competent.

It's easy to forget this when you have deadlines and budgets to meet, and when some of the problems you notice aren't yours to fix, and some of the jobs you have to do are outside of your current abilities.

Sure, no one is going to die in a fireball if I let something slide, but I expect whoever didn't get around to checking they'd removed the nylon from inside the Command Module before the plugs-out test didn't expect a tragic outcome either.

No harm comes from doing a good job.