Self-assessment as PhD tool: some thoughts on process and pedagogy

Well this log hasn't been updated for some time. A part of me is disappointed while another is content with the reality of these past six months. This being the case, I decided to to collect my thoughts on self-assessment as a learning tool in doctoral studies. The following text is therefore highly subjective and somewhat personal in nature.

I became cognisant of monthly self-assessment as a meaningful tool during our doctoral program's start your studies seminar (Väitöskirja käyntiin) in 2014. This monthly review of progress was recommended by my good friend and longtime model for being a historian, Jaakko Tahkokallio. In his seminar presentation Jaakko talked about his process, the good and difficult and the bad. Ever so passingly he mentioned that at some point he undertook monthly evaluations to see, whether thing were progressing as planned. This was a revelation!

I'm a strong believer in developing process agilely. I prefer to understand the meta in any job I do and through that reflect where I need help, development, and tools for better command of my work. As far as I understand this relates to my highly rhythmic temperament. I do routine well and having understood this, can influence my life by intentionally and systematically targeting my routines. I didn't learn this during my studies but with help from significant others, when I decided upon a 30 something health overhaul. Still the same personal, psychological tools apply.

First, understanding emotions related to work, professional identity and professional community has been really hard. Finns (especially CIS men) tend to have built their identity around work and professionalism. This applies extremely well to me. Yet we don't always do emotions that well. Success, failure, interaction and status issues all apply on an emotional level. I found the linked column "Want to lose weight? Train the brain, not the body" by Laurel Mellin extremely helpful in this. As the title shows, this didn't happen in my academic training. Still that's what I've largely used it for. I also participated in a pilot program started by my pension fund (MELA) with TJS Opintokeskus (the info is only in Finnish). This put me in touch with professional work consultants used to operating with academically trained professionals and possibly more importantly with my peers, with whom I've been able to process practically everything.

Peer support is the second essential component I wish to stress herein. Various informal and formal networks have sprung up among PhD students in Helsinki and statewide during my studies. Still, these take time to develop and many can become unintentionally closed groups for friends. The support structures for unfunded doctoral students are still most fragile and many students can and do stay alone with their insecurities, unrealistically high expectations, fears and such. Participating in support structures takes active effort and so far projects like the one linked above, demand having a grant in the first place.

This leads me back to self-assessment. Why did I decide to take part? As I've noted before in an earlier post, it became a perceived need during a monthly review of progress. Mine follows this pattern:

• How did the last month go? Emotions, successes, failures, whatever comes to mind.
• PhD work progress in detail. What got done, what didn't, why?
• Other studies, projects, events and networks: what's happening, what's taking time?
• Content analysis: ideas, problems and possibilities related directly to my thesis.
• Funding issues: I feel this needs to be addressed regularly in order to cope with the monumental feelings of deficiency and frustration involved.
• To do: what's coming up during the next six months or so?

A colleague of mine has developed this further and uses a purpose built IT tool to manage todo-workload. Long to mid term stuff moves into short term lists during self-assessment and so forth. I just use a text file, so whatever works, works.

During my short stint in London, I was supervised in this process: monthly meetings with the professor and rundown on everything I had done and was planning to do. It felt a little heavy handed but this might be related to my own progress and process. Such close guidance will work for some, especially early on. It doesn't do away with personal development as that is a real development goal in learning to become a researcher. Still we could do more in recognising it as such and defending the necessary pedagogical resources involved. As it stands, I feel this issue is insufficiently addressed currently but as always YMMV.

Having said that, things are progressing and they must. The crunch in funding and temporal resources of doctoral education is putting a lot of pressure on university teaching staff and recent cutbacks have seemingly done nothing to help them manage their own responsibilities as support personnel and structures have been slashed and administration restructured. Academia is running up the hill, but is there cliff at end? I hope not.

Personal development in research education is necessary but it should not mean solitary development. Some academic survivors of yore may have learnt inopportune or even downright problematic survival mechanisms but that doesn't mean anyone, anywhere should. I hope recent programs, systemic pedagogical processes and peer help mechanisms survive and flourish. Should they do so, is entirely up to us and that is why at this late hour (in my PhD studies that is) I continue to participate.

Thinking text, research, publishing and the wørd

Over the past few years I've come to the conclusion that MS Word should be dumped, entirely and for good. After embarking on a quest to learn scientific online publishing for Tekniikan Waiheita the Finnish quarterly for the history of technology, these issues have become quite acute. This blog post is therefore merely an early reflection on a process that I assume will take years of effort. Also I need to note that much of this might be quite pedestrian for you technical types. If so, please keep in mind that humanists, (some) social scientists and (many) higher arts types will never code, ever. Thus we need to bridge this technological gap and find open, easy tools for the benefit of all.

First, let's get into the why of things. How did I end up hating the one program I'm really good at?

I've been the layout (con) artist, web editor and technical do-it-yourself everyman for the Finnish Society for Maritime History for many years now. In this role the issues arising in putting out newsletters and publishing conference materials have become very familiar. I became this person due to my passing familiarity with InDesign and fearlessness in the face of html. I'm still rubbish in the latter and can just about muddle through the first. But as academic societies never actually have money for proper professionals, I was it. Luckily for me (and not so much for them) I have friends who do these thing for a living.

Turning various, ill-formed word-documents into html and/or pdfs is a pain. I was slowly getting decent at it though. Experience kept coming in fits and bouts. In 2010 I took part in three concurrent book projects, as an article writer for the Nautica Fennica and the Museum of Kymenlaakso yearbook and a member of the editorial team for the Maritime Centre Vellamo book. This triarchy got me thinking about the ways we write articles and publish them as I was privy to the various parts of doing books at the same time. But this, it turns out, was the old world.

Humanists in particular tend to think in terms of a published text and for this the pdf is ok. That's what it's meant for after all. Telling printers what to do. With the onset of digital first methods, mobile devices and all that, print media has become increasingly obsolete. Academics still tend think with the A4 in mind. We largely still approach our research in a form that is ill suited to digital first dissemination and this – to me – is becoming untenable.

Proper digital tools for a digital world

About three years ago I came up with a thought experiment to illustrate this issue. I introduced to Tim Cook as a historian of technology. He's on his way out from wherever we happen to be in. I have just the elevator ride (let's pretend it's a skyscraper) to answer his question: what kind of a history would you do for Apple? Go!

My answer to this was astonishing: NOT A BOOK! But I love books...

The book is wrong for Apple. A technology company selling mobile devices needs a mobile, technological history. And so I started to dream about the history of tomorrow.

This digitally disseminated history should be easily readable on all mobile devices. This meant html or xml or something such as an end platform. This naturally led to hypertextuality. Sources and the very environment around that story should be presented dynamically, when needed. With this in mind, I started to think about scalability. Not everyone wants to read 500 pages of thick description of systems, networks and people amidst a changing world. Academics often need depth to get into the issues and an executive summary will never do. Have you read the most recent IPCC report? If not, I can assure that it's quite exhaustive. We need to be mindful of this issue though. One way or another the detailed research has to end up compressed into a summary.

Now, most academics I know dislike writing such summaries. There's never enough space to explain things properly. But what if the process of summarizing was built into our research reporting and publishing tools? What if that online Apple history was dynamically expandable from the executive summary – a glorified index in fact – into the most detailed history of the company and all that went into it? What if you could choose when to access that information? The summary expanding into chapter descriptions, those into actual chapters and so on and so forth. Online sources would be presented alongside the text when needed, pictures would appear as needed. Metadata would be embedded into the text and one might rearrange the whole story chronologically, expand just the bits about Woz, or just stick to the curated format the historian saw best for the story.

To a historian this sounds almost magical, but as anyone who's written a book with MS Word or any equivalent also a lot of work after the fact. And herein lies the issue. The tools we use to think, to write our articles, dissertations and the like fail us. Looking at a skeuomorphic A4 on my laptop stops me from seeing that story digitally delivered. Manually inserting all the metadata links demands an armada of technical editors almost no one has resources for. The bloated word editor kills our dreams and through it ties us to the past.

Dreams of tomorrow

Just today I started the process of learning about XML in academic publishing. Thanks to The Federation of Finnish Learned Societies I've now embarked on a trip to someday get our journal digitally natively published, readable by machines and consequently far more easily disseminated by all. Obviously there are massive hurdles strewn across this path but luckily I'm not alone here. No one is. I've included a few links in the end on this.

The big issue is this though. If we plan to publish our research digitally we need to pay attention to the format. Not because it should override the function of academic enterprise, but because it can help us be better at it. I'm sure the book will exist for a long time, and there's much good to say about physical media. I'm not trying to do away with these here. Instead I'm trying to navigate the turbulent waters of impact and significance.

If we have easy to use tools for doing our research, analysis, writing up and publishing, we can actually get things done. Using journal editors time to troubleshoot anonymisation issues related to MS Office metadata creation is frankly a waste. Muddling through e-mail archives for an up to date version of the article one was working on with a colleague after the peer review corrections came, is a waste. Helping academics do their work means getting technology out of their way. Obviously this will never get fully done. We'll always have to learn to use new tools and things will always change in surprising ways.

Do we then want to rely on Silicon Valley start ups or corporate behemoths on our capacity to work? As much good as both may have done, I have my reservations. My recent discussions with colleagues much better versed in such issues, have led me to strongly back open standards for academic work both in higher learning and research. We truly need openness in research data management and dissemination to ensure transparency and quality of research. For a lot of these tools this already is the case, but for others not so much.

What I want to do for me and my research, is bring materials together while I'm conducting analysis and writing things, to have all my quotes in in the original Finnish, Swedish, Russian, English or German always alongside my translations and notes. I want to create searchable metadata on times, people and concepts that I can at a whim connect to any publication bound argument, whether it be an article, a blog post or a my dissertation. To keep a track of my thinking while I move onwards. MS Word will not do. Atlas TI will not do. A new tool for a new time is needed instead. It must be open, for scientists by scientists.

Onwards and upwards, into ruin and desolation

As a historian I feel this issue acutely. Much of the materials, the source I use are still safely stored in archive vaults across the world. Current digital versions are typically barely passable if they exist at all. Finnish archives currently fail to work with Refworks and the like, which led me to not use such otherwise great tools. I felt that the work needed would be better put to a creating my own database. This pained me greatly, but the work had to come first. U.S. based citation cultures and pseudo-paper online journals play into this.

Having discussed these and related issues with easily some hundred people, colleagues on all rungs of the academic world, I've met much despair and hope. Many want to stay with the familiar while others whole-heartedly embrace dreams as I have in this short introduction. Little exchange fits in between and we are all worse for it. Bibliometrics and impact factors aren't going away however. And the only way we, the experts in our respective fields, can have any say in what is measured and how, is to stand up and confront tomorrow.

With these fighting words, I wish to make it known that I at least ant to participate in creating meaningful ways for the academia of tomorrow, to bring my meagre lot as a humanist scholar of technology to this churning maelstrom of a discussion. And maybe, just maybe be part of the development of academic humanities into the digital age.

Further reading on digital academic publishing

Public Knowledge Project kehittää Open Journal Systems -palvelua
Tieteellisten Seurojen Valtuuston Avoimen Julkaisun palvelut
XML Journal Article Tag Suite
USA National Information Standards Organization