A conversation with Brett Bligh and Kyungmee Lee about the new journal in town: Studies in Technology Enhanced Learning
As I maintain a curated open access list of EdTech journals, I was curious when, a few weeks ago, my Twitter feed blew up with the news of a brand new journal. So instead of just adding the journal to the list, I thought that I could make a little blog post about it. I e-mailed the editors of Studies in Technology Enhanced Learning, Brett Bligh and Kyungmee Lee and they agreed to talk to me a bit about their new journal. Firstly, I wanted to get some information about their general rationale for launching STEL; what is the niche they want to fill? But secondly –in line with what the journal list aims to do– I also wanted to gather practical information about what makes the journal unique in terms of editorial practices, to help potential authors decide if their manuscript is a good fit. What follows is a Q&A-style conversation about the journal but also about TEL as a research field and the state of scholarly publishing.
Q: So, of course, I’d like to learn about your motivations to start a new journal. The framing I use for my list is: “what would a potential author would want to know about a journal?”. I try to figure out what makes a given journal unique. (On the other hand, I’m not sure if this really is how it works. Maybe authors are guided by different things, like Impact Factor, as long as the manuscript is generally in scope.) So what do you think about journal identity and how would you position STEL in the broader landscape?
A: The question of journal identity is actually really interesting: Do journals have distinct identities? Because I think people want them to have. But the way it is currently going, with commercial publishers in particular, is that people just have articles and just keep throwing them everywhere until someone accepts them. Indeed, some companies are directly supporting ‘format free publishing’ so articles can be sent to any journal with that publisher, and articles cascading between different journals where they get rejected. So I think it is becoming very clear that many journals don’t have the strength of identity that some people imagine them to have. On the other hand, there is a very small number of journals that have a very distinct identity and, actually, some of them might be a bit off-putting to some authors. For example, I quite like “Postdigital Science and Education” but what they seem to be communicating is “we aren’t really doing technology-enhanced learning at all anymore” and I’m not sure all authors would be comfortable to go that far. So with STEL, it was really about having a distinct identity, the idea of a “scholarly conversation”, but firmly rooted in TEL.
In our institution, we have a very large doctoral program, currently about 120 PhD students. So we’re constantly dealing with people who are new to the TEL field. One thing that frequently comes up, is that they find the research discipline very hard to grasp. Constantly having to explain what TEL is, I think, sparked our interest in having a journal that might actually explore that a little bit.
Also, at our institution, we have a series of events (see here) where we invite authors to discuss their papers with us. Through this, we talk to a lot of authors. And we encounter a pretty common phenomenon where authors will say “Oh, I really wanted to say X and Y, but I couldn’t write it in that way… so I had to leave it out of the article”. This is happening too often to just be a coincidence! I think that many people working in the field actually have their own identity, have their own ideas, and want to be critical, but the publishing ecosystem is not supporting them to do that. So I think there is a whole host of unmet needs and that was the inspiration for starting the journal.
When you write this blog about STEL, I really hope that you can make the connection to other open access journals. So if we think about some examples like IRRODL, Open Praxis, RLT, etc., we have quite a few successful open access journals in the field already. And some of them started with an intention that is very similar to ours, to have a distinct vision and to be critical. But as they become more successful, receiving more manuscripts and having a higher IF, there seems to be pull towards adopting more conventional practices and identities. Apparently, there are strong temptations to go with the flow of working in a more normal way. Maybe we will be blessed to have this dilemmas one day! At the same time, other journals may have kept this mindset a bit more but they receive much less attention, making it harder to pursue their idea of a lively scholarly conversation. We want to avoid that fate of course. But it is a difficult balancing act.
Q: It’s interesting when you look at journals from an outsider’s perspective, you often see that journal names are relatively similar in content, that is, they do not explicitly exclude anything of substance. And then you look at aim and scope and, here too, there is not that much difference. But when you look at the articles themselves, you do notice some real themes and you get a feeling that there are indeed some distinct flavors. But with the incentives pointing toward “let’s try to get highly impactful publications”, I wonder if many people instead just start at the venue with the highest IF and start moving down, slowly dissolving the idea of distinct journal identities.
A: I think it is important to understand that this is all tacit knowledge, it is happening quietly. A lot of it is down to the personal preferences of the editors. So, that’s how we end up with a situation where the aims and scope statements of several journals are very similar but when we look down the articles, you do find some differences. It’s not discussed and it is not easy for an outsider to understand. Then there are experienced people who know what an editor likes and may be able to use that knowledge to get their papers published. But many people just seem to do what you said, they start high [in terms of IF] and just cascade lower until acceptance.
Now, if you read our editorial (see here), I think we are really, really open about what we’re trying to do here. So if we see articles coming in and we are not sure about their fit, we aim to always go back to that original motivation and ask ourselves: “Does this move the agenda forward?”.
Q: Let’s talk about this agenda, the idea of a “scholarly conversation”. In the editorial you mention three distinct, shall we say, components to this vision. Let’s talk about these one at a time and also discuss how these translate into editorial practice. So I guess the question is: “how would an author know that he/she is addressing one or more of these components in his manuscript?”. The first one is critically integrated. What do you mean by that?
A: I don’t think that there is any work in the field that cannot be critically integrated. But papers have to be written in a way that conveys critical integration. So what this means is, you need to give authors freedom. Remember how I mentioned earlier that authors sometimes feel that cannot write what they would like? Some of it simply has to do with word limits. I mean, BJET recently went *up* to 6000 words (from 4000 words). But that kind of limit can still be really challenging, because if you want to be critically integrated, you need space to acknowledge your motivations; what previous work you have read and how that influences your work; did you want to build on it or challenge it? So I think that many projects in TEL actually have clear motivations; but you have to actually write it down! At the end of your paper, you also have to be really clear about how you are contributing and if you’ve achieved your objectives. Usually, the answer is “partially” and acknowledging and dealing with that also takes space. Let’s take C&E’s guide for authors for example. It is openly negative about that. You can read that on their website, it’s no secret [“Avoid extensive citations and discussion of published literature”, see here]. And I think this is seeping into the scholarly practices, where yes, of course, you do have to have a few references at the start but it’s become a bit of a religious statement of faith. What it shows is, you spent some time on a search engine.
There are people in TEL that are very critical, but they are a minority in the field. An example that would come to mind would be someone like Neil Selwyn, a very critical voice. And I really like his stuff. But I feel that this kind of writing is not very evident in ordinary TEL research. A lot of people, after reading his work, may reach the conclusion that he is a smart voice and a keen observer, but how does it relate to their own work? So what we want to see with STEL is that authors are critical of their own work and the field, more than what is the custom in most TEL research. I think we should aim for much more reflective practice throughout more ordinary TEL research and that is what we want to foster in the journal.
Q: Great answer, thank you. Then let’s talk about the second component of a scholarly conversation, the idea of self-awareness. What do you mean by that and how will STEL reflect that goal?
A: Again, I think the field is not valuing self-awareness. I think that we are encouraged to talk as if we were a service industry. Who we service may vary, but, over time, it has brought us to a point where we as a field are not always taken seriously. I think it is absolutely fine to say: “This work that I am doing is not immediately impacting practice or policy”. In fact, it is entirely legitimate to say, when policymakers give us a problem to solve, “that’s actually not the problem”. Many stakeholders would love to have a “silver bullet” and we seem to get caught up in this way of thinking too, despite probably knowing better. What we should be saying is: “Look, technology doesn’t work like that. Technology gets integrated into local practice and works differently in different places”. Although we know this as researchers, I think we need to reflect on that much more explicitly in the research that we publish.
So what are we looking for specifically? We want to focus on “what does the conversation in the field of TEL look like? Can we, as a field, challenge that agenda? How can we move that forward?” It is not realistic that one paper by itself can do that. But maybe a scholarly conversation over the long term can achieve this. It is simply not plausible that every research should yield immediate impact on practice and we should be honest about that.
And finally, self-awareness can also mean that we explicitly examine the field. When I look at other disciplines, it seems common for them to actually analyze their own field, having a critical analysis of what their field is doing. For example, critical literature reviews; why do we not have more of this in TEL? I wonder if it is because we know that the result may be quite grim. If we conduct a review of the use of theory in TEL research, take the most prominent journals over the last five years, I think we know what we are going to find and it’s going to be disheartening. I think we need the freedom and have to be willing to develop ourselves as a field. We would like STEL do be a place to do so over time. There are not going to be any shortcuts.
Q: That sounds great. Then let’s talk about the connected aspect of the scholarly conversation. In your editorial you contrast this with “gap-spotting research”. Can you tell me what you mean by this and how this notion will be implemented into STEL?
A: Yes, so please understand that there is some overlap between these concepts and when we talk about these aspects there might be an element of repetition to it. Gap-spotting research is when somebody looks at the body of literature and says “Somebody has done this, somebody has done that, somebody has done the other, I’ll therefore do this slightly other thing”. I think we give a few examples in the editorial, like slightly different demographics, slightly different settings, slightly newer version of technology. I think the cumulative result of that is a very static research agenda, where you never end up challenging the basic underpinning assumptions. Fundamental to this is that you accept that the results –even that term is problematic– of a previous paper are a closed book. In practice that means that all you have to do is read the conclusion section of every paper. You don’t have to really critically engage with that work, like “how would have I interpreted this or that differently?”
I think it is understandable that when you are writing a paper, there will be a few papers that influence you the most. That’s nearly always the case. You will be aware of other papers but when you think about it, there were only maybe five that really influenced you deep down. So how can we represent this in a paper? We should make this clear, show how we build on them, explicitly critique them and frame our research in light of them. Conducting slight variations of previous work is not that! This means that we cannot merely say “somebody has done this, somebody has done that”. In the literature review we might need a full paragraph to interpret just one paper that has been influential on us and make explicit how it relates to our work.
Going beyond single papers in STEL, we are particularly interested in developing this kind of conversation over time. That means actually having papers that respond to work that was previously published. So we want to develop a scholarly conversation over time. Of course, you can’t see this in the first issue yet as it is, well… the first issue! Also, when an author reads all that I have now said about our ideas, it might come across as intimidating, “Do I really have to do all that?”. And the answer is “no”. It’s the journal itself that we hope will generate this conversation. We merely ask that papers contribute to this in a way that allows a scholarly conversation to develop over time. Each paper might only form a small part of that.
Q: So let’s talk about special issues. How do they figure within you concept for STEL?
A: The power of the special issue is that so many papers with one theme will be published at the same time. Individual critical papers necessarily have a limited impact but when we publish a special issue with many papers aiming to move the conversation forward, this is a different thing entirely. So for example, with the inaugural issue, there are now 30 people talking about the same thing, the importance and role of theory in TEL research. That is powerful. And probably why the journal has had the publicity that it has had.
That is why we really want to lean into special issues. Specifically, we want the special issue editors to have a lot of freedom in terms of content, format, and length, while encouraging them to specifically keep in mind the idea of “scholarly conversation”.
Also, we want to see this as an ongoing conversation. So, even if people didn’t make the deadline for the special issue, commentaries can still be submitted at a later point, extending the conversation beyond the publication of that special issue. And its also likely that special issue themes will be revisited periodically.
You know there’s also a practical concern here. Some journals have large funding behind them. They can utilize large resources in managing the journal. We, however, have no financial support. In some ways that’s a luxury, because we have complete independence. But that is also why we really depend on the editors of special issues, to distribute the workload and make it manageable. Currently, we have special issues planned (see here) for the next few years and we are not yet planning to open up for submissions outside of special issues (though it may well happen one day). So, for now, we are planning on moving forward with the help of special issue editors because otherwise we can’t afford the workload associated with regular submissions.
Q: There is this idea that the process of submitting and publishing a paper has a rather antagonistic side to it. There are plenty of memes about evil reviewer 2 and there even is a Facebook group with around 40k members, where the focus is on discussing bad reviewing experiences. There seems to be a certain negativity to the process, although ideally, it should be positive and constructive. When I read your editorial in the inaugural special issue, it sounded very much like that positive and constructive process. Is that something you aim for specifically with STEL?
A: I think you interpreted that well. It is important and we did aim for this but it is not trivial. In fact, it should not be underestimated how difficult this can be. Our intention in supporting this process was to ask people to really engage with each other’s work. So how did we do that? First of all, we asked authors of the special issue to review papers of other authors of the special issue, to try to foster conversation. Secondly, we tried to make the reviewers very different people. So for every paper, at least one of the reviewers was a PhD student, because we wanted somebody to come at this work with genuinely fresh eyes. And it’s really the authors’ jobs to write a paper that explains, not just reports. Another thing we did, was to encourage reviewers to write up part of their reviews as commentaries afterwards. This is something we want to explore over time. But I don’t think this will be easy, because the field doesn’t have a tradition of working like that. But we were very happy that several reviewers took up our offer!
As STEL aims to foster critical voices, it is especially important that this critique is seen as a positive thing, a way to move an individuals’ paper forward. Because when you are receiving a thoughtful critique, somebody has engaged deeply with your work. We should try to see papers as an act of collaborative knowledge production, where not only the authors are contributors. Peer-reviewers are among the many others also contributing. But for this to happen, we need to value peer-review differently. We need to think more about this in the future but one way to do this in STEL are commentaries.
Q: I have one more thing that I’d like to ask. I saved it for last because I think it is an ambivalent topic but I expect that many authors do care about it. It’s the question about Impact Factor and similar metrics. So I expect that getting indexed will be a goal for STEL but is there anything that you specifically aspire to? For example, there are journal that aim toward reaching a specific Impact Factor…
A: I worry about things like this. I understand why its important, but I also think these metrics can be harmful for journals. For example, publications like commentaries and book reviews are slowly disappearing from the publication landscape. I don’t think it is because people don’t want to write them. I think that the issue is that they don’t get cited as much, so they harm a journals’ metrics, so editors and publishers don’t encourage them. We are interested in this, but only within certain limits. So let me put it this way: If it ever comes to a choice of receiving a higher impact factor or having a thoughtful scholarly conversation, we’ll choose to keep going with the latter.
So in terms of indexing, we’ll have the ISSN soon. That’s hurdle number one. Next step will be the directory of open access journals (DOAJ). Personally, I like it when journals are indexed in Scopus but my sense is that something like that is likely to take at least a couple of years. But first and foremost, we want something that is a bona fide scholarly peer-reviewed journal. So my answer is no, we have no specific goals with respect to Impact Factor or similar metrics. For right now, we have the addition to your list to look for forward to! [laughs].
Joshua Weidlich: Thank you so much for taking the time for this discussion. I look forward to seeing how STEL develops over the next months and years.