Title: Checking the quality of MOOCs using quality criteria
Course: Digivisio DEMO: Planning and creating a MOOC > Section: Creating a MOOC in a virtual learning environment

Once you have completed the draft MOOC, it is time to assess once again the digipedagogical and other solutions for the course. It is worth starting the evaluation process during the course design phase and at the same time begin to report back to yourself on what solutions have been made and why. Such an approach will allow for rapid development later on, for example as tools evolve, and will give the teacher and the course designer some hope as to how the course might work better as a MOOC in the future.

The Digivisio2030 project has created a ready-to-use digital pedagogical quality criteria workbook for teachers, which allows them to review the digital pedagogical quality and successes of their teaching. For MOOCs, the use of these quality criteria is worthwhile from the beginning of the design process.  You can use other quality criteria to assess your MOOC course and one such example is the assessment toolkit for quality assessment of MOOC courses created by Laurea. You can find both quality criteria below, as well as the Digivisio2030 workbook, which you can consult to save you from major problems later on.

Digivisio 2030: E-learning quality criteria support the development of study offerings

Digivisio2030: e-Learning quality criteria: workbook

Laurea: MOOC quality criteria

When you start to look at the quality of the MOOC using quality criteria, proceed for example:

Research-based learning

Research-based learning is the starting point for all higher education. When assessing the quality of your MOOC course as you develop it, do the following:

- Ensure that your course offers the most up-to-date and relevant research.
- In your MOOC course, refer to sources and provide students with a list of references and a supplementary reading list.
- Make use of interdisciplinarity and workplace relevance when creating your material.
- Use research-based (digital) pedagogical solutions.

Modularity

When reviewing, assessing and planning modularity, pay attention to the following:

- Break the content into clear, concise chunks. There may be several modules and within modules there may be several smaller modules.
- Ensure that the learner understands and sees the module structure both inside and outside the module.
- Communicate the size of the modules and the time and workload involved to the learner.
- Ensure that the learner can progress easily and with good guidance from module to module.

Diversity

Diversity in learning supports different types of learners. When considering the digital pedagogical quality of your MOOC, pay attention to the following points.

- Use learning analytics to support student progression and achievement.
- Avoid time and place constraints when creating a course, unless it is a synchronous semi-progressive cMOOC.
- You can provide content in both text and audio. For example, use AI audio production software for audio production or (hint 1.10.2024) Google's NotebookLM service to produce a podcast of your articles.
- Activate students beforehand with questionnaires and surveys, during the lesson with questions and tasks, and after the lesson with feedback and reflection.
- Use a variety of teaching methods and assignment types during your course.
- Take time to plan interactions with both students and material. For example, encourage automatic feedback and invest in creating guidelines.
- Use a variety of assessment methods and explain to the student exactly what the assessment in your MOOC will consist of. A MOOC can also be assessed by a number.

Accessibility

It is worth starting from the beginning to create accessible material. Subsequent revisions are likely to take more time than it would have taken to consider accessibility throughout the process. So pay attention to the following:

- Make use of good guidelines produced by various authorities and other relevant bodies. These include the quality criteria for Open Educational Materials and the accessibility requirements of the AVI.
- Offer the same material in different formats: audio as text, text as audio, video with text response and video with subtitles. It is even recommended to use various AI tools such as Whisper, NotebookLM, NaturalReaders, etc. to create these responses.
- Make sure that your student can easily find the information they need and navigate your online platform effortlessly.
- Create an accessibility statement for your course. This is also a great way to assess the accessibility of your MOOC.

Usability

With a MOOC, course usability becomes a major issue. If the course is difficult to follow, the structure is confusing or difficult to understand, and the content is not accessible to the student, the retention rate of the MOOC will drop like the proverbial cow's tail. So take these things into account when considering and designing the usability of your MOOC course:

- At the outset, carefully consider the learning objectives, content, assessment criteria, workload, and planned duration of the course.
- At the beginning of the modules, indicate what content is covered in each module. Explain what each type of task involves and exercises.
- Create a consistent, repetitive, and clear structure throughout your course that is easy to navigate and within which it is easy to find information on progression, assessment, and content.
- Use meta-texts and instruct students on an ongoing basis. For example, create an easy-to-follow tutorial that constantly reminds students what to do next and how to do it.

To monitor the quality of implementation, it is also essential to continuously collect feedback from students on their studies and to receive ideas for improvement. You can also use the course to be reviewed by another teacher, or you can run the course as a collaborative project with another teacher or a team of teachers, so that more people are responsible for the quality of the course and can pay attention to quality from different perspectives.

Having gone through this list, both when you start planning your course and when you complete your MOOC, you may well be able to identify which points may cause difficulties for students. You can build more support around these points to increase the quality of guidance and enable students to progress smoothly. A good rule of thumb is that the higher the quality of the course, the more students will be reached by its content.

HEI

The university will show the teacher and the student the quality criteria on the basis of which quality digital pedagogy and MOOCs are created.

Teacher:

Teachers produce their material based on quality criteria. The teacher assesses how the content and structure of the MOOC course meets the requirements of the quality criteria and commits to improving the course as the quality criteria evolve.

  Tools:

The tools will help you develop content and build a high-quality and functional whole. The tools address issues such as the modularity of the course, ensuring diversity of content, accessibility issues and the usability of the learning material itself.

In your pedagogical manuscript, please complete the section on ensuring the quality of teaching. Take into account at least the following points.

- What digital pedagogical criteria do you set for your course and its structure?
- How do you take into account the usability of the learning?
- How will you implement the modular and fragmented nature of the course?
- How do you assess and improve the quality of your learning?
- How do you collect feedback on the quality of your studies?

References

Digivisio 2030: E-learning quality criteria support the development of study offerings

Tampere University: Teaching and Learning Centre, What is digital pedagogy?

OpenDigi-toimintamallin käsikirja (in Finnish)

Laurea: MOOC quality criteria