Moodle 2.0: File management and repository - file picker vs course files
I must admit, straight away, that writing this post has left me in tears. As I sob over my keyboard, my head is spinning in conflict about the way file management has been implemented in the brand new version of Moodle: Moodle 2.0. I'm hoping one day I'll be able to write a post called 'Dr. Strangemoodle. Or: How I learned to stop worrying and love the file picker.' That day is not today.
Why a new way of thinking?
One of the arguments that is being churned out by supporters of the new file system (lead Moodle people, I will refer to these as 'new paradigmites'), is that Moodle is not supposed to be a file repository/CMS (content management system – or even a basic equivalent), it is a learning management system (LMS) and we should not be encouraging simply uploading files. Now, I'm not going to criticise the inspirational pedagogic ethos here, but when you consider that providing some form of resource is what the vast majority of current teaching processes require, you would have thought that the file management system would be seen as the most important thing to get right in an LMS or CMS. In my view, we cannot assume all users are at the same online pedagogic level as the new paradigmites. Many tutors have not yet embraced simply uploading a PowerPoint revision aid or preparatory seminar reading.
The 'old' model (which is referred to as 'legacy course files' to cement in our minds that this is the 'old way' of thinking), has a file store area for each course. This area mimics what users (teachers) are used to: it's like windows explorer, with folders and files. Links are then made to these files with 'resources' or even within resources such as HTML web pages which can have relative link paths. In the 'old' model, you upload a file and can refer to it from many resources. If you need to change that file, you overwrite it and it updates all resources.
The common problems with the old way were that you could not access these files from other courses and when you wished to copy an activity (eg a forum or quiz), you had to copy the entire course instead of just copying that activity in order not to lose attached files or quiz question files.
The new model addresses this by attaching files to individual instances of activities or resource links. This also allows more rigorous access control, international filenames and access to everything regardless of where it is located (with the right permissions).
However, my biggest gripe is that we have now swung from one extreme to another. All that was possible previously is now awkward, and all that was impossible previously has been realised. I fear this may be down to a poor requirements analysis, where discussions focused on what was missing on previous versions and downplayed the essential requirements of current users.
Essentially, the Moodle 2.0 file system has been abstracted, so files no longer reside on the server in a logical 'tree' structure with the same filenames they would have on your office computer. This means that in order to decipher the files stored on your server's hard drive you need access to the database file table. Why this level of security through obscurity is required, on an already very secure space on the server without http access, I will never know. It is security through obscurity (read 'pointless') too, the files and content are still there, just renamed with random hash strings and with the file extension removed.
What is worse is that not only can you now not at a glance see which courses are hogging all your server space via the server, but that even within Moodle, in order to get to a simple list of files you have to be creating/editing a resource or be using the text editor! It also means that when using the standard Moodle 2.0 file management system it is impossible to use relative links within files (e.g. an uploaded HTML file pulling in images/css kept locally to the HTML file).
At first, the new repository system was heralded as a way to access files from any part of Moodle. It seemed as if we could finally stop duplicating our files in every course. However, things are not as simple as that. When selecting files from an external repository with Add Resource > File, the Moodle 2.0 file picker in fact copies files from the original source (so every resource/activity has its own instance of the file), instead of providing links to them. Therefore, you have to ensure you create a link with Add Resource > URL.
When using Moodle's inbuilt repository things get slightly more complicated. If you upload a file to the Moodle server files, add it as a file in five courses, spot a mistake and upload a new version, the file does not change in all courses automatically but requires changing the link to the file for every instance. This is because the only way to change something is to delete and reupload (in order to keep other instances as they were). A lot of effort I think you'll agree. Similarly, imagine you upload a file which is then used by other tutors, you spot something dodgy in your original file and wish to remove it, making it offline. If other tutors have used this file, they have their own virtual copies. i.e. they're using version A on the server and you're now using version B, and you won't be able to track them down as deleting your original does not delete their copy. OK, so in a normal CMS or the old course files, there's nothing to stop the second tutor downloading and re-uploading a copy, but when you use the terminology of a 'repository' where you select a file, you are not expecting a new instance of that file to be created if you wish to change it.
ED: For a very clear explanation of the duplication problems caused be linking to files both within Moodle and repositories see this post by Elena Ivanova.
ED: Check the comments below which state that duplication is on the user point of view, and on the server it only occurs at point of re-uploading (or when using external repositories).
Comparison of file management between Moodle and Blackboard
In fact, far from resolving file management issues, the new Moodle 2.0 file 'paradigm' will cause more headaches. What has happened is that files are now associated even more deeply with elements of courses than previously. Although, yes you can access resources across the whole site, this is only leading to duplication as the system copies instead of links. What is required, as much as I really do wish to kill myself for saying it, is something like Blackboard's CMS system.
- Blackboard CMS model: Upload to a central repository (file system), link to the files.
- Moodle 1.X model: Upload to a course repository (course files), link to the files.
- Blackboard LMS model: Upload to an individual activity/resource.
- Moodle 2.0 model: Upload to an individual activity/resource. Or, locate existing resource and Moodle will link to the file.
Storing and accessing files
- Blackboard CMS model: Single, logical structured file system, username-based rights on individual files/folders. Access through CMS section of Blackboard (Content Tab).
- Moodle 1.X model: Course-based logical structured file system, course-based rights on whole course file system. Access through file manager in each course.
- Blackboard LMS model: Abstracted file storage, activity/resource-based rights. Access only through editing resource. Only able to access the items associated with the resource currently being edited.
- Moodle 2.0 model: Abstracted file storage, activity/resource-based rights. Access only through going to edit an activity/resource, but can access other resources and other courses items through the file picker.
This may also best be represented with this little picture (note: consider Moodle 2.0 duplicated file to be a virtual duplication to the user, not on the server, until one of the files change):
But, this is how it is
Sadly though, I don't think the Moodle wider community will win this one. The core Moodlers seem insistent on the merits of this new file system, even though it is counter-intuitive, more costly in terms of server space, and difficult to navigate. Here's a quote from Martin Dougiamas, the lead Moodle developer, from this forum discussion: http://moodle.org/mod/forum/discuss.php?d=157250
"The answer is not to go back to the 1.x design - it's too late for that. If there are issues with the 2.x interface then let's work on them (as we have been for over 2 years). To me it's clear that the general trend in the world is to move towards specialised repositories that do that job really well, and then plug these into application servers like Moodle to use the files. I think most of us involved recognise that there is going to be some re-learning required, that old workflows may have to change, but the result is a more powerful, flexible and international tool with fewer problems."
I don't like to say this, and I do have a lot of respect for the core Moodlers, but as it's been brewing in my head all day I need some form of vent: There is always a danger when one spends time and effort on nurturing a new way of thinking, new code developed, that it leads to becoming blind to the needs of real users. A closeness to one's code can often lead to not stepping back and considering 'this is really not going to work for the average tutor out there.' I admit, that it's nothing great for me to come in at this point after a whole year's development on the system and essentially say 'you're wrong', but the impression that's being pushed from the Moodle forums is that it is us, old workflow type of people, are wrong in how we choose to use Moodle.
It could work
Theoretically, though I've not had a good look at the database structure and linking mechanisms, it shouldn't be too difficult to stop the duplication of files. Indeed, it looks like serious discussion is happening around providing options to either 'link' or 'copy'. If a file is known, a reference stored in the database, then all it requires is to tie that reference to instances of resources/activities, rather than the files. When backing up, that is the point at which you pull the file into the instance of the resource/activity. This database referencing also means that the file picker when browsing system files can instead use the same structure as course files, perhaps with two extra virtual folders for 'forum resources' or 'question resources' etc to help manage that side of things. After all, if there is abstraction in the true system, then the file picker is just a visual representation. Finally, the ability to look at the files (as in the file picker), rename, overwrite and delete from outside a resource/activity editor should be feasible too. It is highly unintuitive to have to go into and edit a resource in order to do something with an uploaded file, when it should be accessible directly.
One of the fundamentals to usability is to ensure the user doesn't have to remember. Unfortunately with the Moodle 2.0 in order to link to a file, you have to remember the course and activity the file was attached to in addition to the filename itself. Adding metadata or a search facility doesn't resolve this problem, as again it's dependent on the user remembering what meta taxonomy is being used, and attaching metadata in the first place (which the average tutor who just wants to get the job done isn't too fussed about).
Overwriting files has also been sidelined. If you attempt to upload a file with a filename that already exists, you get notified, but aren't able to overwrite it. You have to either upload and save as a different file name (what the?!) or first delete the old file before re-uploading a new version. Since the days of DOS you've been prompted with 'File exists: Overwrite? Yes / No / Run away', so why another basic element of file management is missing here I do not know.
One of the suggested approaches is to set up a standard tree-like file structure repository folder. However, the only way to upload files to this repository is outside the Moodle interface, e.g. by FTP. This is not practical for any small-scale or institution-wide system where tutors may want to upload HTML mini-sites. Tutors don't have the right access, neither would we want to give them FTP access to the server!
The good stuff
What the new file repository approach has allowed though is the use of external resource sources, such as Google Docs, Flickr (Flickr Repository and Flickr Public) and Dropbox (Dropbox Repository). Integrating these external systems, which also encourage transferable IT skills (i.e. once someone leaves and institution they won't be using Moodle, but they may continue using various third-party giants like Google Docs), is definitely a step up in creating rich learning environments.
The private repository is also a great. Students have craved for a space to upload their own resources, and even better to be able to pull them directly into forum posts. I think this is definitely going to be one of the more positively received aspects of the new system.
Workarounds for HTML mini-sites and HTML-based learning resources in Moodle 2.0
Thankfully, the old way of uploading, accessing and linking to files still exists. However, it's not activated by default, is scorned upon by the new paradigmites. There are still (probably the majority) of structured courses using Moodle who have HTML-based content, woven together with CSS, images, documents and the like, uploaded in zips and unzipped in Moodle 1.9 preserving the relative paths linked in the HTML code. It took a lot of badgering to get a half-usable solution in the Moodle 2.0 release.
I've posted the solution on the Moodle User Forums and will create a proper guide on this site in due course.
So whilst my heart still belongs to Moodle, my head is having a rough time with the numerous usability flaws with content management. By the language (use of 'legacy' and 'still want') I think that we have a long way to go to address these issues from the user perspective, regardless of the aspirant pedagogical philosophy where Moodle becomes the centre of learning, not a centre of resources. There'll be more posts on Moodle 2.0, hopefully more positive!
To really get your head around what is going on I strongly recommend reading these two posts by Mark Drechsler:
And, if you are a little lazy, just view his SlideShares:
The latest incarnation of the 'bug' tracker around the file system is here:
Some background discussions are on the Moodle forums here: