Setting up the hotfolder module on Drupal

I use the hotfolder module to process the sermons we record every week. I wrote the recording software, so I was able to add the features I needed for this to work. Basically, after recording, the software uploads the sermon to a certain folder on the website and then visits a web page with the cron key to initiate hotfolder processing. During the recording, the software writes information into the file (using MP3 tags) that the custom hotfolder module uses to determine various properties for the new node, including which sermon series it belongs to, and then it creates a new content node (Sermon, and Sermon Series, if the series doesn't exist yet). Our custom module handles the specifics of how the file is processed, but hotfolder triggers the various stages.

So hotfolder is a great module, but it basically has no documentation. OK, there is documentation in the code about how the code works, but you really have to figure out how to configure it on your own.

While I was developing, I found that I did a lot of testing with the same file. To reset hotfolder to do another test, I would do the following:

• Turn on all debugging options and add debug statements as needed in my custom module (using drupal_set_message() or dpm() from the devel module)
• Rename the "incoming" file in the watchfolder to prevent it from getting deleted when I delete the job
• Delete the job so hotfolder won't ignore the file (or you can delete it from the files_managed table manually)
• Click Run Watches button
• Click Run Jobs button
• Check for errors

Settings

Here are the various settings, found at admin/config/services/hotfolder or Configuration > Web services > Hotfolder configurations, if you are using the Administration menu.

Folder Configurations: I am not using this at all. The additional streams is not necessary. I think it's a convenience for setting up an alternate location, similar to public:// or private://, so you don't have to enter the full path to things.

Error Configurations: You'll probably want this set to Debug mode and Verbose while working on your custom module.

Retry Configurations: I suppose it might make sense to have retries in some situations. For my situation, if something fails, it is not likely to succeed after more tries.

Worker Configurations: This is more likely to be used of there are a lot of jobs or jobs that take a long time. I just left it at the defaults.

Controls

I'll only comment on the buttons I've used. I'm not sure exactly what the other buttons would do. If you put a file in your hotfolder and want to force processing, you'll want to 1) run watches and then 2) run jobs. Otherwise, they will get triggered whenever cron runs (or specifically, the hotfolder cron). I am using Elysia Cron on my website to control a few special cron things.

[Run watches]: This runs a check for all of your watches. In other words, it checks whatever incoming folder you have set up to see if there are any files of interest. If there are, a job is created to process them. However, there is no actual processing started at this point. Only the 'prep' trigger is ... triggered.

[Run jobs]: This initiates processing of whatever jobs are in the queue. This is where the real "work" happens.

Watch Configurations

There is a view set up that lists your watch configurations. It seems like this view should be included on the hotfolder configuration page, but I don't see it there. So you can get to it from the views settings (if you have the Views UI enabled): admin/structure/views/view/watch_configurations.

Viewing the watch

On the View tab for the watch, you'll see the configured settings (but not be able to edit them). There is a button here to force running the watch. If there is a job in the queue that hasn't completed, there will also be a button that gives you the option to reset the job. This will reset it to the first stage of processing, but will keep the same settings that were active when the job was first prepped.

Job history

There is a job history tab, under which you can see a filterable listing of jobs. If you have the delete completed option enabled, you might still see jobs here if an error occurred during processing. Otherwise, you'll see jobs that are are still in the queue.

Editing the watch

Click on the Edit link to configure your watch.

Watch ID: Simply gives the watch a name. You can use spaces here.

Stream: If you set up an alternate stream, you would be able to use it here. Otherwise, you just have public and private. Which seems like it would be enough for most uses, really.

Watch folder: The path of the folder to watch, but it's not the full path. For example, if you are using a hotfolder that you would find somewhere like sites/files/myhotfolder/, you would select the public:// stream and enter myhotfolder as the watch folder.

Recursive watch: If this is unticked, only the folder entered as the 'watch folder' would get checked. If you want folders below that one to be checked as well, tick this option.

Delete Completed: Tick this to delete the job and the associated file from the hotfolder after processing is completed. I have it ticked, so during processing, my custom hotfolder module copies the file of interest (the MP3 file and also a couple other files that are uploaded with it). After processing, MP3 file is deleted, but my other files are not... so the custom module moves them instead of copying.

Source Filename Pattern: As it says, this is a regular expression pattern, not a glob pattern that you might think of when thinking about filename. So to find all *.mp3 files, you would use this regular expression: .*\.mp3

Configured Actions: Select which actions should apply to the matching files in the hotfolder. If you implement the form_callback with some settings, this is where your custom settings appear when you select the associated action. I have an action "Process MP3 file" with a setting for my drop-off folder.