Andrew Egeler
Learning how things work.

Reloading Cached Files with inotify

TL;DR: I used the inotify API available in Linux to trigger a reload of data from a directory.

I have a single-threaded application that reads in a folder full of files on startup, does some preprocessing on them, and then caches the results for future use (Yes, I'm still writing posts about my blog software). I'd like to automatically reload and re-preprocess these files when any of them change, but I also don't want to put any additional syscalls I don't need in the hot-path, and I'd prefer not to poll the filesystem. However, the updates themselves are comparatively rare, so I'm going to implement something like this in C++:

int main() {
    loadThings();

    bool needReload = false;
    watchFolderAsync("data/", [&needReload]() {
        needReload = true;
    });

    while(true){
        // block waiting for a remote event
        if(needReload) {
            needReload = false;
            loadThings();
        }

        // do work with things
    }
}

That is, I'm going to use a simple boolean flag. One thread reads the flag and reloads the data before doing any work if it's set. The other thread sets the flag when anything changes in the folder. I've only slowed down my main working thread by the cost of a single branch, and I can avoid any thread syncronization overhead. To implement this, we first need a basic thread structure (note that I capture the callback into the lambda by copy, because it goes out of scope while the thread is still running):

#include <sys/inotify.h>

void watchFolderAsync(std::string folder, std::function<void(void)> callback)
{
    std::thread thr([callback]() {
        // TODO: watch folder and trigger callback() when needed
    });
    thr.detach()  // just let it free-run; we don't need to keep track of it
}

Now that I have all the setup out of the way, I can actually work on the inotify pieces. man inotify will give you all the gory details, but essentially, we need to:

  1. Initialize inotify, which will give us a file descriptor
  2. Add a watch
  3. Read the inotify file descriptor to get events

Essentially, inotify gives us an open "file handle" that we can read events from. Note that I'm not listing any cleanup steps - this thread will run for the entire life of the program, and the operating system will clean up for us on process exit.

So, first we initialize things. We acquire an inotify fd, and watch for some events:

// replacing the TODO above:
int fd = inotify_init();
if (fd < 0) return;
int wd = inotify_add_watch(fd, folder.c_str(),
    IN_CREATE | IN_MODIFY | IN_MOVED_TO | IN_CLOSE_WRITE
);
if (wd < 0) return;

// TODO: read events from fd

Now we can just do normal blocking read() calls with the fd. Each event will be read as a struct inotify_event plus a variable length C string giving the path to the modified file. Note that we need to make sure to use a big enough read buffer - if the buffer isn't big enough to hold sizeof(struct inotify_event) + sizeof(path) + 1, the read will fail, because inotify will refuse to split one event across multiple reads. In my case, I don't care what the event actually was, so I'll ignore the actual data and just simply trigger the callback anytime we get data:

// replacing the TODO above:
size_t bufSize = 512 + sizeof(struct inotify_event);
while (true) {
    char buf[bufSize];
    if(read(fd, buf, bufSize) < 0) // just exit the thread on error; note
        break;                     // that exiting the program will reach this
    cb();
}

And that's all I need! I now have a simple-to-use C++ abstraction over inotify.

Full implementation for reference:

#include <sys/inotify.h>

void watchFolderAsync(std::string folder, std::function<void(void)> callback)
{
    std::thread thr([callback]() {
        int fd = inotify_init();
        if (fd < 0) return;
        int wd = inotify_add_watch(fd, folder.c_str(),
            IN_CREATE | IN_MODIFY | IN_MOVED_TO | IN_CLOSE_WRITE
        );
        if (wd < 0) return;
        size_t bufSize = 512 + sizeof(struct inotify_event);
        while (true) {
            char buf[bufSize];
            if(read(fd, buf, bufSize) < 0)
                break;
            cb();
        }
    });
    thr.detach()  // just let it free-run; we don't need to keep track of it
}