Adventures with mmap

This week I started at the Recurse Center, a self directed program where everyone is working at becoming a better programmer. If you’ve been considering it, you should definitely do it! It’s even more awesome than you’ve heard!

The first project I’m working on is a distributed in-memory datastore. But it’s primarily an excuse to play around with stuff I’ve been reading about and haven’t gotten around to! This is the story of my adventure with mmap.

The mmap function in Go’s syscall package returns a byte slice with the contents of the file you’re interested in. I passed it a file descriptor, the size of the mapping I wanted (this turned out to be the capacity of the returned slice), the memory protection needed and the type of mapping.

It looks like the memory protection needs to match the flags passed when you open the file (to obtain the file descriptor).

func (db *db) mmap(size int) {
    fmt.Println("mmapping: ", size)
    data, err := syscall.Mmap(db.fd, 0, size, syscall.PROT_WRITE|syscall.PROT_READ, syscall.MAP_SHARED)
    if err != nil {
        fmt.Println("Error mmapping: ", err)
    }
    db.data = data
}

MAP_SHARED is what makes this fun — changes you make to the byte array are carried through to the file. My understanding is that this happens “eventually”, which is why msync exists. From this code it looks like it calls fput to do the writing.

func (db *db) resize(size int) {
    fmt.Println("Resizing: ", size)
    err := syscall.Ftruncate(db.fd, int64(size))
    if err != nil {
        fmt.Println("Error resizing: ", err)
    }
}

func (db *db) open() {
    fmt.Println("Getting file descriptor")
    f, err := os.OpenFile(db.filename, os.O_CREATE|os.O_RDWR, 0)
    if err != nil {
        fmt.Println("Could not open file: ", err)
    }
    db.fd = int(f.Fd())
    db.file = f
}

func (db *db) extend(size int) {
    db.file.Close()
    db.open()
    db.resize(size)
    db.mmap(size)
}

I’m using ftruncate to extend the file before mmaping it again. Doing a bunch of ftruncates resulted in “invalid argument” errors after the first hundred and fifty or so (I hadn’t been doubling the size).

On OS X, this is either because the file descriptor references a socket (and not a file) or the file descriptor is not open for writing. So I’m currently opening the file each time I need to extend it, to get a fresh file descriptor. I’m also closing using the old descriptor because I wound up having too many files open.

Arpith Siromoney 💬

Adventures with mmap