As I mentioned in my last post, I’m writing a distributed in-memory database in Go. As a first step towards this, I wrote a server that listens to set/get requests and stores the data in a map. In Go, each incoming request is handled in its own goroutine. However, a map is not safe for concurrent use. Here’s how I approached the problem:
Maps can be protected using locks, but inspired by this post on sharing memory by communicating, I wondered if there was way to use channels instead.
func (db *db) writer() {
for {
req := <-db.writeChan
db.dataMap[req["key"]] = req["value"]
b, err := json.Marshal(db.dataMap)
if err != nil {
fmt.Println("Error marshalling db: ", err)
}
if len(b) > len(db.data) {
db.extend(len(b))
}
copy(db.data, b)
}
}
func (db *db) handler(w http.ResponseWriter, r *http.Request, ps httprouter.Params) {
switch r.Method {
case "GET":
key := ps.ByName("key")
value := db.dataMap[key]
fmt.Fprintln(w, value)
case "POST":
m := make(map[string]string)
m["key"] = ps.ByName("key")
m["value"] = r.FormValue("value")
db.writeChan <- m
fmt.Fprintln(w, r.FormValue("value"))
}
}
When a POST request is received, the key and value are passed to the writer goroutine via a channel. This goroutine is in charge of writing to the underlying map, and handles requests one at a time. However, I haven’t figured out how to handle GET’s — the map is being read by multiple goroutines.
Do you know of a better way to do this? Let me know!
Bonus fun fact: I ran into this bug.
Special thanks to Nifty Nei, Irina Gossman, Daniel Cadden and Kamron Saniee!