Advanced Squeryl/Play Integration

There’s a question on Stack Overflow about how best to integrate Squeryl and Play.

Since Play is an external framework, you might guess that you want to have quite tight integration, using SessionFactory.externalTransactionAdapter. However, the right answer turns out to be looser integration, using SessionFactory.concreteFactory, which means you have to use explicit inTransaction blocks.

So what’s the problem? Well, Play’s database support is deliberately minimal. It provides a connection pool, and handles evolutions for you, but otherwise it’s up to you how you use it. It won’t commit or rollback your transactions unless you ask it to, which means you’re probably best off using inTransaction blocks anyway.

Problem solved, right? Well, almost. There’s still a slight impedance mismatch between Play and Squeryl. Play uses an asynchronous event-driven model, whereas Squeryl is designed for a connection-per-thread model.

So, for example, if you have code like this:

def sendAsteroid = Action { req =>
  import org.squeryl.PrimitiveTypeMode._
  import DinosaurSchema._
  inTransaction {
    val cretaceousDinosaurs = from(dinosaurs)(d => where(d.era === "Cretaceous") select(d)).toList
    Async {
      Akka.future {
        for (dinosaur <- cretaceousDinosaurs.par) {
          dinosaur.extinct = true
          dinosaur.save
        }
        Ok("All cretaceous dinosaurs killed")
      }
    }
  }
}

Then your code won’t work the way you expect.

The problem here, is that you don’t know what thread this code will run on. The “val cretaceousDinosaurs =” line will run on the current thread, but after that, all bets are off. Akka is free to run its futures on any thread it wants, and even the iterator is sneakier than it looks. Notice the .par in cretaceousDinosaurs.par? That tells you that this iteration can and will be farmed out to a thread pool, to speed things up.

Now, you could solve this problem by putting an inTransaction inside the iterator block. However, if you do this, then every dinosaur will be killed in its own transaction. It won’t be atomic.

I faced exactly this issue (apart from the dinosaurs) on a project I was working on. The solution we found was to manage our own transactions, and use using rather than inTransaction. It’s still a little unwieldy to manage all our own transactions, so we wrote a new Action wrapper, which minimises the amount of boilerplate required. It’s on GitHub.

Using DistributedAction, the above code becomes:

def sendAsteroid = DistributedAction { implicit session =>
  Action { req =>
    import org.squeryl.PrimitiveTypeMode._
    import DinosaurSchema._
    import DistributedAction.borrowTransaction
    val cretaceousDinosaurs = borrowTransaction {
      from(dinosaurs)(d => where(d.era === "Cretaceous") select(d)).toList
    }
    Async {
      Akka.future {
        for (dinosaur <- cretaceousDinosaurs.par) {
          borrowTransaction {
            dinosaur.extinct = true
            dinosaur.save
          }
        }
        Ok("All cretaceous dinosaurs killed")
      }
    }
  }
}

And all our dinosaurs are now killed atomically.

Notice the implicit session. Because code can hop from thread to thread, and Play doesn’t support any means of storing “request-local” data, the only way of making a session available to borrowTransaction is via variables (I suppose we could create a wrapper, that wraps a Request[A] in a Request[A,SomeContextObject], but this would probably create more problems than it solves). The use of implicits means that this session passing doesn’t need to be explicit.

Now, there are a few things that could still be improved. Because of the way Squeryl manages sessions, it’s not safe to use the same Session concurrently from two different threads, as they can tread on each others toes in Session.cleanup. We work around this in DistributedAction by synchronizing on the session. This isn’t an ideal solution, as it’s real easy to use borrowTransaction in a way that deadlocks.

In theory it should be possible to refactor Session.scala to allow it to be used from multiple threads at the same time, by making the variables _statements and _resultSets Thread local (this would mean that a session could be “bound” to multiple threads at the same time). In practice, I know Squeryl relies on some pretty deep magic under the hood, so there’s a pretty good chance something would get subtly broken. Perhaps a problem for another day.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

w

Connecting to %s