At HubSpot, we have a pretty wide array of different things being used for the webapps running behind the scenes. This isn’t surprising. There’a also some home-grown scripts (in python, as that’s the scripting language of choice… something I’m not complaining about) to take care of deploying the various webapps. It works, but I really want to get it doing a bit more so that it’s more useful and also get the different scripts doing a bit more sharing of code so that we can improve one place and get the benefits for everything.
Given that this seemed like a pretty typical problem, I figured I’d take a look and see what open source projects exist out there to see if any of them were suitable or could be at least close to a good fit for what we need and want. Unfortunately, I was kind of disappointed…
- Capistrano seems to be the big player in this arena. It was originally written for Rails and still very very strongly shows that heritage. This isn’t necessarily bad, but it makes it a lot harder to get to work if you’re not doing something that’s rails-like. There are some people who have gotten some things working with Java app deployments for tomcat, but they all feel a bit hacky. The other downside for me/us is that Capistrano is very much Ruby-based, both in how its own deployment language looks as well as some of the “how it depends on things working” aspects. Also, the fact that it’s written in Ruby and thus a little bit more difficult for us to hack on if/when we run into problems is a point against. So it’s probably a non-starter for now, or at least a pretty difficult sell
- Fabric is written in python and seems to be following in the footsteps of Capistrano. Right now, it’s far far simpler. This is in some ways good but some of the pieces that we’d want (eg, scm integration) aren’t there and so I’d have to write them. And I’m not sure if the Fabric devs are really interested in expanding in that way; haven’t sent email yet, but planning to tomorrow to feel it out.
- Config Management + Binary deployment is the approach taken in Fedora Infrastructure for app deployment and it seems to be working pretty well there. It might be something to get to eventually, but that’s going to be a longer term thing and I’m not actually convinced that it’s really the best approach. For Fedora it grew out of only a couple of things which could be considered “webapps” and a lot of system config that has turned much later into more webapps. It also pre-supposes a bit more homogenous of an environment than we use at HubSpot from the work I did there
- Func is something that a few people have been working on that I keep wanting to find a use for but it seems a little less well suited to doing a lot of java app building/deployment given that it’s more https/xml-rpc based than shell based.
- Roll your own is what we’re doing now and what it seems like is pretty common. I don’t necessarily like this, but it’s certainly the path of least resistance
So, what am I missing? Is there some great tool out there that I haven’t come found that you’re using for Java (and more) webapp deployments? Bonus points if its python-based and pretty extensible.
I don’t really think of Func as a deployment tool. Though I would like to eventually build a deployment/orchestration tool atop of it. That said, it looks like a Fabric is heading in the direction I was thinking of.
zc.buildout has quite a few recipes for python programs. I’ve never used it with jetty or Tomcat though.
I’m not sure what’s wrong with cfengine / puppet for config management and just using .deb packages (or .rpm). At least in debian, it seems simple enough.
What about apt or yum?
@claimid.com/alikins Yeah, Fabric is the closest… unfortunately, still pretty far. I sent a mail to their list to see if there’s any chance of some of the things I’m going to need being interesting to them. We’ll see what they say. I mostly looked at Func as Seth suggested it but yeah, not there really right now.
@scottt.id.fedoraproject.org/ I hadn’t seen buildout, but it looks pretty tied to zope and plone and without any real notion of remote systems.
@pwnguin.net/, @slashdotaccount: The problem with using Linux packages is that it presupposes that either all the developers are running Linux and can thus build the type of package you’ve chosen (not true) or that you have some central, dedicated building resources (also not the case). Otherwise, the route of building packages can work great. It’s just not appropriate here.
For Java application you can try to simply use maven with specific plugin.
Check this presentation:
http://www.slideshare.net/wakaleo/automated-deployment-with-maven-going-the-whole-nine-yards
I’m a big proponent of using koji, rpm, and yum for app deployment, with puppet for config management and consistency assurance. It’s what we’re using at $DAYJOB with around two dozen j2ee apps across ~1000 systems.
In general, where capistrano, maven, and others fail is providing integration with the sysadmins’ tools. Most of those tools try to recreate a dev-centric environment on every system, rather than promoting and enforcing reproducibility in any environment.
Plus, allowing rpm and yum to handle dependency resolution means that your code can have dependencies on packages provided by the distro without requiring your devs and sysadmins to document this multiple times in multiple locations.