Announcing ami-creator

I’ve been having to build some new CentOS images to be used with EC2 for work recently. I went into it thinking that it shouldn’t be too big of a deal. I know that some work had been going on in this area and Fedora 14 is now available on EC2, so I figured I could convince the same toolchain to work.

Unfortunately, I was pretty disappointed with my options.

  • Do some building by hand on an actual instance, then do the bundling and upload off of the running instance.
  • Some of the ThinCrust stuff initially looked promising, but it seems like it’s largely unmaintained these days and the ec2 conversion bits didn’t really work at this point. I was able to get my initial images this way, but mostly by having a wrapper shell script of doom that made me sad.
  • There’s always the rPath tools, but I wanted to stick to something more native and fully open source
  • The new kid on the block is apparently BoxGrinder but I found it to be a lot over-complicated and not that robust. I’m sorry, but generating your own format that you then transform into a kickstart config and even run through appliance-creator via exec from your ruby tool just felt wrong. No offense, but just felt like a lot more than I wanted to deal with

So, I sat down and spent an evening hacking and have the beginnings of a working ami-creator.
It’s pretty straight-forward and uses all of the python-imgcreate stuff that’s used to build Fedora live images. Your input is a kickstart config and out the other side pops an image that you can bundle and upload to EC2.

Thus far, I’ve tested it to build CentOS 5 and Fedora 14 images. I’m sure there are some bugs but at this point, it’s worth getting it out for more people to play with. Hopefully it’s something that’s a lot simpler and more accessible for people to build images and I think it will also fit in a lot better with having Fedora release engineering building the EC2 images in Fedora 15 if they want.

One of the big outstanding pieces that I still want to add is the necessary bits to be able to (optionally) go ahead and upload and register as an AMI with your EC2 account.  But release early, release often.

Comments, etc appreciated in all the normal ways.

Minor update: switched the repo to live on github instead

8 thoughts on “Announcing ami-creator”

  1. Hi Jeremy!

    You mentioned BoxGrinder in your post. I want defend it a bit here if you don’t mind.

    1. Over-complication

    It has a plugin architecture – you must install plugins you want to use. BoxGrinder does a lot of things. It does produce the base image, then this image can be converted to selected virtualization/cloud format and at the end the it can be delivered where you want (sftp, s3, cloudfront, etc). The goal is to have one tool that supports many virtualization/cloud platforms.

    2. Own format

    Own format was developed because BoxGrinder is meant not only for RHEL-family. The format is prepared to create appliances for other operating systems that don’t use kickstarts too. Fedora and Co. plugins were first because it was obvious to support own stuff first :)

    And I personally think YAML format is more readable for a normal user (not being a Fedora hacker) who is interested in getting job done. This format has also notion of inheritance where you can override for example partition scheme or hardware allocation. I don’t want to argue which format is better – they’re different to meet different kind of problems.

    Funny, but I commited yesterday support for native definition files (https://issues.jboss.org/browse/BGBUILD-113), especially for kickstart files: https://issues.jboss.org/browse/BGBUILD-73

    3. Executing appliance-creator

    Yes, I agree with this point. If there is a way to use OS native installer (see https://github.com/clalancette/oz) I’ll use it.

    P.S. No – I don’t took you post as offensive.

    Thank you!

  2. Hi !

    I’m really interested in your project. I have 2 questions for you.

    1) By

    “Do some building by hand on an actual instance, then do the bundling and upload off of the running instance.”,

    do you mean building a system “by hand” on an attached EBS Volume, then snapshot it and register it as an AMI ?

    2) Have you heard about the boto Python interface to AWS ?

  3. @raphdg
    1) Yes.
    2) Yep, I know all about boto and will probably be using it as I add the bits to do the actual uploads. Although boto has some (surprising to me) holes in the pieces for creating AMIs right now from the quick look I did. But my intent will be a) get it working in ami-creator and then b) refactor to get it into boto upstream.

  4. @goldmann Don’t mind at all. And I know that’s what you’re going for. I just think it’s over-complex. KISS and all that.

    Also, lots of people have done lots of things with formats that aren’t kickstart… but kickstart configs have been alive and kicking for well over a decade now. I realize that they’re not what you’re used to, but there are tons of admins that use them every day. And the real win is that they’re used for *everything* so I only have to learn it once and I’m good

  5. Hi Jeremy !

    Talking about kickstarts, do you think it would be possible to do the following :

    Launch a CentOS 5.5 instance
    Attach an EBS volume
    Mount that volume
    Create a boot partition and add a compatible vmlinuz + initrd (modified so it integrates a kickstart cfg file) in it
    Create the grub menu.lst and specify the kickstart path
    Snapshot that volume
    Start an instance with hd00 pvgrub so it boots with grub, grabs the menu.lst and start the install process

    I tried it but grub does not seem to care about the ks parameter in the menu.lst…

  6. @raphdg
    I’d like to eventually add support to create an EBS backed AMI. I’m not quickly going down that route because a) I don’t need it right away for work and b) I’m hoping Amazon adds an API that’s a bit saner for doing so in the not too far future ;)

  7. Hi -

    The boto library does not directly address AMI creation but you might want to check out euca2ools because it does provide Python tools to create, sign and upload AMI’s. We have talked about factoring that code out and including it in boto. Any thoughts on that?

    Mitch

    1. Yep, there have been a couple of times I considered pulling code out of euca2ools to use for uploading the amis. But it ended up that after we got to a working image, it wasn’t something I had to do often enough to push more on it ;-)

Comments are closed.