Amazon’s EC2 service is great for being able to roll out new servers quickly and easily. It’s also really nice because we don’t ever have to worry about physical hardware and can just spin up more instances as we need them for experimenting or whatever.
Unfortunately, they’re still stuck in the dark ages with the newest AMIs available for Fedora being Fedora 8 based. With Fedora 12 around the corner, that’s two years old — something of an eternity in the pace of distribution development. I’d love to help out and build newer images, but while anyone can publish an AMI and make it public, you can’t publish newer kernel images, which really would be needed to use the newer system.
So, if you’re reading this at Amazon or know of someone I can talk with to try to move this forward, please let me know (katzj AT fedoraproject DOT org). I’d really strongly prefer to continue with Fedora and RHEL based images for our systems as opposed to starting to spin up Ubuntu images for the obvious reasons of familiarity.
The rumours of my death are largely unfounded. I’ve just been either busy working or trying to relax while not on a computer since this is as much of a “break” as I get.
I have, though, done various updates to twitter and identi.ca if you have some obsessive need to know what I’ve been doing. It hasn’t been that exciting, though. Basically boils down to the following relatively short list
- Went cross-country skiing a couple of times. With the very wintry weather we’ve had thus far this winter, it’s been something good to be able to get outside and do as it hasn’t exactly been ideal biking weather
- FUDCon F11 was held in Cambridge at MIT. Since it was in E51 and I knew where things were, I spent a fair bit of time running around. I had some good conversations, but didn’t give any presentations and didn’t really get any hacking done with the hackfest
- The SDM 09s have started and I helped some with their first design challenge. Was fun to watch and they seem a good bunch
- Have been trying to read a fair bit and so made good progress on my book backlog. Still hoping to finish that before classes start back up
- Some poking and prodding in the hopes of getting Fedora 11 alpha out the door in a semi-decent shape
- More work on the new initramfs tooling, although it’s making slower progress than I’d really like
- Getting extra sleep
As davej was alluding to, we've started trying to make progress on a new initramfs infrastructure that can be used across distributions instead of just being done for a single distribution. Thus far, the code is fairly early and proof-of-concept-y, but I can successfully boot root partitions, root on LVM or root on encrypted LVM. The next step is to clean up the build side so that I can switch to using it with my laptop. I wanted to go ahead and get the code out first, though.
The majority of the ideas behind the code are that we want to use the same tools and utilities that exist on the booted system (so regular bash, regular udev, no nash, no statically linked nonsense, no klibc) and also that we want to not have the initramfs as a slow-down. The latter means that we run pretty much everything off of udev rules right now so that as soon as the devices are available, we mount and switch into the rootfs.
So go ahead 'git clone git://fedorapeople.org/~katzj/dracut.git' and read the README, HACKING and TODO files. Also sent to lkml to get the mailing list started. More hands more than welcome…
At this point, the semester is almost entirely over. While I still have one day of each class left, pretty much every assignment is done and turned in. Only have to finish up the principles assignment for System Architecture and that's mostly a matter of sitting down and throwing some together from the notes that I've got from the semester.
Overall, it's been a good semester. The workload ended up being a bit higher than I expected, but it was probably what I should have thought. I knew that System Architecture was going to be time-consuming, but it was still more so than I thought. Similarly, Project Management ended up requiring more time both for the homeworks and the project than I really expected from the outset.
Those along with working to get all of the Fedora 10 bits working on the OLPC meant that it was just a very busy semester. But, now it's actually the time when I get nearly two monhs to relax and “just” work. Well, and I also am hoping to try to get some progress underway for my thesis so that I don't have to do it all while juggling classes also. It should still be a good sort of break, though. And then, it's on to the spring semester. Which I still need to figure out what I'm going to take — suggestions welcome
For some reason, the end of this semester seems to be coming early this time around. Although the end of classes isn't until the middle of December, it seems that pretty much everything is due before Thanksgiving. Which, coupled with trying to get Fedora 10 out the door is going to make the next few weeks a pile of pain.
So if the updates from me seem sparse, that's why. And then of course, to make things even more fun, I came down with something the end of last week which I haven't quite managed to completely shake. Although at this point, it might just be my usual congestion for this time of year.
I'm somewhat excited about the new gnome-shell stuff that has been blogged about a few times by various people since the GNOME summit a few weeks ago. And so when Owen posted that there was some initial code available, I had to check it out, even if it was pretty retro GNOME 0.9 at the moment. There was, though, one important piece missing…
(Click the link for the movie)
Thanks to a pointer from warren, I've just pushed a workaround to the spins-kickstart repo to help some of the graphical glitches that have been seen on on the OLPC. This includes the blacked out buttons and also the cut-off text. Unfortunately, this probably won't make the Fedora 10 Preview release next week, although it will be in the final release. If you'd like to test it yourself, you can edit /etc/X11/xorg.conf and add a line
Option “MigrationHeuristic” “greedy”
below the EXA line.
As an update to my post last week, Snapshot 2 of Fedora 10 should be available via bittorrent now. (I had intended to post this update on Friday, but then my main laptop decided to start randomly and regularly dying on me :-/)
Again if you ensure that you have the latest livecd-iso-to-disk script, there are a few changes and improvements with Snap2
- If you pass –swap-size-mb to the script, you'll get a swap file set up which gets automatically activated on boot. As mentioned by others, having some swap makes a huge difference.
- We set up the inernal nand (flash) to be used for /home. We don't reuse the homedir from the main XO OS but the data is written there so you can update images without losing changes
- Related to that, we now reset the overlay on every boot. This means the only saved bits are the things in /home. But this lets us worry (less) about filling the overlay and causing things to crash.
On the other hand, from my quick reading of mailing lists so far, there may also be some other performance regressions vs Snap1. Testing and details from people appreciated as always.
As the Fedora 10 release cycle moves forward, we are now in the stage where there are regular “snapshot” releases planned for every week. These snapshots are just a 'point in time' test of rawhide and released based on whether they work and if not, then we end up just waiting for the next week. The intent is to keep the required effort level relatively low but still give a basis for testing between the beta and the preview release.
On Friday, the first of these post-beta snapshots was released. This includes a set of live images which can run without modifications on the OLPC. This is nice as it will make me not have to do builds of images and host them on fedorapeople anymore
With this image plus the updated livecd-iso-to-disk, the performance of the image on the XO is much better. How is this achieved? The single biggest thing that helps is changing to putting the ext3fs on the SD card as opposed to the squashfs. This means we're not paying the penalty of decompression. The downside is that this requires a 4 GB (or more) SD card as opposed to a 2 GB one.
A second thing which helps (significantly) is not using a RAM based overlay and instead putting the overlay on the SD card. There are some upsides and some downsides to doing this, but at this point, the upsides seem to outweigh the downsides. This will even more be the case for this week's snapshot with a few of the things I'm planning to have ready for that (more details below…).
So that said, what is the invocation of livecd-iso-to-disk that I'm using (and thus would recommend to testers)? Presently, it's
~/src/fedora/livecd-tools/tools/livecd-iso-to-disk.sh –force –xo –overlay-size-mb 512 – ~/Desktop/Live/F10-Snap1-i686-Live.iso /dev/mmcblk0p1
This sets up a 512 MB (persistent) overlay on the SD card and with –xo, we set up a boot/olpc.fth as well as moving over the embedded ext3fs instead of the squashfs.
So the obvious question is what's next/on my plate to make sure is in better shape for this week's snapshot?
- Fix up handling so that we can easily set up having some swap on the SD card. This again feels like a situation where we're better off doing than not. Most of the mechanism should be present in Snap1, but for some reason, I'm not seeing /mnt/live being properly mounted from the live initscript. This should be a pretty straight-forward thing if someone wanted to take a look and send me a patch The live initscript itself lives in the spin-kickstarts repo
- Non-persistent, disk-based overlay. The “persistence” is kind of kludgey in that it can pretty easily fill up and cause problems. Unfortunately, with dm-snapshot, there's not a lot we can do about that. Hopefully we'll get some form of unionfs in the future, but that definitely won't land for F10. This should just be a matter of changing the initrd to recognize a flag and recreate the snapshot store on the SD card when requested. The initrd is created from mkliveinitrd in the mkinitrd package.
- Persistent /home on the internal nand (mtd0). Again, this should be mostly ready to go with Snap1, but was acting weird the end of last week for me. Should get mounted within the live initscript
- Start making a dent in some of the reported bugs
- Investigate the question which was asked of just sticking the ext3fs on the SD card directly and not doing the “normal” live image bits for the overlay. While this is interesting to explore, I have a sinking suspicion that it's not something that's really doable in the timeframe of Fedora 10, although it is certainly one that bears looking at more for Fedora 11.
Took a long weekend from pretty much everything work related (although I had to do some school work) and then got back this evening and spent way too much time on some homework for System Architecture. This process was made more frustrating by OpenOffice deciding to crash and corrupt its own backup file in the process. I'm so glad that office software really hasn't improved :-/
So if you've sent an email and expected to see a response from me, hopefully I'll get to it tomorrow. Direct mail looks quiet, but I see there are some 800 messages on various Fedora related lists for me to get through…
For those chomping at the bit about Fedora on OLPC, the new snapshot images that went up on Friday should work better on the XO and integrate everything that's needed without requiring a special OLPC build. You'll also want the updated <a href="http://katzj.fedorapeople.org/olpc/livecd-iso-to-disklivecd-iso-to-disk which copies the uncompressed ext3fs.img onto your SD card rather than the squashfs. You'll definitely need a 4 GB SD card instead of a 2 GB one for that and there are also some corner cases like running out of disk space that I haven't fixed up in the script, hence why it's not in git yet. And if none of that makes sense, I'm intending to get a more verbose update up tomorrow.
But right now, it's after midnight and I was up before 5 am to head down to a bike race and so I'm exhausted and going to go crash into bed for a few hours so that I can wake up in the morning and maybe even make it to class on time…