Hackage at Baltimore

After the ICFP 2010 conference proper had come to a close, I came along for some Haskell-related revelry in Baltimore, from Thursday evening (Sept 30) to Sunday night (Oct 3). My paper-reading queue is chock full, and my wallet less so. Anyway, I had a good time and met some amazing people. I was there for a good reason, though.

I gave a talk at the Haskell Implementors’ Workshop about Hackage, which you can find at http://vimeo.com/15464003. It’s 35 minutes in total.

The presentation part is a straightforward overview. Open discussion starts at about 16:30. You can get the slides here [PDF].

I hope it gives you a better idea of where Hackage is going. During the weekend, I had some great discussions about Hackage, and comparing functional languages, and finance (of all things). Even better, there was actual solid planning. On Sunday, Duncan Coutts and I materialized a plan for switching over to the new Hackage. It’s up online at the Hackage trac wiki, and the current revision looks something like this:

  1. *Live mirroring (user-immutable, all accounts are historical)
    • Get archive.tar.gz of all ~10,000 packages on Hackage
    • Investigate unmirrorable packages (e.g. binembed-example, network-info, old-time)
    • Get cabal-installs pointing at it
  2. Implement backup for newer features (not all essential):
    • Download statistics
    • Candidates
    • Preferred versions + deprecation
  3. Get data migration (schema updates) working more smoothly
  4. *Live server beta testing (user-mutable, all accounts are active)
    • Disable registration; main Hackage accounts imported in
    • Still mirroring the main Hackage
    • Changes made here will be wiped out when server is fully deployed
  5. Configure server with Apache to support the tracs, support https on Hackage
  6. When ready to deploy: turn off upload on current Hackage
  7. Construct export tarball with these features:
    • core (packages, user db, admin list)
    • upload (trustees, maintainers)
    • tags (based on categories, initially)
    • distro (from current files: arch + debian, eventually exherbo + ubuntu)
    • download (from logs, give expected format to Galois log holders)
    • versions (deprecated packages, preferred-versions)
  8. Wipe server state and restore from tarball
  9. *Switch!

Throughout all this there will be testing for backups and performance. The starred items are the significant ones that’ll be announced. They look like “use it with cabal-install!”, “use it as you please unofficially!”, and “use it as you please officially!”. If you’d like to learn more about some of the ideas behind hackage-server, the architecture document is a good starting point, as well as past blog posts and the features themselves.

About these ads

October 4, 2010. Uncategorized.

One Comment

  1. Jacob Stanley replied:

    I noticed that you listed one of my packages, network-info, as unmirrorable. Feel free to contact me and I’ll be happy to do what I can to fix it up.

    There’s definitely something weird going on because Hackage lists it as an executable when actually it’s a library. It does have an executable called ‘test-network-info’, but that is only built if the the flag ‘test’ is set to true.

    It’s also not a normal situation because of the fact that the test executable depends on the library (for obvious reasons) and they’re both in the same cabal file. This configuration has only been supported since cabal 1.8.0.4.

    Hopefully the developments being done for cabal test will make this kind of thing easier.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Trackback URI

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: