Policy on the New Hackage Server

I’ve been working on the newer hackage-server as part of Google Summer of Code. It has user accounts, editable access control lists (user groups), and a system to hook in any number of pre-upload checks. It can utilize all of these to set the policy for how it filters its data. So how does it do that presently?

User groups

There are three important user groups: admins, package trustees, and package maintainers. Some server updates require membership in these groups; membership can be edited with a simple interface.

  • Admins perform administrative tasks. They can create accounts, change anyone’s password, delete an account, make server backups, and modify the members of the other user groups. They can also modify the package index in ways not allowed by normal uploads.
  • There is one package maintainer group per package. When a package is uploaded and no versions of it existed previously, the maintainer group is created with the uploader as the sole member. Maintainers can add other maintainers. Members of this group can then upload new versions of the package, edit its preferred versions and deprecated status, upload documentation, manage build reports, and other maintenance tasks. Of course, they don’t have to.
  • Package trustees are package maintainers for all packages. They can add and remove maintainers for any package, and perform any action per package that maintainers can.

It’s not set in stone, or even etched on papyrus, who the admins and trustees are actually going to be. Initially, package maintainers will be anyone who’s uploaded a version of a given package.

Other features provide their own user groups as well. One thing about their implementation is that they are entirely decentralized: there’s no section in the code which lists all of the user groups. There is a user → group mapping, but it’s updated only in response to the groups themselves being modified. Other groups, editable by admins, include:

  • Distro maintainers: can indicate which packages are available under which Linux distributions in their binary repositories. This information is available on package pages for those who prefer distro packages, as well as in list form.
  • Mirrorers: these are accounts for scripts which copy packages from one Hackage to another. Presently this is implemented in a batch-difference mode from hackage-scripts to hackage-server and is run periodically.

Uploading

Uploading follows these steps:

  1. Uploader POSTs to /packages/ with package=[package tarball]
  2. Make sure the user is logged in and get their user info
  3. Put the package file in a temporary directory for incoming files
  4. Get the package’s cabal file, parse it, and check it’s valid. Get the package name and version.
  5. Fail if the package version is already in the main database
  6. If maintainers exist for the package, make sure the user is in the maintainer group
  7. Run pre-upload hooks; these can indicate errors and cause the upload to fail
  8. Move the package to blob file storage and add it to the main index
  9. Run all of the post-upload hooks, updating secondary indices to keep them in sync
  10. Redirect to the new package page (or display an error)

Account registration

I wish I could give you the process for account registration, but the truth is that it’s still undecided. The present system involves requesting an account via email. This could still work with the new hackage-server, technically. There are a few reasons why this kind of process could be refined: there can be several admins; account creation no longer requires access to the server’s filesystem (.htpasswd); and account maintenance of all of Hackage is a lot for one person.

Possible approaches include:

  • Admins create accounts, possibly requested from some kind of web interface.
  • Anyone can self-register partial accounts which can do everything but upload, but can e.g. edit tags, write comments, or vote. These can be transformed from partial to total accounts by admins (perhaps also using a ticket system).
  • Let anyone self-register for an account and start uploading (it’s worked for rubygems.org).

So…

The newer hackage-server comes with some nifty defaults, including more detailed ways to maintain packages. There are some guiding principles to consider when making policies: for example, packages shouldn’t be any harder to upload than they are now (which is not very). Another principle is quality assurance (see “A radical Hackage social experiment“). The above system has been developed with these in mind. Last but not least, there’s the community’s experience with current Hackage policies. How can they be improved?

About these ads

August 14, 2010. Uncategorized.

6 Comments

  1. Tom Lokhorst replied:

    What about a system where anyone can edit packages? Kind of like Stack Overflow, or Wikipedia. Where “anyone” is someone with a registered account and at least one “upvote” for having uploaded a package (or some other reasonable minimum requirement).

    It sometimes happens that I can’t use packages, because they haven’t been updated to work with the latest GHC release. A single, one line change might fix the issue, a change I can make locally on my system. However, it would be nice if I could share that fix with others on a central system like Hackage.

    The wiki aspect works quite nicely on Stack Overflow. It uses history and attribute to prevent abuse. I don’t know if this would translate well to a software package system like Hackage, but it might.

  2. Chris Done replied:

    I have the same problem as Tom. I find a lot of packages which need updating to the latest Prelude, or have one small bug that needs fixing. It would be nice if updating a package wouldn’t be down to one person.

    Supposing I want/need a change on a project. On Github I’d fork that project, do the changes, and then send a pull/merge request to the original author.

    You could actually integrate with Github, if you wanted; it provides feeds, .tar.gz exports of projects, etc. Or not.

    Anyway, if I could fork a Hackage package and then send a “merge request”, either the maintainer(s) or trustees could review it and approve it to become the official latest package version.

    Just some ideas.

    • cogracenotes replied:

      Great ideas. One new feature the server has, by the way, is a second package index where maintainers can upload ‘candidate’ package versions, which are not put in the official index, but which can be moved there by maintainers. Extending this system would allow anyone to upload candidate versions of packages which could be downloaded by others and even merged in.

      In lieu of a radical change to Hackage’s model, I don’t think there are many reasons why not to formalize the current maintainer upload system, particularly when there are those (trustees) who can circumvent it in special cases. The main package index should be stable, but I think most people agree Hackage should be more than just its main package index.

    • Duncan Coutts replied:

      Note that with the permission of one of the maintainers you can upload a new version of the package. So if you email the maintainer with the patch and they do not have time to make a new release then you can offer to do it for them. The maintainer does retain control however, they get to make that decision.

  3. moonmaster9000 replied:

    great work!! is there also going to be a frontend redesign of hackage? the 1999 ui is embarrasing.

    as far as user accounts / permissions goes, I am 110% for following the rubygems.org approach. let anyone create an account and start uploading new packages. any system more restrictive than that hurts the community and language for no good reason.

  4. Jens replied:

    Great work!! :-)

    So when will we see it go into production? :)

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: