Secure FreeBSD ports tree updating
Portsnap is a system for securely downloading and updating a compressed
snapshot of the FreeBSD ports tree, and using this compressed snapshot to
extract or update a (uncompressed) copy of the ports tree.
Historically, most people have used CVSup
to keep their ports tree up to date, but CVSup has a number of
Portsnap avoids these problems by operating over HTTP (using FreeBSD's
fetch(1) utility and a new experimental pipelined
HTTP client), signing the snapshots using OpenSSL, and using more
sophisticated delta compression to distribute
CVSup is insecure. The protocol uses no encryption or signing, and any
attacker who can intercept the connection can insert arbitrary data into
the tree you are updating.
CVSup isn't end-to-end. Related to the previous point, this means that
anyone who can compromise a CVSup mirror can feed arbitrary data to the
people who are using that mirror.
CVSup isn't designed for frequent small updates. While CVSup is very good
at distributing CVS trees, and is very efficient for updating a tree which
has been significantly changed (eg, by a month or more of commits), it
transmits a list of all the files in the tree, which makes it quite
inefficient if only a few files have changed.
CVSup uses a custom protocol. This can cause problems for people behind
firewalls -- outgoing connections on port 5999 need to be permitted -- and
it needs a heavyweight server (cvsupd).
In order to compare the bandwidth usage of cvsup and portsnap for frequent
updates of the FreeBSD ports tree, I used each method to update across
a 58-hour period from 12:15 AM on January 13th, 2005 to 10:15 AM on
January 15th, 2005. While cvsup used 6388kB of bandwidth (2418kB up,
3969kB down), portsnap only used 370kB (108kB up, 262kB down) -- an
improvement of over a factor of 17. In addition, cvsup used 212
seconds of cpu time compared to portsnap's 32 seconds of cpu time -- a
difference of over a factor of six -- and 534 seconds of wall-clock
time compared to portsnap's 75 seconds of wall-clock time (but since
the wall-clock time depends upon network congestion and server loads,
it isn't particularly significant).
Portsnap stores a compressed snapshot of the ports tree (around 40 MB)
on disk, by default in /usr/local/portsnap. This compressed snapshot
can then be extracted as needed (eg, into /usr/ports).
Portsnap is in the FreeBSD base system for all versions from 5.5
upwards (including 6.0, which was released before 5.5); users of
earlier FreeBSD releases can install portsnap from the ports
I have a nice portsnap usage graph showing the
number of systems running each version of portsnap (as approximated by
the updates they fetch).
Install sysutils/portsnap from the FreeBSD ports tree.
To fetch a compressed snapshot, or update your current compressed
snapshot, run `portsnap fetch`.
To extract the ports tree, run `portsnap extract`.
After extracting a ports tree, to update it to reflect changes in the
compressed snapshot, run `portsnap update` -- this is much faster than
`portsnap extract` because it avoids extracting directories which haven't
Please note that when extracting a copy of the FreeBSD ports tree,
portsnap will remove existing files. If you have any local changes
(eg, extra patches which you've added into a port's files/ directory)
then you'll have to put those back after updating the ports tree.
Please send any comments about this to me at "cperciva" at this domain.