One other change that went largely unnoticed however was in the downloads section of the website, specifically the FTP area. This is a web interface over the content on the PostgreSQL FTP site, ftp.postgresql.org, that gives users a nice way to browse the files and directories on the site. On the old website, when the user clicked to download a file they would then be taken to a page of flags where they could select a mirror site to download the file from. The mirrors were all third party servers over which we had no control, aside from an automated system to ensure we only listed those which had content which was no more than 48 hours out of date. This arrangement made sense years ago when bandwidth was more precious, however with the bandwidth available to us these days it’s really just clunky and inconvenient for users to have to choose one of 75 flags to reach a server that may not be entirely up to date.
To resolve this, as the new website infrastructure went live we also pushed a new download infrastructure into production. What you’ll see now is direct links to files on download servers we run ourselves, from the website. This gives us a number of advantages:
- The user has a much slicker experience when downloading, both in terms of the workflow, and often the speed of downloads (because some of the old mirrors were much faster than others).
- We can push out files to the download servers in minutes, rather than days.
- We can collect meaningful statistics to help us understand what users are downloading.
- We can automatically (and invisibly) disable download servers in the event of problems, within minutes.
- The site can be accessed at ftp://ftp.postgresql.org/, rather than using one of the individual mirror hostnames we used in the past.
- Because there are no third party mirrors, there are no inconsistent paths to the content (one mirror in the past may have had content under /pub/mirrors/postgresql/ whilst another may have used /u/postgres/).
- We can now offer downloads over HTTP using the same paths as FTP – you can use http://ftp.postgresql.org/pub/… or ftp://ftp.postgresql.org/pub/…
- We’ve also opened up RSYNC access to all users, where previously only registered mirror sites could rsync the downloads from us: rsync://ftp.postgresql.org::pgsql-ftp.