Postgres Enterprise Manager, or PEM as we tend to call it, is based on the Open Source pgAdmin project which I started over thirteen(!) years ago to give users a graphical tool for managing and developing their Postgres databases. PEM builds on pgAdmin by adding a number of features for sysadmins, DBAs and developers with a focus on management of multiple servers, adding features such as:

- A centralised directory of servers allows all users in the organisation to quickly and easily manage any server.

- Monitoring and logging of hundreds of different metrics from database objects, the Postgres server, and the host/operating system.

- Over 130 different alert templates that may be used to build customised alerts to warn when aspects of your database go outside of the norm. A number of preconfigured alerts help get you up and running quickly and easily.

- A Capacity Manager that allows you to generate reports based on the metrics collected from your databases, Postgres servers and hosts/operating systems. Metrics from different aspects of your system can be charted together, allowing you to correlate database events with operating system activity for example. Linear trend analysis features allow you to project usage trends into the future to help with capacity planning.

- The Postgres Expert analyses your Postgres servers and database schemas, advising on possible misconfigurations or sub-optimal designs. This “DBA in a box” can analyse your entire Postgres deployment in seconds.

- A number of Performance Monitoring Dashboards give the DBA and sysadmin a global overview of the state of the database servers, with the ability to drill down and examine the status of individual hosts, Postgres servers and databases at a glance.

- A SQL Profiler for developers working with Postgres. This tool allows users to capture workloads running on their server and examine each query in detail, viewing the query plan and resource/execution statistics for each individual query, and aggregate statistics for each different query executed with the workload captured. Poorly running queries can also be run through the Index Advisor when work working with Postgres Plus Advanced Server advising on and optionally creating new indexes to improve query performance.

PEM uses a multi-tier architecture. Each managed host runs a copy of an efficient agent (written in C++, and available on Linux, Windows and Solaris) which executes tasks and reports metrics from the host/operating system and one or more Postgres servers running on the host, back to the PEM server. The PEM server is an instance of PostgreSQL 9.0 typically run on a dedicated host (running Windows or Linux), which includes various additional software components including functionality for managing SSL certificates and a dedicated instance of the PEM agent responsible for tasks such as alert processing and garbage collection. The PEM client runs on users Windows or Linux workstations and provides the user interface to the system allowing the user to connect to the PEM server and use all the PEM features, or connect to unmanaged servers as you would with pgAdmin. One feature that can be used on servers that aren’t managed by PEM (for example, a server on a laptop) is the SQL Profiler. All that’s required to use the SQL Profiler is a plugin to the Postgres server that you wish to profile which hooks into the server to log the trace data – ideal for developers.

As far as I’m aware, PEM is the first management tool for Postgres that offers anything even close to this feature set, and we’re already working on adding more for the next version, but while we’re hacking on that why not give the first version a try.

While I don’t think this is some sort of coordinated plan by Larry to bring down open source in general, I think this is indicative of the pain of integrating Sun in to Oracle and we can expect more of this in the future. On the positive side, I think the reactions by the PostgreSQL and Eclipse communities really highlight the power of the open source process. In both cases, the communities had solutions quickly in the wake of the mess created by Oracle.

object MainDBC {

  import scala.dbc._
  import scala.dbc.Syntax._
  import syntax.Statement._

  def main(args: Array[String]): Unit = {
    val db = database("jdbc:postgresql://localhost/scala","postgres","password")
    val res = db.executeStatement {
      (select fields (("pgbench_tellers.tid" of integer)
                  and ("pgbench_tellers.bid" of integer)
                  and ("pgbench_branches.bbalance" of integer))
         from "pgbench_tellers, pgbench_branches"
        where "pgbench_tellers.bid = pgbench_branches.bid")
    }

    println(" tid   bid   bbalance")
    println("----- ----- ----------")
    for(val i <- res) {
      for(val f <- i.fields) {
        val r = f.content.sqlString
        print(r + "      ".dropRight(r.length))
      }
      println()
    }
  }
}

There are a number of object layers available that may be much easier to use, but the nice thing about Scala running on the JVM is that you can leverage the low level JDBC interface and eliminate much of the awkwardness. Below is an example of the same query but using JDBC.

object MainJDBC {

  import java.sql.{Connection, DriverManager, ResultSet}

  def main(args: Array[String]): Unit = {
    classOf[org.postgresql.Driver]
    val db = DriverManager.getConnection("jdbc:postgresql://localhost/scala","postgres","password")
    val st = db.createStatement

    val res = st.executeQuery(
               "SELECT pgbench_tellers.tid, pgbench_tellers.bid, pgbench_branches.bbalance" +
               "  FROM pgbench_tellers, pgbench_branches " +
               " WHERE pgbench_tellers.bid = pgbench_branches.bid")

    println(" tid   bid   bbalance")
    println("----- ----- ----------")
    while (res.next) {
        for(val i <- 1 to res.getMetaData.getColumnCount) {
          val r = res.getInt(i).toString
          print(r + "      ".dropRight(r.length))
        }
        println
    }
    db.close
  }
}

The other interesting trend was the types of applications organizations are considering for use in the cloud. While there is much talk about replatforming existing application onto Cloud Computing infrastructures, many organizations seem to people getting their feet wet with new applications. They seem to be fairly traditional applications and not leveraging the elasticity of the cloud and they are just leveraging the financial benefits of not needed to procure new hardware. This trend show a lot of great promise for PostgreSQL given that Oracle does not have a favorable licensing model in virtualized environments. While Oracle is expensive in traditional environments, much of the cost benefits are eroded when Oracle is needed in a cloud environment. This is leading many people to open source solutions and PostgreSQL is a natural fit for many Oracle users.

The Commission’s investigation showed that another open source database, PostgreSQL, is considered by many database users to be a credible alternative to MySQL and could be expected to replace to some extent the competitive force currently exerted by MySQL on the database market.

This article, titled in part, “If MySQL fails, there’s always PostgreSQL”, puts it even more succinctly. I know many MySQL people worked very hard campaigning against approval, and I am sure they are disappointed by the outcome. Postgres users should thank them because even though they lost, their work has increased the visibility of all open source databases, including PostgreSQL.

There is much to debate about the many points Monty makes in this blog entry but one in particular deserves a mention here.

“Q:  Why can’t everyone just switch to PostgreSQL?

PostgreSQL is a great database; I am friends with many of the PostgreSQL core developers.

The problems with PostgreSQL are:

• It’s not compatible with MySQL (different feature sets and different support by various applications) and it’s far from trivial (in many cases practically impossible) to convert MySQL applications to PostgreSQL and vice versa.
• It doesn’t have a single strong company backing that MySQL has to deliver high class support globally.
• The PostgreSQL market is also, as far as I know, dominated by EnterpriseDB that provides a closed source version of PostgreSQL, which is not good enough for companies standardizing on open source.

So for the Oracle/Sun/MySQL case, PostgreSQL is not an answer that would help approve the deal, the market share is too small.”

Needless to say we at EnterpriseDB could not disagree more with this point of view.

The reality is there are many applications that have successfully migrated from MySQL to PostgreSQL.  There are many reasons that organizations are already migrating from MySQL to PostgreSQL and they are all in direct contradiction to Monty’s assertions that PostgreSQL is not a credible alternative to MySQL. Let’s address his points one at a time:

It’s not compatible with MySQL (different feature sets and different support by various applications) and it’s far from trivial (in many cases practically impossible) to convert MySQL applications to PostgreSQL and vice versa.

It’s true that MySQL and PostgreSQL are not completely compatible out of the box; for instance, PostgreSQL is fully SQL compliant and MySQL is not.  However, there are tools from EnterpriseDB and features in the latest release of PostgreSQL 8.4 that help make this task much easier.   To claim that the task is impossible is far from true and very misleading.

It doesn’t have a single strong company backing that MySQL has to deliver high class support globally.

Here at EnterpriseDB we provide enterprise class support, services, and training for PostgreSQL, and deliver these products globally with our own staff and via partners.   The Postgres community is a large, successful, and growing independent open source community that has never been controlled by a single vendor; this was the same for Linux when it launched and as it is structured today.   EnterpriseDB brings the same value to the Postgres community that Red Hat brings to the Linux community.  We think this is the best of both worlds for users: a large and independent community with a permissive license but a strong enterprise-focused vendor to turn to when needed.

A few notes about EnterpriseDB’s strength as a vendor:

• We are backed by three of the most well-known technology venture capitalists in the business.
• We are currently finishing our strongest year ever as a business.
• We expanded our business reach into new markets in Europe and Asia
• We have successfully signed more than 150 new customers this year alone, and
• We also acquired two new strategic investors in IBM and Red Hat.
• Arguably EnterpriseDB is as strong if not stronger as a commercial open source vendor than MySQL was, at the same stage of maturity.

The PostgreSQL market is also, as far as I know, dominated by EnterpriseDB that provides a closed source version of PostgreSQL, which is not good enough for companies standardizing on open source.

This argument is quiet simply misinformed, and is in direct contradiction to his previous point that the Postgres community doesn’t have a strong company backing it.  EnterpriseDB’s commercial distributions of PostgreSQL are named Postgres Plus, there are two versions of Postgres Plus – Standard and Advanced.   Our Postgres Plus Standard Server distribution is fully open source and uses a BSD license.   Standard Server adds great documentation, installers, and certifies the distribution with other products and open source projects typically used in conjunction with PostgreSQL but it remains a fully open source product.   Postgres Plus Advanced Server does have some closed source extensions to provide Oracle compatibility functionality for those customers that wish to leverage their investment in Oracle skills to develop and use PostgreSQL-based database in their applications. However, the core source code is built upon the PostgreSQL distribution.

In closing we believe Monty’s black and white argument that PostgreSQL isn’t a credible alternative for MySQL seems self-serving while discounting what is actually taking place in the open source database market.   We concede that there are MySQL-based applications that may be too costly to move to PostgreSQL or may not require the fully functioning RDBMS features that PostgreSQL provides…in those cases moving to PostgreSQL may not be a practical alternative.  However, PostgreSQL is a credible alternative for MySQL and one that should be considered by those looking for alternatives.

• The involvement of security professionals from the National Security Agency (NSA) and Tresys
• A dedicated security community meeting in Maryland (report)
• News interest in this feature because of government and credit card security requirements

It seems there is now increased user and developer interest in implementing this feature. The development would be done in several stages by adding:

• SE-Linux support for existing Postgres permissions
• Native and SE-Linux mandatory access control (MAC)
• Native and SE-Linux row-level security

The community is still in an exploratory stage and it might take months for a firm plan to be accepted.

There have already been two excellent Postgres blog posts about this issue (1. 2), so I just want to make three observations:

• I never thought Oracle cared enough about MySQL to delay the merger, e.g. MySQL was not mentioned in the merger announcement, and MySQL makes up a small portion of Sun’s revenue. I am guessing either MySQL is more important than Oracle revealed, or Oracle is resisting the EU objections out of principle or stubbornness. (Perhaps there is some advantage to Oracle in delaying the merger.)
• Oracle users rarely migrate to MySQL, so I don’t understand the anti-competitive objection to the merger. Even then-CEO Marten Mickos said in 2003 that MySQL complements and does not compete against Oracle, so it is hard to understand why the EU is objecting to the purchase on monopolistic grounds. As much as MySQL tried to position itself by adding enterprise features, the effort was incomplete, and based on the limited number of people who port applications from Oracle to MySQL, probably not very successful.Of course, Postgres works well for both MySQL and Oracle workloads based on the number of people who port applications every day, and Postgres will remain a viable open source database alternative no matter what happens to MySQL.
• There is an argument that dual-licensing is required to create successful open source software companies. Of course Red Hat, other GPL-only software companies, and Postgres companies are doing just fine, so it is hard to see how this argument makes sense. It might be a requirement if you expect to pay all the MySQL developers, which is the way MySQL has always operated in the past. Postgres and most open source projects rely much more on volunteers and on multiple companies supporting developers who work in a cooperative fashion; MySQL was an aberration in this area.Interestingly, some MySQL users are suggesting a compromise of changing MySQL to use the Apache/BSD license, like Postgres’s, which certainly is easier for companies, but not a requirement. There is a few thoughtful articles (1, 2, 3) about the licensing issue.

I think the big concern Postgres people have is that many of the things being said about this merger are either wrong or MySQL-specific and portray open source, and open source databases specifically, in an inaccurate way.