What does “cloudified” mean for a database? It’s the addition of technology that allows you to:

• Automatically scale in response to increased demand
• Set up highly available multi-node database clusters in minutes, not days
• Pay for your database as if it were a utility

You can read more about Postgres Plus Cloud Database here:
http://cloud.enterprisedb.com

As we looked at the landscape of available cloud databases, four things became very clear:

• Before the launch of Postgres Plus Cloud Database, only Micrsoft SQL Server (solely via SQL Azure) and Oracle’s MySQL (solely via Amazon RDS) offered enhanced versions of a relational database to take advantage of inherent cloud scaling technology.
• PaaS and other cloud platform vendors needed a database vendor to deliver a valuable cloud compatible database component for their platform to bring maximum support, features and value to their customers.
• NoSQL options are varied and will continue to grow AND (not OR) old-school SQL RDBMS are as highly in demand in the cloud as they are in the data center.
• Just putting “cloud” in the name does NOT make a database “cloudified”

We are changing all that with this product launch. We are giving users a new option to run either the latest community version of PostgreSQL or our flagship Postgres Plus Advanced Server with Oracle database compatibility features on Amazon EC2. Perhaps more important is that we have architected Postgres Plus Cloud Database to be universally pluggable with any PaaS vendor platform. To that end, our work with leading platforms from CloudBees and Engine Yard will bring enhanced database capabilities to the growing number of developers adopting their technologies. And we are extending that work to include OpenStack, HP, Red Hat’s OpenShift and many others, making Postgres Plus Cloud Database the new standard relational database for all cloud platforms.

Postgres Enterprise Manager, or PEM as we tend to call it, is based on the Open Source pgAdmin project which I started over thirteen(!) years ago to give users a graphical tool for managing and developing their Postgres databases. PEM builds on pgAdmin by adding a number of features for sysadmins, DBAs and developers with a focus on management of multiple servers, adding features such as:

- A centralised directory of servers allows all users in the organisation to quickly and easily manage any server.

- Monitoring and logging of hundreds of different metrics from database objects, the Postgres server, and the host/operating system.

- Over 130 different alert templates that may be used to build customised alerts to warn when aspects of your database go outside of the norm. A number of preconfigured alerts help get you up and running quickly and easily.

- A Capacity Manager that allows you to generate reports based on the metrics collected from your databases, Postgres servers and hosts/operating systems. Metrics from different aspects of your system can be charted together, allowing you to correlate database events with operating system activity for example. Linear trend analysis features allow you to project usage trends into the future to help with capacity planning.

- The Postgres Expert analyses your Postgres servers and database schemas, advising on possible misconfigurations or sub-optimal designs. This “DBA in a box” can analyse your entire Postgres deployment in seconds.

- A number of Performance Monitoring Dashboards give the DBA and sysadmin a global overview of the state of the database servers, with the ability to drill down and examine the status of individual hosts, Postgres servers and databases at a glance.

- A SQL Profiler for developers working with Postgres. This tool allows users to capture workloads running on their server and examine each query in detail, viewing the query plan and resource/execution statistics for each individual query, and aggregate statistics for each different query executed with the workload captured. Poorly running queries can also be run through the Index Advisor when work working with Postgres Plus Advanced Server advising on and optionally creating new indexes to improve query performance.

PEM uses a multi-tier architecture. Each managed host runs a copy of an efficient agent (written in C++, and available on Linux, Windows and Solaris) which executes tasks and reports metrics from the host/operating system and one or more Postgres servers running on the host, back to the PEM server. The PEM server is an instance of PostgreSQL 9.0 typically run on a dedicated host (running Windows or Linux), which includes various additional software components including functionality for managing SSL certificates and a dedicated instance of the PEM agent responsible for tasks such as alert processing and garbage collection. The PEM client runs on users Windows or Linux workstations and provides the user interface to the system allowing the user to connect to the PEM server and use all the PEM features, or connect to unmanaged servers as you would with pgAdmin. One feature that can be used on servers that aren’t managed by PEM (for example, a server on a laptop) is the SQL Profiler. All that’s required to use the SQL Profiler is a plugin to the Postgres server that you wish to profile which hooks into the server to log the trace data – ideal for developers.

As far as I’m aware, PEM is the first management tool for Postgres that offers anything even close to this feature set, and we’re already working on adding more for the next version, but while we’re hacking on that why not give the first version a try.

I was pleasantly surprised how well EnterpriseDB has been received in Europe. In my presentation, I showed a heat map of PostgreSQL being used around the world. PostgreSQL is heavily used in Europe and North America. So it was great to hear that developers understood what our solution entailed and how involved EnterpriseDB is with the PostgreSQL community.

Second, attendees urged us to work with Red Hat to integrate PostgreSQL into Red Hat solutions. Recently, we announced that EnterpriseDB joined Red Hat’s OpenShift Partner Program by offering EnterpriseDB Postgres Plus Advanced Server as a database service in the cloud (DaaS). The Red Hat OpenShift Program, combined with EnterpriseDB’s Postgres Plus Advanced Server, lets developers build scalable cloud database systems, support popular developer languages and move Oracle applications to the cloud easily and quickly with all of the benefits, stability, innovation and low cost of open source software. It’s a step in the right direction, but attendees wanted more, which excited us!

The opportunity for PostgreSQL in open source today is rapidly growing and the adoption of PostgreSQL will continue to gain traction in the months to come. This week, we announced general availability of our product Postgres Plus Advanced Server 9.0. Improvements to our product include enriched database performance and availability, and increased Oracle compatibility–now entering it’s seventh generation of compatibility—and support for the HP-UX operating environment on the Intel Itanium platform.

To learn more about my keynote given at Red Hat Open Source Day in Rome you can download the full slide deck on our web site.

Thanks to the EnterpriseDB engineering team and all the customers whose feedback went into 9.0!

During the conference a map was displayed that showed all PostgreSQL users groups in the United States. Boston MA, where I was born and have lived the past 28 years was noticeably missing. Upon seeing this, I decided it was time to start the Boston area PostgreSQL users group.

On May 24th we had the first meet of the Boston Area PUG. We had 23 people attend from about 12 different companies. Ed Boyajian spoke about the status and recent growth of PostgreSQL. In addition,  a great local company (TripAdvisor) spoke about their large scale PostgreSQL deployment.

For those of you who are interested in attending our next meeting, please see this URL: http://pugs.postgresql.org/bospug

Ultimately, what I really want to share is my journey starting this group.

I started by asking Bruce Momjian for the formula to start a PUG. He gave me the first two steps and subsequently coached me through out the process. Here are those first two steps:

1. Send e-mail to pgsql-announce@postgresql.org announcing the time and location of the meeting.
2. Post the event on the PostgreSQL Events site.

So that is what I did. Over the next several days I received seven responses from pgsql-announce mail that I had sent. However, things began to speed up when I sent a note to the ex-odi mailing list. ODI, stands for Object Design Inc. This was a great Boston based company that I worked at in the early and mid-90s. We made the world’s greatest Object Database and the company was loaded with top technical talent. It was eventually acquired by Progress Software however along the way it’s alumni created and participated in many Boston area companies. From the message I sent to the alumni group, I received about ½ dozen responses from people who wanted to come to our PostgreSQL user group meeting. However, what really kicked it up into high gear was a response from Amrith Kumar. Prior to the user group meeting, I had never met Amrith. Turns out he is starting a company with my former boss and office mate at ODI: Ken Rugg. Amrith was very anxious to help and began suggesting additional things for us to do to organize the event and promote attendance. Some of his specific suggestions were using meetup.com and eventbright.com.

I didn’t realize until that moment sites like this have become the cornerstones for user group meetings. We publicized our first meeting via Event Bright. It was great and easy to work with. It was fun to watch people sign up and the list of attendees grow over the course of several weeks. However, it turns out the meetup.com in what is most commonly used by PostgreSQL User Groups. Therefore, we have switched to using it going forward. If you are interested, see if you can find us on meetup.com. If not please let me know (tom.kincaid@enterprisedb.com).

The next big break was when Don O’Neill, who had recently become the Director of Engineering at Trip Advisor, contacted me. He was very anxious to get involved and help. I immediately asked him to speak about their PostgreSQL deployment at TripAdvisor. TripAdvisor has been using PostgreSQL for a mission critical deployment and profitable deployment for many years. In addition, their head quarters are in the Boston area.

While this was all coming together, in the background I connected with Selena Deckelmann who helped get us set up on pugs.postgresql.org.

As mentioned previously, the speakers were great. However, what I found surprising was how much I enjoyed the team spirit of the event. Everybody wanted to figure out how to make the group a successful learning and networking venture for all of us. Some people were offering ideas for future topics they could present on and other people were offering to host the meeting. Gurjeet Singh, a high powered EDB PostgreSQL consultant / PostgreSQL hacker / EDB Product developer set up a Facebook group and a twitter tag. We had begun a new journey together to be a successful Boston based PUG.

I wish you luck on your journey starting your own PUG and for those of you in the Boston area, I hope you will join us. We meet the second Tuesday of every month. Our next meeting is July 12th, 2011. Bruce has agreed to drop by and speak about the Query Planner.

It’s no secret that smart Web companies as well as other modern organizations use distributed memory caching architectures to speed up database response time. The most common piece of software utilized is memcached, although other similar technologies exist.

One of the high-performance additions offered in Postgres Plus Advanced Server is Infinite Cache, which is a more flexible, more feature rich, and easier-to-use distributed caching solution than memcached. An Infinite Cache setup allows database systems to access cache memory outside the physical database server by accessing memory caches on other machines that are in the database server’s network. The end result is that a greater amount of overall memory is made available to a database, which results in an entire database being cached in many cases. Naturally, such a thing has the potential to result in much faster response times for database queries and DML work.

Does Infinite Cache really work? In short, yes it does. The example below (obtained via the pgbench utility) shows the TPS benefits derived from using Infinite Cache for various workloads that include a 4-node Infinite Cache configuration:

One of the drawbacks of various memcached-styled configurations, including 8.4 and earlier versions of Infinite Cache, is that any new machines that are added to the overall configuration result in the entire cache having to be reset. This can be problematic because applications that depend on 24×7 fast query response times experience a performance disruption in their system as the cache ‘warms’ back up.

In Postgres Plus Advanced Server 9.0, Infinite Cache has been enhanced to greatly minimize this situation. In 9.0, when a new node is added to an Infinite Cache configuration, only part of the cache is flushed and must be “re-warmed”.  The percentage of the cache that needs to be re-populated depends on how many nodes exist in the current configuration (the more nodes that currently exist, the less cache that needs re-populating). As an example, in a 4-node Infinite Cache configuration, if a 5^th node is added, 80% of the cache is retained, whereas prior to 9.0, all of it would be lost.

Try out the new Infinite Cache in Postgres Plus Advanced Server 9.0 now. We’ve just released Beta 4, which will be our last beta before our RC and GA builds. Be sure to let us know what you think via our 9.0 Beta forum.

Thanks, as always, for your support of PostgreSQL and EnterpriseDB!

Last week it was announced that Microsoft is buying Skype for a reported $8.5 billion in cash. What was not reported in this recent acquisition news cycle is an interesting fact — Skype runs on the open source database PostgreSQL. Since Microsoft has a SQL database of its own – SQL Server – it’s a fair question to ask if this is the beginning of the end of Skype on PostgreSQL?

There’s more than a few ways to speculate on what Microsoft plans to do with the database infrastructure of Skype. Some argue that the prospects for generating revenue off Skype far outweigh having Skype be a poster-child deployment for SQL Server. With making money as the first priority, Microsoft may never be motivated to migrate from PostgreSQL to SQL Server.

Another valid argument is that most users will never know nor care what database is behind Skype as long as it works well. Just look at the current news cycle – there are thousands of articles, blogs, radio and TV outlets reporting on this story. I could not find a single report that mentions PostgreSQL in the context of the Skype acquisition news. Given the low public profile of PostgreSQL at Skype, Microsoft doesn’t need to answer yet about the irony of their new purchase running on open source PostgreSQL and not on Microsoft SQL Server.

What do you think? Is this the beginning of the end of Skype running on PostgreSQL?

“The court system in the US state of Wisconsin is using some 100 PostgreSQL databases for the core application used by 72 county circuit courts. About 3000 users directly connect to the production databases in the various court houses and there are dozens of Web applications receiving millions of hits each day.”

Not too shabby, eh?

Download the new 9.1 beta and give synchronous replication (along with everything else) a try and let us know what you think.

About a year or so ago, surveys (done separately) came out from both InformationWeek and TDWI that showed the top reasons IT professionals switched databases. Listed in both surveys as one of the top five reasons for a switch was “poor data load performance”. With the unprecedented growth in data being experienced by nearly all successful businesses, it should come as no surprise that database pro’s are needing to push a lot of data into databases these days, and the faster they get the data in, the faster a company can use that data to make key business decisions.

One of the upcoming enhancements I’m excited about in Postgres Plus Advanced Server 9.0 is our parallel load capability. We’ve provided a load utility (EDB*Loader) that is an improvement over the PostgreSQL COPY command for a while. However, in Postgres Plus Advanced Server 9.0, our talented engineering team has now enhanced it to load a single table in parallel with multiple input files. By simply adding the clause PARALLEL=TRUE, a DBA or developer can initiate multiple sessions and load a table much faster than with the PostgreSQL COPY command or previous versions of the EDB*Loader utility.

Some preliminary testing our team has done highlights the kind of benefits you’ll soon have. The below graph represents the times for loading a 160 million row table using the general PostgreSQL COPY command vs. Advanced Server’s new parallel loader, which was run on a 4-CPU, quad-core, Intel 64-Bit Linux box with 64GB of RAM, half of which was used for a 32GB RAM disk:

Another way to view the same data:

The load time from the community PostgreSQL COPY command goes from 542 seconds to just 87 seconds with a parallel load using 4 sessions. Going from 4 parallel sessions to 8 or 16 saves even more time.  The number of MB’s per second increases from 42MB/sec for the PostgreSQL COPY command to 262MB/sec for the parallel load (and up to nearly 400MB/sec with more sessions):

The percentage speedup from the PostgreSQL COPY command equates to an impressive 825% (and up to about 1200% if more sessions are added):

Of course, your mileage will vary depending on your hardware, data, and table structure.  But in general, you should experience much faster load times with the new Postgres Plus Advanced Server 9.0 parallel loader.

Another nice enhancement in the EDB*Loader is that it can now load non-empty tables, whereas previous versions could only load – in direct mode – empty tables.

You can download the beta of Postgres Plus Advanced Server 9.0 now and try the new parallel loader (as well as a number of other new features) out for yourself. Be sure to drop us a line and let us know what you think.

Thanks, as always, for your support of PostgreSQL and EnterpriseDB!