As a followup to my scaling talk, I have written a draft of another talk, The Future of Postgres Sharding. It starts by explaining the advantages of sharding as a scaling option. It then covers future enhancements to individual Postgres features that, while useful on their own, could be combined to provide a powerful built-in Postgres sharding capability. I am hopeful this talk will help guide the community discussion of implementing built-in sharding.
I just returned from two weeks in Russia, and I am happy to report that Postgres is experiencing strong growth there. I have regularly complained that Russian Postgres adoption was lagging, but the sanctions have tipped the scales and moved Russia into Postgres hyper-adoption mode. New activities include:
- Postgres Day in Saint Petersburg in July
- Moscow meetings in September and October with 180 attendees
- October Postgres Day in Saint Petersburg
- Strong Postgres presence and booth at Highload++
- 2015 conferences in Moscow and Saint Petersburg
- Postgres community development process
- Comparison of Postgres to proprietary databases
- Future direction of Postgres
Due to the increased popularity of Postgres, conference organizers are more confident about future conferences and are announcing their conference dates earlier, perhaps also to attract speakers. These are the conferences already announced for 2015:
- February, FOSDEM, Brussels (dedicated Postgres day)
- February, PgConf.Russia, Moscow
- February, SCALE, Los Angeles (dedicated Postgres day)
- March, PGConf US, New York City
- June, PGCon, Ottawa
- July, PG Day'15 Russia, Saint Petersburg
- September, PGDay Campinas 2015, Campinas, Brazil
- October, PostgreSQL Conference Europe, Vienna
- November, pgbr, Porto Alegre, Brazil
I don't remember ever seeing this many Postgres conferences scheduled this far in advance. (Most of these are listed on the Postgres Wiki Events page.) If you know of any other scheduled 2015 conferences, please post the details as a comment below.Post a Comment
PostgreSQL Conference Europe has just finished and I delivered two new presentations at the conference (a first for me). Postgres Scaling Opportunities summarizes scaling options, and Flexible Indexing with Postgres summarizes indexing options. I have been submitting these presentations to conferences for many months but this is the first conference to have chosen them. The talks were well attended and generated positive feedback.
The talks are more surveys rather than focussing on specific technologies. It might be beneficial for future conferences to have more survey-oriented talks, as many attendees are not Postgres experts.
I am now heading to Russia for two weeks, presenting in St. Petersburg and Moscow. I will also be visiting with our server developers in Russia who created some very significant features for Postgres 9.4.Post a Comment
If you are near Philadelphia, you are invited to attend the 2014 Postgres Pool Party at my home:
- When: Saturday, July 26, 2pm to 7pm
- Where: my home in Newtown Square, Pennsylvania (directions)
- What: barbecue, swimming, and good conversation
All Postgres users, developers, and groupies are invited, including their families. Please RSVP via email by July 20.Post a Comment
While I have been to many Postgres user conferences, this felt like my first corporate Postgres conference. Presenters from multinational banks Goldman Sachs and Morgan Stanley explained how and why they use Postgres in their organizations. These are trend-setting organizations, and their public embrace of Postgres will have lasting benefits.
In fact, one hallway discussion was how to enable large organizations like these, particularly those with significant legal and regulatory requirements, to work with the Postgres community. Some companies have employees post from non-company email accounts like Gmail, while others contract with consulting companies to work with the Postgres community on their behalf. Unfortunately, neither of these approaches have the companies working with the community openly. There were also many requests for improved auditing.
I just attended and presented at ConFoo, where I always learn new things. This year, the best talk I attended was by our own Magnus Hagander. Called Integrated Cache Invalidation for Better Hit Rates, it addresses the difficult task of controlling web page cache refresh by providing a reliable mechanism for invalidating the cache when the database changes. It shows how to use triggers and pgq to communicate changes to a Varnish web frontend.
Caching in front of a database is often risky. However, with triggers, pgq, and Varnish, proper invalidation can be accomplished, dramatically increasing the scalability of database-backed websites.Post a Comment
EnterpriseDB decided to put me on the spot recently by inviting queries from the community via social channels as I prepared for teaching a new PostgreSQL course. It’s a familiar spot for me. I regularly field questions about PostgreSQL given my role in the community and I was happy to get a question through Twitter.
The question, which came from a Twitter user in Sweden, was: “Is native lightweight data change tracking on the roadmap? (Similar to what MSSQL is offering today.)”
MS SQL Server enables applications to determine DML changes (insert, update and delete) operations through change tracking and change data capture. In PostgreSQL, we have three solutions related to DML change tracking.
The first is streaming replication, which ships a binary change log to a standby server. However, since it is binary, it isn’t easy to process externally. Second is Slony, which uses triggers to accumulate DML changes into tables that are periodically shipped to standby servers. Slony allows per-table replication.
Finally, the third option, which the community is working on, is logical changeset logging that can be used by any application wanting to track database changes. This will be used eventually to implement more fine-grained replication solutions. The following link will take you to an overview of the community PostgreSQL project, which should be completed in the next few years though the logical changeset part may be done much sooner: http://bit.ly/1hoDSb0
Hope this answers the question. If you have others, please reach out to us via Twitter, Facebook or Google+. Find the buttons for these social outlets, and LinkedIn, on our homepage: http://bit.ly/1mHkwDi
Bruce Momjian is a Senior Database Architect at EnterpriseDB and co-founder of the PostgreSQL Global Development Group.Tweet
As the creator of the initial major release notes, I am aware of the many changes that go into every release. What I often cannot see is the pattern of feature growth that the Postgres community accomplishes on a yearly basis.
This keynote presentation from PGConf EU contained a graphic that gave me a new perspective. The presentation by EnterpriseDB's Keith Alsheimer includes valuable research and quotes, but slide 18 is the image that caught my eye. It takes Postgres features and groups them into three areas:
- Easy to use / deploy
- High-end enterprise requirements
- New workloads / platforms
With examples under each section, the diagram clearly shows Postgres development targeting three different segments. With this visual, it is easier for me to see which segment a new feature targets, and when a feature might benefit one segment while harming another.
Originally released in 2006, pgpool-II is the Swiss Army Knife of Postgres middleware tools, allowing connection pooling, replication, load balancing, and parallel query. With the ascendency of PgBouncer and streaming replication, its role has changed, but its position as an intermediary between database clients and servers has allowed it to retain a unique and valuable toolset. For example, pgpool's ability to detect and redirect read-only queries is used in many read-balancing solutions, including EnterpriseDB's cloud database offering. It is also uniquely positioned to handle server fail-over by redirecting client requests.
Unfortunately, with all these features, pgpool's complexity has hampered its adoption. Fortunately, a new focus on reliability has greatly increased its usefulness. Six months ago, EnterpriseDB employee and now pgpool committer Muhammad Usama started working on pgpool stability by restructuring the code and importing Postgres's memory manager into pgpool. Lead pgpool developer Tatsuo Ishii is now using the Coverity software scanner to find and fix pgpool bugs. Even VMWare's Heikki Linnakangas has joined to help. A new regression test suite has also been developed. Pgpool developers are even considering removing parallel query to simplify the code and help focus development efforts.
These are exciting times for pgpool. The number of fixes in November alone is massive, and a new release with these fixes is already being planned. As middleware, pgpool is in a unique position to provide powerful database features, and it is great to see it growing in that role.Post a Comment