pgbouncer

PostgreSQL 如何查找TOP SQL (例如IO消耗最高的SQL) (包含SQL优化内容)

北城余情 提交于 2020-09-24 13:25:55
目录 背景 一、安装pg_stat_statements 二、加载pg_stat_statements模块 三、配置pg_stat_statements采样参数 四、创建pg_stat_statements extension 五、分析TOP SQL pg_stat_statements输出内容介绍 最耗IO SQL 最耗时 SQL 响应时间抖动最严重 SQL 最耗共享内存 SQL 最耗临时空间 SQL 六、重置统计信息 慢SQL到底慢在哪里? 七、慢SQL、TOP SQL优化示例 八、常见优化点汇总 参考 背景 数据库是较大型的应用,对于繁忙的数据库,需要消耗大量的内存、CPU、IO、网络资源。 SQL优化是数据库优化的手段之一,优化什么SQL效果最佳呢?首先要了解最耗费资源的SQL,即TOP SQL。 从哪里可以了解数据库的资源都被哪些SQL消耗掉了呢? 资源分为多个维度,CPU,内存,IO等。如何能了解各个维度层面的TOP SQL呢? pg_stat_statements插件可以用于统计数据库的资源开销,分析TOP SQL。 一、安装pg_stat_statements 如果您使用的是云数据库,跳过安装,到create extension 部分。 pg_stat_statements是PostgreSQL的核心插件之一。可以在编译PostgreSQL时安装,也可以单独安装。

How does pgbouncer behave when transaction pooling is enabled and a single statement is issued?

蓝咒 提交于 2020-07-09 19:29:04
问题 I'm using pgbouncer on a web app and most threads begin with a BEGIN and end with a COMMIT or a ROLLBACK, so we're using transaction pooling and everything is fine. However, we also have some processes which don't use transactions: instead, they just issue commands one after another. I believe that, under transaction pooling, every command is a transaction by itself, just the way it is when you're connected directly to the server, and perhaps every command is getting a different connection

What's the worst case scenario if one runs individual queries when pgbouncer has pool_mode=transaction?

感情迁移 提交于 2020-06-13 03:52:19
问题 Let's assume following config for pgbouncer : pool_mode=transaction reset_query=discard all reset_query_always=0 If I have a connection that does begin transaction ... commit transaction then it's pretty simple because during the transaction is open, pgbouncer will reserve the backend connection for this client connection only. However, if a client app instead sends select select pg_advisory_lock(123); begin transaction isolation level serializable ... commit transaction select select pg

postgresql相关开源软件及架构简介

风格不统一 提交于 2020-04-27 18:13:36
1、PgBouncer PG数据库的一个轻量级连接池工具,功能及特点如下: 1)缓存后端PG数据库的连接,当前端应用请求时,分配连接池中的连接给应用,从而充分利用了系统资源。 2)允许应用创建比连接池更多的连接,并未应用连接提供排队等候功能,这样既充分利用了资源,也保证了应用端的高并发。 3)可以对应用连接进行限制,这样,可以起到系统稳定和安全作用。 4)PgBouncer的通信效率非常高。 5)PgBouncer通过C语言实现,短小精悍,消耗很少的系统资源。 6)PgBouncer支持三种连接池模式:会话级别,事务级别,语句级别。 2、Slony-I PG数据库的一款逻辑复制工具,功能及特点如下: 1)Slony-I支持级联复制,一个节点为订阅者的同时,也可以作为下一级的数据提供者,数据的原始生产者对数据的修改,会在各级订阅者之间传播。 2)Slony-I集群中的各节点上都需要有slon守护进程,以处理复制中的事件,例如:配置事件,同步事件。 3)Slony-I可以通过slonik工具进行管理和配置,其具备处理脚本的能力。 4)Slony-I具备强大逻辑复制功能的同时,也有些限制,例如:大对象的复制,DDL复制,用户权限复制等。 3、Bucardo PG数据库的一款具备双向同步功能的工具,功能及特点如下: 1)可以为PG实现双master架构,5

Postgres - ERROR: prepared statement “S_1” already exists

不想你离开。 提交于 2020-02-26 07:54:29
问题 When executing batch queries via JDBC to pgbouncer, I get the following error: org.postgresql.util.PSQLException: ERROR: prepared statement "S_1" already exists I've found bug reports around the web, but they all seem to deal with Postgres 8.3 or below, whereas we're working with Postgres 9. Here's the code that triggers the error: this.getJdbcTemplate().update("delete from xx where username = ?", username); this.getJdbcTemplate().batchUpdate( "INSERT INTO xx(a, b, c, d, e) " + "VALUES (?, ?,

Spring Boot 2.1 App without HikariCP Connection pooler

柔情痞子 提交于 2019-12-24 21:16:05
问题 I want to use my Spring Boot 2.1 App without any connection pooler (HikariCP in this case) since the default pooler is HikariCP ! How should I go ahead and implement this ? The use case is I want to use a common db pooler (pgBouncer) for all my application instances and other applications ! I cannot achieve this when each Spring Boot app runs with its own implicit connection pooler(HikariCP). How should I implement this use case ? Is this a better solution for common database connection

psycopg2 + pgbouncer. Async mode with gevent error

試著忘記壹切 提交于 2019-12-21 21:26:43
问题 I've got an application psycopg2 + pgbouncer + gevent. Asynchronous application, ie one process serves multiple requests. Asynchronous access to database appeared in the latest version of psycopg2, if rather then 2.2. But just in this release introduces bug for which there is a lot of disconnections from pgbouncer. In the pgbouncer logs there are entries: 2011-10-04 12:16:38.972 4590 LOG C-0x1b3f490: database/user@10.58.65.143:43849 login successful: db=database user=user 2011-10-04 12:16:38

Not able to open connection from PgBouncer (Connection attempt timed out)

橙三吉。 提交于 2019-12-11 08:49:29
问题 We are using PgBouncer as the middle ware for connection pooling for our spring-based application. Within the application, we are using HikariCP for application-level connection pooling. We have set maxLifeTime value equal to one hour for HikariCP and idleTimeOut is set to 30 seconds. I have observed that we are getting this error when app get a request after a long idle time: Caused by: org.postgresql.util.PSQLException: Connection attempt timed out. at org.postgresql.Driver$ConnectThread

Find source of query through pgbouncer

别说谁变了你拦得住时间么 提交于 2019-12-11 06:38:12
问题 I am trying to figure out as much as I can about what host a particular type of update query is coming from. My database is PostgreSQL 9.0.23 locally running pgbouncer for connection pooling. The pooling type is transaction pooling. A trigger in the database saving information from pg_stat_activity tells me about the pgbouncer connection, but doesn't help me figure out who is behind that. The database in question is a production database under load. Therefore removing pgbouncer is not an