tacnode

The Statistics Collector

The Statistics Collector

Tacnode's statistics collector is a subsystem that supports collection and reporting of information about server activity. Presently, the collector can count accesses to tables and indexes in both disk-block and individual-row terms. It also tracks the total number of rows in each table, and information about vacuum and analyze actions for each table. It can also count calls to user-defined functions and the total time spent in each one.

Tacnode also supports reporting dynamic information about exactly what is going on in the system right now, such as the exact command currently being executed by other server processes, and which other connections exist in the system. This facility is independent of the collector process.

Viewing Statistics

Several predefined views, listed in Table 11.1, are available to show the current state of the system. There are also several other views, listed in "monitoring-stats-views-table", available to show the results of statistics collection. Alternatively, one can build custom views using the underlying statistics functions, as discussed in "monitoring-stats-functions".

When using the statistics to monitor collected data, it is important to realize that the information does not update instantaneously. Each individual server process transmits new statistical counts to the collector just before going idle; so a query or transaction still in progress does not affect the displayed totals. Also, the collector itself emits a new report at most once per PGSTAT_STAT_INTERVAL milliseconds (500 ms unless altered while building the server). So the displayed information lags behind actual activity. However, current-query information collected by track_activities is always up-to-date.

Another important point is that when a server process is asked to display any of these statistics, it first fetches the most recent report emitted by the collector process and then continues to use this snapshot for all statistical views and functions until the end of its current transaction. So the statistics will show static information as long as you continue the current transaction. Similarly, information about the current queries of all sessions is collected when any such information is first requested within a transaction, and the same information will be displayed throughout the transaction. This is a feature, not a bug, because it allows you to perform several queries on the statistics and correlate the results without worrying that the numbers are changing underneath you. But if you want to see new results with each query, be sure to do the queries outside any transaction block. Alternatively, you can invoke pg_stat_clear_snapshot(), which will discard the current transaction's statistics snapshot (if any). The next use of statistical information will cause a new snapshot to be fetched.

A transaction can also see its own statistics (as yet untransmitted to the collector) in the views pg_stat_xact_all_tables, pg_stat_xact_sys_tables, pg_stat_xact_user_tables, and pg_stat_xact_user_functions. These numbers do not act as stated above; instead they update continuously throughout the transaction.

Some of the information in the dynamic statistics views shown in Table 11.1 is security restricted. Ordinary users can only see all the information about their own sessions (sessions belonging to a role that they are a member of). In rows about other sessions, many columns will be null. Note, however, that the existence of a session and its general properties such as its sessions user and database are visible to all users. Superusers and members of the built-in role pg_read_all_stats can see all the information about all sessions.

Table Dynamic Statistics Views

View NameDescription
pg_stat_activityOne row per server process, showing information related to the current activity of that process, such as state and current query. See pg_stat_activity for details.

pg_stat_activity

The pg_stat_activity view will have one row per server process, showing information related to the current activity of that process.

Table pg_stat_activity View

Column Type Description

datid oid

OID of the database this backend is connected to

datname name

Name of the database this backend is connected to

pid integer

Process ID of this backend

leader_pid integer

Process ID of the parallel group leader, if this process is a parallel query worker. NULL if this process is a parallel group leader or does not participate in parallel query.

usesysid oid

OID of the user logged into this backend

usename name

Name of the user logged into this backend

application_name text

Name of the application that is connected to this backend

client_addr inet

IP address of the client connected to this backend. If this field is null, it indicates either that the client is connected via a Unix socket on the server machine or that this is an internal process such as autovacuum.

client_hostname text

Host name of the connected client, as reported by a reverse DNS lookup of client_addr. This field will only be non-null for IP connections, and only when "guc-log-hostname" is enabled.

client_port integer

TCP port number that the client is using for communication with this backend, or -1 if a Unix socket is used. If this field is null, it indicates that this is an internal server process.

backend_start timestamp with time zone

Time when this process was started. For client backends, this is the time the client connected to the server.

xact_start timestamp with time zone

Time when this process' current transaction was started, or null if no transaction is active. If the current query is the first of its transaction, this column is equal to the query_start column.

query_start timestamp with time zone

Time when the currently active query was started, or if state is not active, when the last query was started

state_change timestamp with time zone

Time when the state was last changed

wait_event_type text

The type of event for which the backend is waiting, if any; otherwise NULL. See Table 11.3.

wait_event text

Wait event name if backend is currently waiting, otherwise NULL. See Table 11.4 through Table 11.12.

state text

Current overall state of this backend. Possible values are:

  • active: The backend is executing a query.

  • idle: The backend is waiting for a new client command.

  • idle in transaction: The backend is in a transaction, but is not currently executing a query.

  • idle in transaction (aborted): This state is similar to idle in transaction, except one of the statements in the transaction caused an error.

  • fastpath function call: The backend is executing a fast-path function.

  • disabled: This state is reported if "guc-track-activities" is disabled in this backend.

backend_xid xid

Top-level transaction identifier of this backend, if any.

backend_xmin xid

The current backend's xmin horizon.

query_id bigint

Identifier of this backend's most recent query. If state is active this field shows the identifier of the currently executing query. In all other states, it shows the identifier of last query that was executed. Query identifiers are not computed by default so this field will be null unless "guc-compute-query-id" parameter is enabled or a third-party module that computes query identifiers is configured.

query text

Text of this backend's most recent query. If state is active this field shows the currently executing query. In all other states, it shows the last query that was executed. By default the query text is truncated at 1024 bytes; this value can be changed via the parameter "guc-track-activity-query-size".

backend_type text

Type of current backend. Possible types are autovacuum launcher, autovacuum worker, logical replication launcher, logical replication worker, parallel worker, background writer, client backend, checkpointer, archiver, startup, walreceiver, walsender and walwriter. In addition, background workers registered by extensions may have additional types.

Table Wait Event Types

Wait Event TypeDescription
ActivityThe server process is idle. This event type indicates a process waiting for activity in its main processing loop. wait_event will identify the specific wait point; see Table 11.4.
BufferPinThe server process is waiting for exclusive access to a data buffer. Buffer pin waits can be protracted if another process holds an open cursor that last read data from the buffer in question. See Table 11.5.
ClientThe server process is waiting for activity on a socket connected to a user application. Thus, the server expects something to happen that is independent of its internal processes. wait_event will identify the specific wait point; see Table 11.6.
ExtensionThe server process is waiting for some condition defined by an extension module. See Table 11.7.
IOThe server process is waiting for an I/O operation to complete. wait_event will identify the specific wait point; see Table 11.8.
IPCThe server process is waiting for some interaction with another server process. wait_event will identify the specific wait point; see Table 11.9.
LockThe server process is waiting for a heavyweight lock. Heavyweight locks, also known as lock manager locks or simply locks, primarily protect SQL-visible objects such as tables. However, they are also used to ensure mutual exclusion for certain internal operations such as relation extension. wait_event will identify the type of lock awaited; see Table 11.10.
LWLockThe server process is waiting for a lightweight lock. Most such locks protect a particular data structure in shared memory. wait_event will contain a name identifying the purpose of the lightweight lock. (Some locks have specific names; others are part of a group of locks each with a similar purpose.) See Table 11.11.
TimeoutThe server process is waiting for a timeout to expire. wait_event will identify the specific wait point; see Table 11.12.

Table Wait Events of Type Activity

Activity Wait EventDescription
ArchiverMainWaiting in main loop of archiver process.
AutoVacuumMainWaiting in main loop of autovacuum launcher process.
BgWriterHibernateWaiting in background writer process, hibernating.
BgWriterMainWaiting in main loop of background writer process.
CheckpointerMainWaiting in main loop of checkpointer process.
LogicalApplyMainWaiting in main loop of logical replication apply process.
LogicalLauncherMainWaiting in main loop of logical replication launcher process.
PgStatMainWaiting in main loop of statistics collector process.
RecoveryWalStreamWaiting in main loop of startup process for WAL to arrive, during streaming recovery.
SysLoggerMainWaiting in main loop of syslogger process.
WalReceiverMainWaiting in main loop of WAL receiver process.
WalSenderMainWaiting in main loop of WAL sender process.
WalWriterMainWaiting in main loop of WAL writer process.

Table Wait Events of Type BufferPin

BufferPin Wait EventDescription
BufferPinWaiting to acquire an exclusive pin on a buffer.

Table Wait Events of Type Client

Client Wait EventDescription
ClientReadWaiting to read data from the client.
ClientWriteWaiting to write data to the client.
GSSOpenServerWaiting to read data from the client while establishing a GSSAPI session.
LibPQWalReceiverConnectWaiting in WAL receiver to establish connection to remote server.
LibPQWalReceiverReceiveWaiting in WAL receiver to receive data from remote server.
SSLOpenServerWaiting for SSL while attempting connection.
WalSenderWaitForWALWaiting for WAL to be flushed in WAL sender process.
WalSenderWriteDataWaiting for any activity when processing replies from WAL receiver in WAL sender process.

Table Wait Events of Type Extension

Extension Wait EventDescription
ExtensionWaiting in an extension.

Table Wait Events of Type IO

IO Wait EventDescription
BaseBackupReadWaiting for base backup to read from a file.
BufFileReadWaiting for a read from a buffered file.
BufFileWriteWaiting for a write to a buffered file.
BufFileTruncateWaiting for a buffered file to be truncated.
ControlFileReadWaiting for a read from the pg_control file.
ControlFileSyncWaiting for the pg_control file to reach durable storage.
ControlFileSyncUpdateWaiting for an update to the pg_control file to reach durable storage.
ControlFileWriteWaiting for a write to the pg_control file.
ControlFileWriteUpdateWaiting for a write to update the pg_control file.
CopyFileReadWaiting for a read during a file copy operation.
CopyFileWriteWaiting for a write during a file copy operation.
DSMFillZeroWriteWaiting to fill a dynamic shared memory backing file with zeroes.
DataFileExtendWaiting for a relation data file to be extended.
DataFileFlushWaiting for a relation data file to reach durable storage.
DataFileImmediateSyncWaiting for an immediate synchronization of a relation data file to durable storage.
DataFilePrefetchWaiting for an asynchronous prefetch from a relation data file.
DataFileReadWaiting for a read from a relation data file.
DataFileSyncWaiting for changes to a relation data file to reach durable storage.
DataFileTruncateWaiting for a relation data file to be truncated.
DataFileWriteWaiting for a write to a relation data file.
LockFileAddToDataDirReadWaiting for a read while adding a line to the data directory lock file.
LockFileAddToDataDirSyncWaiting for data to reach durable storage while adding a line to the data directory lock file.
LockFileAddToDataDirWriteWaiting for a write while adding a line to the data directory lock file.
LockFileCreateReadWaiting to read while creating the data directory lock file.
LockFileCreateSyncWaiting for data to reach durable storage while creating the data directory lock file.
LockFileCreateWriteWaiting for a write while creating the data directory lock file.
LockFileReCheckDataDirReadWaiting for a read during recheck of the data directory lock file.
LogicalRewriteCheckpointSyncWaiting for logical rewrite mappings to reach durable storage during a checkpoint.
LogicalRewriteMappingSyncWaiting for mapping data to reach durable storage during a logical rewrite.
LogicalRewriteMappingWriteWaiting for a write of mapping data during a logical rewrite.
LogicalRewriteSyncWaiting for logical rewrite mappings to reach durable storage.
LogicalRewriteTruncateWaiting for truncate of mapping data during a logical rewrite.
LogicalRewriteWriteWaiting for a write of logical rewrite mappings.
RelationMapReadWaiting for a read of the relation map file.
RelationMapSyncWaiting for the relation map file to reach durable storage.
RelationMapWriteWaiting for a write to the relation map file.
ReorderBufferReadWaiting for a read during reorder buffer management.
ReorderBufferWriteWaiting for a write during reorder buffer management.
ReorderLogicalMappingReadWaiting for a read of a logical mapping during reorder buffer management.
ReplicationSlotReadWaiting for a read from a replication slot control file.
ReplicationSlotRestoreSyncWaiting for a replication slot control file to reach durable storage while restoring it to memory.
ReplicationSlotSyncWaiting for a replication slot control file to reach durable storage.
ReplicationSlotWriteWaiting for a write to a replication slot control file.
SLRUFlushSyncWaiting for SLRU data to reach durable storage during a checkpoint or database shutdown.
SLRUReadWaiting for a read of an SLRU page.
SLRUSyncWaiting for SLRU data to reach durable storage following a page write.
SLRUWriteWaiting for a write of an SLRU page.
SnapbuildReadWaiting for a read of a serialized historical catalog snapshot.
SnapbuildSyncWaiting for a serialized historical catalog snapshot to reach durable storage.
SnapbuildWriteWaiting for a write of a serialized historical catalog snapshot.
TimelineHistoryFileSyncWaiting for a timeline history file received via streaming replication to reach durable storage.
TimelineHistoryFileWriteWaiting for a write of a timeline history file received via streaming replication.
TimelineHistoryReadWaiting for a read of a timeline history file.
TimelineHistorySyncWaiting for a newly created timeline history file to reach durable storage.
TimelineHistoryWriteWaiting for a write of a newly created timeline history file.
TwophaseFileReadWaiting for a read of a two phase state file.
TwophaseFileSyncWaiting for a two phase state file to reach durable storage.
TwophaseFileWriteWaiting for a write of a two phase state file.
WALBootstrapSyncWaiting for WAL to reach durable storage during bootstrapping.
WALBootstrapWriteWaiting for a write of a WAL page during bootstrapping.
WALCopyReadWaiting for a read when creating a new WAL segment by copying an existing one.
WALCopySyncWaiting for a new WAL segment created by copying an existing one to reach durable storage.
WALCopyWriteWaiting for a write when creating a new WAL segment by copying an existing one.
WALInitSyncWaiting for a newly initialized WAL file to reach durable storage.
WALInitWriteWaiting for a write while initializing a new WAL file.
WALReadWaiting for a read from a WAL file.
WALSenderTimelineHistoryReadWaiting for a read from a timeline history file during a walsender timeline command.
WALSyncWaiting for a WAL file to reach durable storage.
WALSyncMethodAssignWaiting for data to reach durable storage while assigning a new WAL sync method.
WALWriteWaiting for a write to a WAL file.
LogicalChangesReadWaiting for a read from a logical changes file.
LogicalChangesWriteWaiting for a write to a logical changes file.
LogicalSubxactReadWaiting for a read from a logical subxact file.
LogicalSubxactWriteWaiting for a write to a logical subxact file.

Table Wait Events of Type IPC

IPC Wait EventDescription
AppendReadyWaiting for subplan nodes of an Append plan node to be ready.
BackendTerminationWaiting for the termination of another backend.
BackupWaitWalArchiveWaiting for WAL files required for a backup to be successfully archived.
BgWorkerShutdownWaiting for background worker to shut down.
BgWorkerStartupWaiting for background worker to start up.
BtreePageWaiting for the page number needed to continue a parallel B-tree scan to become available.
BufferIOWaiting for buffer I/O to complete.
CheckpointDoneWaiting for a checkpoint to complete.
CheckpointStartWaiting for a checkpoint to start.
ExecuteGatherWaiting for activity from a child process while executing a Gather plan node.
HashBatchAllocateWaiting for an elected Parallel Hash participant to allocate a hash table.
HashBatchElectWaiting to elect a Parallel Hash participant to allocate a hash table.
HashBatchLoadWaiting for other Parallel Hash participants to finish loading a hash table.
HashBuildAllocateWaiting for an elected Parallel Hash participant to allocate the initial hash table.
HashBuildElectWaiting to elect a Parallel Hash participant to allocate the initial hash table.
HashBuildHashInnerWaiting for other Parallel Hash participants to finish hashing the inner relation.
HashBuildHashOuterWaiting for other Parallel Hash participants to finish partitioning the outer relation.
HashGrowBatchesAllocateWaiting for an elected Parallel Hash participant to allocate more batches.
HashGrowBatchesDecideWaiting to elect a Parallel Hash participant to decide on future batch growth.
HashGrowBatchesElectWaiting to elect a Parallel Hash participant to allocate more batches.
HashGrowBatchesFinishWaiting for an elected Parallel Hash participant to decide on future batch growth.
HashGrowBatchesRepartitionWaiting for other Parallel Hash participants to finish repartitioning.
HashGrowBucketsAllocateWaiting for an elected Parallel Hash participant to finish allocating more buckets.
HashGrowBucketsElectWaiting to elect a Parallel Hash participant to allocate more buckets.
HashGrowBucketsReinsertWaiting for other Parallel Hash participants to finish inserting tuples into new buckets.
LogicalSyncDataWaiting for a logical replication remote server to send data for initial table synchronization.
LogicalSyncStateChangeWaiting for a logical replication remote server to change state.
MessageQueueInternalWaiting for another process to be attached to a shared message queue.
MessageQueuePutMessageWaiting to write a protocol message to a shared message queue.
MessageQueueReceiveWaiting to receive bytes from a shared message queue.
MessageQueueSendWaiting to send bytes to a shared message queue.
ParallelBitmapScanWaiting for parallel bitmap scan to become initialized.
ParallelCreateIndexScanWaiting for parallel CREATE INDEX workers to finish heap scan.
ParallelFinishWaiting for parallel workers to finish computing.
ProcArrayGroupUpdateWaiting for the group leader to clear the transaction ID at end of a parallel operation.
ProcSignalBarrierWaiting for a barrier event to be processed by all backends.
PromoteWaiting for standby promotion.
RecoveryConflictSnapshotWaiting for recovery conflict resolution for a vacuum cleanup.
RecoveryConflictTablespaceWaiting for recovery conflict resolution for dropping a tablespace.
RecoveryPauseWaiting for recovery to be resumed.
ReplicationOriginDropWaiting for a replication origin to become inactive so it can be dropped.
ReplicationSlotDropWaiting for a replication slot to become inactive so it can be dropped.
SafeSnapshotWaiting to obtain a valid snapshot for a READ ONLY DEFERRABLE transaction.
SyncRepWaiting for confirmation from a remote server during synchronous replication.
WalReceiverExitWaiting for the WAL receiver to exit.
WalReceiverWaitStartWaiting for startup process to send initial data for streaming replication.
XactGroupUpdateWaiting for the group leader to update transaction status at end of a parallel operation.

Table Wait Events of Type Lock

Lock Wait EventDescription
advisoryWaiting to acquire an advisory user lock.
extendWaiting to extend a relation.
frozenidWaiting to update pg_database.datfrozenxid and pg_database.datminmxid.
objectWaiting to acquire a lock on a non-relation database object.
pageWaiting to acquire a lock on a page of a relation.
relationWaiting to acquire a lock on a relation.
spectokenWaiting to acquire a speculative insertion lock.
transactionidWaiting for a transaction to finish.
tupleWaiting to acquire a lock on a tuple.
userlockWaiting to acquire a user lock.
virtualxidWaiting to acquire a virtual transaction ID lock.

Table Wait Events of Type LWLock

LWLock Wait EventDescription
AddinShmemInitWaiting to manage an extension's space allocation in shared memory.
AutoFileWaiting to update the postgresql.auto.conf file.
AutovacuumWaiting to read or update the current state of autovacuum workers.
AutovacuumScheduleWaiting to ensure that a table selected for autovacuum still needs vacuuming.
BackgroundWorkerWaiting to read or update background worker state.
BtreeVacuumWaiting to read or update vacuum-related information for a B-tree index.
BufferContentWaiting to access a data page in memory.
BufferMappingWaiting to associate a data block with a buffer in the buffer pool.
CheckpointerCommWaiting to manage fsync requests.
CommitTsWaiting to read or update the last value set for a transaction commit timestamp.
CommitTsBufferWaiting for I/O on a commit timestamp SLRU buffer.
CommitTsSLRUWaiting to access the commit timestamp SLRU cache.
ControlFileWaiting to read or update the pg_control file or create a new WAL file.
DynamicSharedMemoryControlWaiting to read or update dynamic shared memory allocation information.
LockFastPathWaiting to read or update a process' fast-path lock information.
LockManagerWaiting to read or update information about “heavyweight” locks.
LogicalRepWorkerWaiting to read or update the state of logical replication workers.
MultiXactGenWaiting to read or update shared multixact state.
MultiXactMemberBufferWaiting for I/O on a multixact member SLRU buffer.
MultiXactMemberSLRUWaiting to access the multixact member SLRU cache.
MultiXactOffsetBufferWaiting for I/O on a multixact offset SLRU buffer.
MultiXactOffsetSLRUWaiting to access the multixact offset SLRU cache.
MultiXactTruncationWaiting to read or truncate multixact information.
NotifyBufferWaiting for I/O on a NOTIFY message SLRU buffer.
NotifyQueueWaiting to read or update NOTIFY messages.
NotifyQueueTailWaiting to update limit on NOTIFY message storage.
NotifySLRUWaiting to access the NOTIFY message SLRU cache.
OidGenWaiting to allocate a new OID.
OldSnapshotTimeMapWaiting to read or update old snapshot control information.
ParallelAppendWaiting to choose the next subplan during Parallel Append plan execution.
ParallelHashJoinWaiting to synchronize workers during Parallel Hash Join plan execution.
ParallelQueryDSAWaiting for parallel query dynamic shared memory allocation.
PerSessionDSAWaiting for parallel query dynamic shared memory allocation.
PerSessionRecordTypeWaiting to access a parallel query's information about composite types.
PerSessionRecordTypmodWaiting to access a parallel query's information about type modifiers that identify anonymous record types.
PerXactPredicateListWaiting to access the list of predicate locks held by the current serializable transaction during a parallel query.
PredicateLockManagerWaiting to access predicate lock information used by serializable transactions.
ProcArrayWaiting to access the shared per-process data structures (typically, to get a snapshot or report a session's transaction ID).
RelationMappingWaiting to read or update a pg_filenode.map file (used to track the filenode assignments of certain system catalogs).
RelCacheInitWaiting to read or update a pg_internal.init relation cache initialization file.
ReplicationOriginWaiting to create, drop or use a replication origin.
ReplicationOriginStateWaiting to read or update the progress of one replication origin.
ReplicationSlotAllocationWaiting to allocate or free a replication slot.
ReplicationSlotControlWaiting to read or update replication slot state.
ReplicationSlotIOWaiting for I/O on a replication slot.
SerialBufferWaiting for I/O on a serializable transaction conflict SLRU buffer.
SerializableFinishedListWaiting to access the list of finished serializable transactions.
SerializablePredicateListWaiting to access the list of predicate locks held by serializable transactions.
SerializableXactHashWaiting to read or update information about serializable transactions.
SerialSLRUWaiting to access the serializable transaction conflict SLRU cache.
SharedTidBitmapWaiting to access a shared TID bitmap during a parallel bitmap index scan.
SharedTupleStoreWaiting to access a shared tuple store during parallel query.
ShmemIndexWaiting to find or allocate space in shared memory.
SInvalReadWaiting to retrieve messages from the shared catalog invalidation queue.
SInvalWriteWaiting to add a message to the shared catalog invalidation queue.
SubtransBufferWaiting for I/O on a sub-transaction SLRU buffer.
SubtransSLRUWaiting to access the sub-transaction SLRU cache.
SyncRepWaiting to read or update information about the state of synchronous replication.
SyncScanWaiting to select the starting location of a synchronized table scan.
TablespaceCreateWaiting to create or drop a tablespace.
TwoPhaseStateWaiting to read or update the state of prepared transactions.
WALBufMappingWaiting to replace a page in WAL buffers.
WALInsertWaiting to insert WAL data into a memory buffer.
WALWriteWaiting for WAL buffers to be written to disk.
WrapLimitsVacuumWaiting to update limits on transaction id and multixact consumption.
XactBufferWaiting for I/O on a transaction status SLRU buffer.
XactSLRUWaiting to access the transaction status SLRU cache.
XactTruncationWaiting to execute pg_xact_status or update the oldest transaction ID available to it.
XidGenWaiting to allocate a new transaction ID.

Table Wait Events of Type Timeout

Timeout Wait EventDescription
BaseBackupThrottleWaiting during base backup when throttling activity.
PgSleepWaiting due to a call to pg_sleep or a sibling function.
RecoveryApplyDelayWaiting to apply WAL during recovery because of a delay setting.
RecoveryRetrieveRetryIntervalWaiting during recovery when WAL data is not available from any source (pg_wal, archive or stream).
VacuumDelayWaiting in a cost-based vacuum delay point.

Here is an example of how wait events can be viewed:

SELECT pid, wait_event_type, wait_event FROM pg_stat_activity WHERE wait_event is NOT NULL;
pid  | wait_event_type | wait_event
------+----------------+------------
2540 | Lock            | relation
6644 | LWLock          | ProcArray
(2 rows)

On this page