SQL Server CDC (Change Data Capture) Existing Data Seed
I have an SQL Server Database with 100's of tables with as many rows in each and I'm turning on CDC for some reporting. Since data was created/inserted BEFORE turning on CDC, the CDC tables are current...
View ArticleChange Tracking across several Tables
I'm looking into providing DB level change tracking within my system. I need to be able to track changes at an entity level, rather than just individual tables.Right now, we have triggers on each of...
View ArticleAutomatically detect table name in MSSQL Server database using stored function
I'm working on implementing Change Data Tracking in a SQL Server database and I set up a single table to test changes. In the past, changes have been made manually, but I'd like to create a way to...
View ArticleCDC - I cannot disable that on the second instance
I have change data capture enabled for most of the tables in one of my databases. Database is hosted on SQL Server instance with version 2016.I have some testing to be done, so I wanted to restore...
View ArticleHow i can get changes in DB postgresql without triggers
I have database, and in this database ~10 users changes rows, how i can get this changes and user who changed, without triggers. I tried read WAL, and there no info about who what change.
View ArticleClarifications on Maintenance jobs & CDC on upgrade to ENTERPRISE EDITION
Raising these on behalf of our DBA team. We are looking at moving to ENTERPRISE SQL SERVER 2022 soon and would need help clarifying a few things:We are looking at a 3-NODE setup:(1) NODE-1: Primary(2)...
View ArticleAnyone using CDC and having performance issues extracting net_changes? We...
So 8 tables are being tracked with CDC "@support_net_changes=1". 7 of them are ok in regards to performance. As some of these tables are only 10-40 million rows and data doesn't change that...
View ArticleSQL Server DB Log keeps growing / CDC enabled database
I've enabled change data capture on a SQL Server database.The database is in full recovery mode.The LOG backups of the database are being done hourly.The capture jobs was running without any issues -...
View ArticleCDC Capture Instance or State to ensure referential integrity
I'm implementing an SSIS/CDC based data warehouse solution and have a question relating to referential integrity on the destination database. As an example, I have INVENTORY_TRANSACTION in one table...
View Articleinvalidating slot "a" because its restart_lsn 6093/6BB6F1B0 exceeds...
we use google cloud sql (version is PostgreSQL 15.5), the config as follows:config, and max_slot_wal_keep_size = -1, but slot always become lost wal status (restart LSN become null) when running for...
View ArticleRunning sp_repldone on a secondary node of an Availability Group with @reset...
I have a two-node SQL Server 2019 Availability Group with a database that has CDC enabled for several tables. The CDC is used to keep track of transactions that have not been captured for population...
View ArticleCDC Capture job fails with 'Cannot insert duplicate key row in object...
When the CDC Capture job runs, it fails with this error:Cannot insert duplicate key row in object cdc.Table_CT' with unique index 'CT Clustered Index.This index is a SQL Server CDC created index on...
View ArticleAWS DMS Task getting failed after completion with error as AlwaysOn BACKUP-ed...
I have created DMS task for migrating data from one RDS MS SQL Server instance to another RDS MS SQL Server instance with "Full load + ongoing replication", and data copied to target instance but...
View ArticleDo old inserts get treated as new CDC events during node bootstrap?
When a new Cassandra node bootstraps and receives data from existing tables for which Change Data Capture (CDC) is enabled, do old inserts get treated as new CDC events? I don't see any reference to...
View ArticleCDC Error - NULL Source Schema and Source Table - Error msg on both enable...
Today I re-enabled CDC on the primary node of an availability group. When the jobs were created the cleanup job failed with the following error:Could not delete change table entries made obsolete by a...
View ArticleSQL Server: Querying Change Data Tables directly?
I am messing around with Change Data Capture on SQL Server.Documentation for table containing the change data (cdc.<capture_instance>_CT) says:We recommend that you do not query the system tables...
View ArticleCDC cleanup job -- history
On SQL Server 2019, I enabled CDC on some tables about 2 days ago. The CDC cleanup job is scheduled to run every day and the retention period is 3 days.When I checked the job history of the cleanup...
View ArticleRDS Mysql replica cdc performance
I have a RDS Mysql which has high read/write volume. ~10k IOPS on an average and peaks up to 16k IOPS.I have another RDS mysql replica of the main. We are using this replica to capture cdc logs and...
View Article_CT records with identical __$seqval
There is a CDC enabled table of the following form:create table dbo.foo ( id int not null primary key, a char(1), b char(1), c char(1), d char(1), e char(1), updated datetime not null default...
View Articleis it a good idea to use GTID for tenant meta data?
i want To build an audit log for tracking user activity using Kafka Connect CDC (Change Data Capture) and MySQL, and set a custom GTID (Global Transaction Identifier) before each transaction, here’s a...
View ArticleClik here to view.
Bug in how CDC jobs record run_duration in sysjobhistory
Normally SQL Agent job history is recorded in msdb.dbo.sysjobhistory with run_duration in the format DD:MM:HH:SS, -- so for example a value of 105 means 1 minute and 5 seconds.However the CDC capture...
View ArticleUnable to alter CDC tables
I have an issue with being unable to alter a CDC table that previously caused us no issue.We run alter statements on our cdc tables to add a column to the tables that we now are starting to face an...
View ArticleTable partitioning for CDC enabled table
Is there any issue you have faced enabling table partitioning for CDC enabled tables?I have done some quick testing and table partitioning for CDC enabled tables is working well in terms of switching...
View ArticleWeird behaviour in CDC SQL Server - could be a bug?
Something I can't understand this behavior and it seems to me bug in SQL Server: I have enabled CDC for table Editorial_schedule_index_cards, now when I update a record from this table with this...
View ArticleHow can I make CDC stop capturing a seemingly infinite amount of agent job...
My most important production server uses Change Data Capture (CDC) a lot, maybe for about 25 tables. The relevant agent job shows well over 1,000 steps as in progress. Because they are in progress,...
View ArticleRestore of Database with CDC Failing
I'm taking a backup of a SQL Server 2019 database with CDC enabled, and trying to restore it to a different instance. I don't need CDC on target database so I don't have the "keep_cdc" option in the...
View ArticleReplace Managed Instance replication with CDC and ADF or Azure Function?
The company I work for put everything on Azure. We use SQL Server replication to move data from one big collection db server (Managed Instance) to our other database servers (say 20 in total).Every day...
View ArticlePeerDB Initial Snapshot Performance Impact on Standby PostgreSQL
I have set up a Change Data Capture (CDC) pipeline using PeerDB to mirror tables from a PostgreSQL standby read replica to ClickHouse.• The PostgreSQL database contains terabytes of data.• The initial...
View ArticleCDC is causing SQL Dumps
Some background. This database was using replication on a previous server version 2012. The replication stopped working. We then upgraded to 2019. I was unable to remove replication objects on the...
View ArticlePostgreSQL Logical Replication Slot Lagging with Empty Tables—Why Does Setup...
I’m running a wal-consumer pod in a PostgreSQL environment that streams WAL logs from a replication slot. However, I noticed that the order in which I set up my services affects whether the consumer...
View Article