none
CDC for oracle Network requirements RRS feed

  • Question

  • Dear,

    I would like to inform about the requirements, specially related to the network. What
    if Oracle and SQL Server or on separated locations across the world, will an
    ISDN connection be enough or do we need fibernet. Will this be sufficient for
    the connection & datatransfer between CDC services and LogMiner? we expect around 10 millions (changed) records a day.

    What if there are more transactions than we can process, could be related to the slow network connection?

    Is there a kind of buffer for when transactions are comming in, while we are processing them?

    Thanks,

    Bram

    Wednesday, August 8, 2012 9:59 AM

Answers

  • Yes, Bram,

    the network stability is an absolutely crucial point.

    CDC is not worth it over an unreliable network and no solution would be better because nothing will make the data shuttle over a broken or hiccuping network link.

    You 'll need some redundancy and the capability to restart a failed transaction in any case.

    May be this is an opportunity to use SQL Azure, this way you get the best of the two worlds - reliability and data access on demand in one place


    Arthur My Blog

    • Marked as answer by BramDe Friday, August 10, 2012 8:29 AM
    Thursday, August 9, 2012 3:36 PM
    Moderator

All replies

  • Dear,

    I would
    like to inform about the requirements, specially related to the network. What
    if Oracle and SQL Server or on separated locations across the world, will an
    ISDN connection be enough or do we need fibernet. Will this be sufficient for
    the connection & datatransfer between CDC services and LogMiner?

    What if there are more transaction than we can process, maybe related to the slow
    network connection?

    What will happen when there are problems with the logfile from oracle

    Thanks,

    Bram

    • Edited by BramDe Wednesday, August 8, 2012 7:03 AM
    • Moved by Shulei Chen Thursday, August 9, 2012 2:03 AM (From:SQL Server Tools)
    • Merged by Eileen Zhao Wednesday, August 15, 2012 9:58 AM duplicate
    Tuesday, August 7, 2012 2:39 PM
  • Hi Bram,

    Change Data Capture (CDC) is new in SQL Server 2012 for Oracle which allows you to continually copy data from Oracle to SQL Server. This feature is helpful in two common business scenarios:

    1. Migrating from Oracle database to SQL Server 2012 systems – you can synchronize critical tables throughout the migration timeframe by copying changes made to the Oracle tables into the SQL Server 2012 tables.
    2. Data Warehousing using a SQL Server 2012 system. Changes can be capture from your Oracle database and sent to SQL Server 2012 databases to keep the data warehouse in sync.

    For more details, please refer to the following resources:

    Besides, this is a SQL Server Integration Services (SSIS) related question, I would like to suggest you to open thread in SSIS forums for getting better help.


    Best Regards,
    Ray Chen

    Wednesday, August 8, 2012 7:45 AM
  • Thanks for the response and the links Ray.

    I have found a lot of info, but nothing related to the network requirements.

    I will try the SSIS forum.

    thx

    Wednesday, August 8, 2012 9:51 AM
  • Bram,

    do you use SQL Server 2012 SSIS with CDC?

    10 mil a day is huge, you need the best network speed possible, you need to engage vendors to demonstrate robustness and reliability.

    In my experience the biggest dependency is where in the world the communication points are. 

    Local buffering may need to be implemented in a special form.

    I was on one project where we implemented a special middle-tier solution from BEA - Tuxedo - it is capable of intelligently receiving or sending data in form of load balancing and enduring transaction atomicity and completeness. 


    Arthur My Blog

    Wednesday, August 8, 2012 2:23 PM
    Moderator
  • Hi Arthur,

    Thanks for the reply. We are currently looking for the best solution. The communication is ment to populate a DWH. Yes, we are using SQL 2012. If the network depedency is that crucial, (network connection between the two countries is not that good), I doubt that CDC will be the best solution. Maybe we should ask the source to take an incremental extract and send it over, to process?

    I will have a look to this Tuxedo solution, but as it is ment for a DWH, maybe other solutions are better?

    thanks, Bram

    Thursday, August 9, 2012 7:24 AM
  • Hello,

    Check out the video in the link below. It may help you.

    http://technet.microsoft.com/en-us/sqlserver/jj218898.aspx

    Thursday, August 9, 2012 9:04 AM
  • Yes, Bram,

    the network stability is an absolutely crucial point.

    CDC is not worth it over an unreliable network and no solution would be better because nothing will make the data shuttle over a broken or hiccuping network link.

    You 'll need some redundancy and the capability to restart a failed transaction in any case.

    May be this is an opportunity to use SQL Azure, this way you get the best of the two worlds - reliability and data access on demand in one place


    Arthur My Blog

    • Marked as answer by BramDe Friday, August 10, 2012 8:29 AM
    Thursday, August 9, 2012 3:36 PM
    Moderator
  • Hi Arthur,

    Thanks a lot, you helped me a lot around this topic!

    Friday, August 10, 2012 8:29 AM