How do you configure TRIRIGA for Tivoli Directory Integrator (TDI)?


You can configure TRIRIGA to use Tivoli Directory Integrator as its ETL runtime engine to run ETLJobItems from within TRIRIGA.

Before you begin

Install Tivoli Directory Integrator, if not already installed, on all the TRIRIGA systems that could run a TDI ETL Job Item.  During the TDI install:

  • Make note of the installation directory you enter on the Destination panel. You will enter this value later in TRIRIGAWEB.properties.
  • Select either installation type. TRIRIGA requires only the TDI Server component.
  • When prompted for the location of the Solution Directory, you can select any option. TRIRIGA specifies its own solution directory at runtime.  However selecting the option “Use Install Directory” may simplify troubleshooting.
  • Make note of the value you enter in the Server Port field on the Server Port Values Panel. You will enter this value later in TRIRIGAWEB.properties.
  • Clear the “Start the Configuration Editor” check box on the Install Complete panel.
  • Note: This step is very important for TDI/TRIRIGA integration to work. After you have installed Tivoli Directory Integrator, update it with the recommended fix packs (per TRIRIGA support matrix). TDI must be at least at FP04 (7.1.1.4) or it  will not automatically start the TririgiaETLDispatch.xml assembly line which will result in ETL job items failing to run successfully.

Procedure

  1. Edit TRIRIGAWEB.properties file to enable TRIRIGA to manage TDI server.  Set the following properties…
  2. Install a JDBC driver library so that Tivoli Directory Integrator can use it to access TRIRIGA database…
  3. Edit TDI global.properties file to allow TRIRIGA to check and stop the TDI server from localhost without requiring authentication and authorization certificates. Set the api.remote.ssl.on property to false to tell TDI to trust requests from localhost…
  4. Start Tivoli Directory Integrator Agent from TRIRIGA Admin Console and verify that it starts successfully…

[Admin: This post is related to the 08.03.16 post about installing, upgrading, or uninstalling TRIRIGA TDI, and the 05.01.16 post about documentation on developing TDI with TRIRIGA. To see other related posts, use the TDI tag.]

Continue reading

Why is there a JDBC exception with TRIRIGA on WebSphere and DB2?


I am getting a strange error when I am trying to deploy to TRIRIGA with the WebSphere Application Server (WAS). TRIRIGA doesn’t come up and throws an exception. Interestingly, when I point a Liberty application on the same DB2 database, it works well. I have confirmed with my network team that this is not a network connectivity issue. Here are the error logs:

Caused by: org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is java.sql.SQLException: DB2 SQL Error: SQLCODE=-805, SQLSTATE=51002, SQLERRMC=NULLIDR1.SYSSH200 0X5359534C564C3031, DRIVER=4.18.60 DSRA0010E: SQL State = 51002, Error Code = -805...

Have you checked to see that both the Liberty server and the WAS server are actually using the exact same version of the DB2 driver (db2jcc4.jar)?

[Admin: To see other related posts, use the DB2 tag or JDBC tag.]

Continue reading

Can you stop the database and leave the TRIRIGA JVMs running?


What are the concerns about stopping my database for maintenance and leaving IBM TRIRIGA JVMs (JBoss, WebLogic, WebSphere) up and running at this point? Will they be reconnecting automatically after my database is up and running again? I need to programmatically schedule database maintenance for my TRIRIGA system.

When the database is down, the application server (JBoss, WebLogic, WebSphere) will be receiving connection issues to the JDBC component and JVMs will stop responding after that. If the database comes up again, the application server will not reconnect the JVM automatically. The JVM needs to be restarted manually after that.

The best practice for database maintenance requiring database shutdown will always be to shutdown all applications and sessions connected to it BEFORE the database itself. It gives systems the time to close the ongoing transactions gracefully.

If you need to coordinate database maintenance and JVMs automatic restarts, you need to create a batch script to manage that. This is a customized script (not under IBM TRIRIGA support) that will be stopping the JVMs first, then starting the database maintenance itself (likely stopping the database first), then restarting the database and firing commands to restart the application server IBM TRIRIGA JVMs.

Continue reading

Getting an Oracle connection test failure during install


During install (upgrade to TRIRIGA 3.5.1 on existing Oracle 12c), I noticed that the installer appears to be using a deprecated connection method for testing the data schema.

  • It is using: tridata@jdbc:oracle:thin:@server:1521:tririga
  • Instead of: tridata@jdbc:oracle:thin:@server:1521/tririga (Notice the slash “/” instead of a colon.)

I am getting a connection test failure. But outside the installer, I can manually connect with the username/password without issue. Since my last install, I have moved from Oracle 11g to 12c. Since this change, I have never gone through a TRIRIGA upgrade. Any ideas why my connection test is failing? And if it is because of the colon instead of the slash “/”? How do I work around this inside the installer?

Support for this, and Oracle 12c R2 is tentatively planned for our next release, that is scheduled to come out at the end of this year. Until our next GA release, Oracle 12c with pluggable database (PDB) is not supported and will not work.

Continue reading

Is there any documentation on developing Tivoli Directory Integrator (TDI) with TRIRIGA?


I’m currently working on an interface which will load the users and organizations into TRIRIGA from an external system. The data loading work will be based on flat files, since the source system can’t provide the file with a format served by TRIRIGA. I plan to use TDI to consume the incoming file, transform it, and then write the transformed data into staging tables in TRIRIGA (I’m using DataConnect).

Since the TDI script will be invoked by TRIRIGA, there must be some point where TDI development will be different from a classic TDI development, such as how we should get the JDBC connection with the parameters (JDBC URL, username, password, etc.) passed by TRIRIGA. In the IBM Knowledge Center, most of the articles on TDI are aimed to calculate the metric table. Is there any documentation that introduces the TDI development under TRIRIGA in a general way? Thanks in advance for your help!

How to write TDI-based ETLs for TRIRIGA is documented in the Application Building for Performance Framework [PDF] user guide. Also, in the IBM Knowledge Center, check out Performance Framework > Data structures > ETL integration > Defining and Maintaining ETL transforms > Using ETLs with IBM Tivoli Directory Integrator Configuration Editor.

Continue reading

Why do you see “Max connections reached” errors in WebSphere?


Why does the system get frozen and I see “Maximum number of connections has been reached” error messages in WebSphere FFDC logs when running a IBM TRIRIGA server? Why is this happening? The system gets frozen and WebSphere FFDC logs ([WAS_install_root]/profiles/[profile]/logs/ffdc) report the following error:

[MM/DD/YY HH:MM:SS:mmm TTT] FFDC Exception:com.ibm.websphere.ce.j2c.ConnectionWaitTimeoutException SourceId:Max connections reached ProbeId:869 Reporter:com.ibm.ejs.j2c.PoolManager@98575e96
com.ibm.websphere.ce.j2c.ConnectionWaitTimeoutException: CWTE_NORMAL_J2CA1009
...
Maximum number of connections has been reached, and the connection request has been waiting longer than:ConnectionWaitTime. Two possible solutions : increase the max number of connections, or increase the:ConnectionWaitTime.:
Maximum Connections = :100
Current number of connections = :100
Connection Wait Timout = :180

Your IBM WebSphere server/JVM (IBM TRIRIGA server) has reached its JDBC Data Source Maximum Connection limit, and the system will stop responding to new processes requesting available connections to be used…

Check that the JDBC Data Source Connection Pool’s connections are set by server (WebSphere servers/JVMs). If the IBM TRIRIGA Best Practices for System Performance has been followed, all configuration necessary is in place. On page 18, under “3.1.3 Connection Pool”, it reads…

Continue reading