Malhar input operator for connecting to Teradata

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Malhar input operator for connecting to Teradata

Vivek Bhide
Hi,

I would like to know if there is any input operator available to connect and read data to Teradata? I tried using the JdbcPOJOPollInputOperator but the queries it forms are not as per teradata syntax and hence it fails

Regards
Vivek
Reply | Threaded
Open this post in threaded view
|

Re: Malhar input operator for connecting to Teradata

Thomas Weise-2
The JDBC poll operator uses a query translator that does not support all dialects in the OS version.

For a one-off with Oracle, the following override for post processing did the trick:

    @Override

    protected String buildRangeQuery(Object key, int offset, int limit)

    {

      String query = super.buildRangeQuery(key, offset, limit);

      if (!store.getDatabaseDriver().contains("OracleDriver")) {

        return query;

      }

      // hack the query for Oracle 12c since jooq does not support in OS version

      // https://www.jooq.org/doc/3.9/manual/sql-building/sql-statements/select-statement/limit-clause/

      // https://www.jooq.org/javadoc/3.8.x/org/jooq/SQLDialect.html

      // example LIMIT 1 OFFSET 2  =>  OFFSET 2 ROWS FETCH NEXT 1 ROWS ONLY

      String mysqlClause = String.format("limit %s offset %s", limit, offset);

      if (offset == 0) {

        mysqlClause = String.format("limit %s", limit);

      }

      String oracleClause = String.format("OFFSET %s ROWS FETCH NEXT %s ROWS ONLY", offset, limit);

      query = query.replace(mysqlClause, oracleClause);

      LOG.debug("Rewritten query: {}", query);

      return query;

    }

Also, the operator in release 3.7 has a number of other bugs, which you find fixed in master. Plus, there is another PR open that is necessary to make it work with tables where data is purged.

Thomas


On Mon, Aug 28, 2017 at 3:42 PM, Vivek Bhide <[hidden email]> wrote:
Hi,

I would like to know if there is any input operator available to connect and
read data to Teradata? I tried using the JdbcPOJOPollInputOperator but the
queries it forms are not as per teradata syntax and hence it fails

Regards
Vivek



--
View this message in context: http://apache-apex-users-list.78494.x6.nabble.com/Malhar-input-operator-for-connecting-to-Teradata-tp1843.html
Sent from the Apache Apex Users list mailing list archive at Nabble.com.

Reply | Threaded
Open this post in threaded view
|

Re: Malhar input operator for connecting to Teradata

Vivek Bhide
Thanks Thomas for workaround and heads up about the bugs :)

Let me see if I can get this working in my usecase

Regards
Vivek
Reply | Threaded
Open this post in threaded view
|

Re: Malhar input operator for connecting to Teradata

Thomas Weise-2

On Mon, Aug 28, 2017 at 5:46 PM, Vivek Bhide <[hidden email]> wrote:
Thanks Thomas for workaround and heads up about the bugs :)

Let me see if I can get this working in my usecase

Regards
Vivek



--
View this message in context: http://apache-apex-users-list.78494.x6.nabble.com/Malhar-input-operator-for-connecting-to-Teradata-tp1843p1845.html
Sent from the Apache Apex Users list mailing list archive at Nabble.com.

Reply | Threaded
Open this post in threaded view
|

Re: Malhar input operator for connecting to Teradata

Vivek Bhide
Hi Thomas,

I could get the poller operator working for teradata with few of tweaks to
your workaround and from your latest code base from PR

Thanks again

Regards
Vivek



--
Sent from: http://apache-apex-users-list.78494.x6.nabble.com/