Tracking Hive version in CDAP

Go to CDAP console -> management -> on top right we can see Version – 4.0.1

http://docs.cask.co/cdap/4.0.1/en/faqs/general.html
What Hadoop distributions can CDAP run on?
CDAP 4.0.1 has been tested on and supports CDH 5.0.0 through 5.6.x; HDP 2.0 through 2.3.x; MapR 4.x and 5.x, and Apache Bigtop 1.0.

Now check HDP
https://hortonworks.com/blog/available-now-hdp-2-3/

Latest: https://hortonworks.com/products/data-center/hdp/

As on March 15th 2017, no support for Hive 2.0
———

Hive / Hive UI

Apache Hive comes with CLI
Hive CLI and Beeline CLI
https://cwiki.apache.org/confluence/display/Hive/AdminManual+Installation

How to connect with local Beeline CLI to remote connection:
https://cwiki.apache.org/confluence/display/Hive/HiveServer2+Clients
–This includes JDBC example

http://gethue.com/sql-editor/
This is excellent user interface

Now …time to work with Hive
https://www.tutorialspoint.com/hive/hive_create_table.htm

Giving DDL to create required script
https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL

STORED AS ORC Stored as ORC file format. Supports ACID Transactions & Cost-based Optimizer (CBO). Stores column-level metadata.
https://cwiki.apache.org/confluence/display/Hive/LanguageManual+ORC#LanguageManualORC-HiveQLSyntax

————————————-
>show tables;

>CREATE TABLE IF NOT EXISTS ZIPCODES
( ID INT, zip_code INT,
county STRING, state STRING, inserted_ts TIMESTAMP)
COMMENT ‘This table is used to store zip codes.’
STORED AS ORC;

>describe ZIPCODES;

>SHOW TBLPROPERTIES ZIPCODES;

>SHOW CREATE TABLE ZIPCODES;

>SHOW TRANSACTIONS;

–Gives Hadoop and Hive version
set system:sun.java.command;
–Client Side
set hive.support.concurrency = true;
set hive.enforce.bucketing = true ;
set hive.exec.dynamic.partition.mode = nonstrict;
set hive.txn.manager = org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;
set hive.compactor.initiator.on = true;
set hive.compactor.worker.threads = 1 ;
–a positive number on at least one instance of the Thrift metastore service


Time for DML
https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DML

How to work with ACID tables in hive
http://www.remay.com.br/blog/how-to-work-with-acid-tables-in-hive/

https://hortonworks.com/hadoop-tutorial/using-hive-acid-transactions-insert-update-delete-data/

Comparison with HBase
https://hortonworks.com/blog/adding-acid-to-apache-hive/

https://www.cloudera.com/documentation/enterprise/5-8-x/topics/hive.html

https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL

—————–
Conclusion: As on March 2017 Hive version in Hartonworks is 1.2.
https://hortonworks.com/blog/available-now-hdp-2-3/
– This don’t have support for transactions
– To get support, we need to wait for 2.x to be part of hortonworks HDP 2.x
– For now decouple tables..outside process can directly write to Hive table A. and update table B with its status.
– From BigData application use CDAP/HDP abstraction or use Apache Phoenix (https://phoenix.apache.org/) API to merge and manage data required for business layer.
– Enabling global transactions and enabling table level transactions is confusing and still lot of clarity needs to come in the system.
—————–