diff --git a/README.md b/README.md index 71ea78f..c284dda 100644 --- a/README.md +++ b/README.md @@ -31,6 +31,12 @@ typeMap exists because Hive types and Phoenix types are not 1 to 1. "string|varc The "jars" property allows end-users to supply a comma separated list of jars which need to be available on the classpath of the executors (JVM libraries, etc) +***Tables can be salted using the two salt properties;*** + +DST_TABLE_NAMESalt is a number declaring the number of salt buckets to use for the table, the tablename must be all uppercased + +salt is a number declaring the number of salt buckets to use for any table not listed in the above property (default salt buckets) + ***Saving Phoenix tables into Hive:*** To query Phoenix tables and save them back to Hive, change "destination=phoenix" to "destination=hive". When saving tables to Hive, "format" specifies the intended file-format (ORC, Parquet, Avro, text, etc). diff --git a/src/main/scala/com/github/randerzander/HiveToPhoenix.scala b/src/main/scala/com/github/randerzander/HiveToPhoenix.scala index 64d248e..7d82359 100644 --- a/src/main/scala/com/github/randerzander/HiveToPhoenix.scala +++ b/src/main/scala/com/github/randerzander/HiveToPhoenix.scala @@ -64,6 +64,11 @@ object HiveToPhoenix{ command += field.name + " " + dstType + "," } command += " constraint my_pk primary key ("+pk+"))" + if (props.contains(dstTables(i)+"Salt")){ + command += " SALT_BUCKETS = "+props.getOrElse(dstTables(i)+"Salt","10"); + }else if(props.contains("salt")){ + command += " SALT_BUCKETS = "+props.getOrElse("salt","10"); + } println("INFO: DESTINATION DDL:\n" + command) // Execute Phoenix DDL getConn(jdbcClass, connStr).createStatement().execute(command)