-
Notifications
You must be signed in to change notification settings - Fork 16
/
RELEASE_NOTES.txt
260 lines (223 loc) · 18.1 KB
/
RELEASE_NOTES.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
Release Notes - Hive - Version 2.1.1
** Sub-task
* [HIVE-13409] - Fix JDK8 test failures related to COLUMN_STATS_ACCURATE
* [HIVE-13549] - Remove jdk version specific out files from Hive2
* [HIVE-13587] - Set Hive pom to use Hadoop 2.6.1
* [HIVE-13593] - HiveServer2: Performance instrumentation for HIVE-12049 (serializing thrift ResultSets in tasks)
* [HIVE-13723] - Executing join query on type Float using Thrift Serde will result in Float cast to Double error
* [HIVE-13798] - Fix the unit test failure org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ivyDownload
* [HIVE-13803] - More aggressive inference of transitive predicates for inner joins
* [HIVE-13860] - Fix more json related JDK8 test failures
* [HIVE-13901] - Hivemetastore add partitions can be slow depending on filesystems
* [HIVE-13965] - Empty resultset run into Exception when using Thrift Binary Serde
* [HIVE-14028] - stats is not updated
* [HIVE-14039] - HiveServer2: Make the usage of server with JDBC thirft serde enabled, backward compatible for older clients
* [HIVE-14191] - bump a new api version for ThriftJDBCBinarySerde changes
* [HIVE-14276] - Update protocol version in TOpenSessionReq and TOpenSessionResp
* [HIVE-14277] - Disable StatsOptimizer for all ACID tables
* [HIVE-14291] - count(*) on a table written by hcatstorer returns incorrect result
* [HIVE-14421] - FS.deleteOnExit holds references to _tmp_space.db files
* [HIVE-14428] - HadoopMetrics2Reporter leaks memory if the metrics sink is not configured correctly
* [HIVE-14902] - HiveServer2: Cleanup code which checks for ThriftJDBCSerde usage
* [HIVE-14912] - Fix the test failures for 2.1.1 caused by HIVE-13409
* [HIVE-14957] - HiveSortLimitPullUpConstantsRule misses branches when parent operator is Union
* [HIVE-15094] - Fix test failures for 2.1.1 regarding schema evolution with DECIMAL types
* [HIVE-15154] - Fix rest of q test file changes in branch-2.1
** Bug
* [HIVE-9423] - HiveServer2: Provide the user with different error messages depending on the Thrift client exception code
* [HIVE-10022] - Authorization checks for non existent file/directory should not be recursive
* [HIVE-11402] - HS2 - add an option to disallow parallel query execution within a single Session
* [HIVE-13191] - DummyTable map joins mix up columns between tables
* [HIVE-13264] - JDBC driver makes 2 Open Session Calls for every open session
* [HIVE-13369] - AcidUtils.getAcidState() is not paying attention toValidTxnList when choosing the "best" base file
* [HIVE-13392] - disable speculative execution for ACID Compactor
* [HIVE-13539] - HiveHFileOutputFormat searching the wrong directory for HFiles
* [HIVE-13590] - Kerberized HS2 with LDAP auth enabled fails in multi-domain LDAP case
* [HIVE-13610] - Hive exec module won't compile with IBM JDK
* [HIVE-13648] - ORC Schema Evolution doesn't support same type conversion for VARCHAR, CHAR, or DECIMAL when maxLength or precision/scale is different
* [HIVE-13704] - Don't call DistCp.execute() instead of DistCp.run()
* [HIVE-13725] - ACID: Streaming API should synchronize calls when multiple threads use the same endpoint
* [HIVE-13744] - LLAP IO - add complex types support
* [HIVE-13749] - Memory leak in Hive Metastore
* [HIVE-13809] - Hybrid Grace Hash Join memory usage estimation didn't take into account the bloom filter size
* [HIVE-13836] - DbNotifications giving an error = Invalid state. Transaction has already started
* [HIVE-13874] - Tighten up EOF checking in Fast DeserializeRead classes; display better exception information; add new Unit Tests
* [HIVE-13932] - Hive SMB Map Join with small set of LIMIT failed with NPE
* [HIVE-13934] - Configure Tez to make nocondiional task size memory available for the Processor
* [HIVE-13985] - ORC improvements for reducing the file system calls in task side
* [HIVE-13991] - Union All on view fail with no valid permission on underneath table
* [HIVE-13997] - Insert overwrite directory doesn't overwrite existing files
* [HIVE-14000] - (ORC) Changing a numeric type column of a partitioned table to lower type set values to something other than 'NULL'
* [HIVE-14003] - queries running against llap hang at times - preemption issues
* [HIVE-14004] - Minor compaction produces ArrayIndexOutOfBoundsException: 7 in SchemaEvolution.getFileType
* [HIVE-14012] - some ColumnVector-s are missing ensureSize
* [HIVE-14022] - left semi join should throw SemanticException if where clause contains columnname from right table
* [HIVE-14024] - setAllColumns is called incorrectly after some changes
* [HIVE-14027] - NULL values produced by left outer join do not behave as NULL
* [HIVE-14031] - cleanup metadataReader in OrcEncodedDataReader
* [HIVE-14034] - Vectorization may fail with compex OR conditions
* [HIVE-14038] - miscellaneous acid improvements
* [HIVE-14041] - llap scripts add hadoop and other libraries from the machine local install to the daemon classpath
* [HIVE-14045] - (Vectorization) Add missing case for BINARY in VectorizationContext.getNormalizedName method
* [HIVE-14055] - directSql - getting the number of partitions is broken
* [HIVE-14059] - Missing license headers for two files
* [HIVE-14062] - Changes from HIVE-13502 overwritten by HIVE-13566
* [HIVE-14071] - HIVE-14014 breaks non-file outputs
* [HIVE-14072] - QueryIds reused across different queries
* [HIVE-14073] - update config whiltelist for sql std authorization
* [HIVE-14076] - Vectorization is not supported for datatype:VOID error while inserting data into specific columns
* [HIVE-14079] - Remove file, method and line number from pattern layout
* [HIVE-14083] - ALTER INDEX in Tez causes NullPointerException
* [HIVE-14090] - JDOExceptions thrown by the Metastore have their full stack trace returned to clients
* [HIVE-14092] - Kryo exception when deserializing VectorFileSinkOperator
* [HIVE-14098] - Logging task properties, and environment variables might contain passwords
* [HIVE-14111] - better concurrency handling for TezSessionState - part I
* [HIVE-14114] - Ensure RecordWriter in streaming API is using the same UserGroupInformation as StreamingConnection
* [HIVE-14122] - VectorMapOperator: Missing update to AbstractMapOperator::numRows
* [HIVE-14132] - Don't fail config validation for removed configs
* [HIVE-14136] - LLAP ZK SecretManager should resolve _HOST in principal
* [HIVE-14137] - Hive on Spark throws FileAlreadyExistsException for jobs with multiple empty tables
* [HIVE-14140] - LLAP: package codec jars
* [HIVE-14144] - Permanent functions are showing up in show functions, but describe says it doesn't exist
* [HIVE-14147] - Hive PPD might remove predicates when they are defined as a simple expr e.g. WHERE 'a'
* [HIVE-14152] - datanucleus.autoStartMechanismMode should set to 'Ignored' to allow rolling downgrade
* [HIVE-14153] - Beeline: beeline history doesn't work on Hive2
* [HIVE-14163] - LLAP: use different kerberized/unkerberized zk paths for registry
* [HIVE-14172] - LLAP: force evict blocks by size to handle memory fragmentation
* [HIVE-14173] - NPE was thrown after enabling directsql in the middle of session
* [HIVE-14175] - Fix creating buckets without scheme information
* [HIVE-14176] - CBO nesting windowing function within each other when merging Project operators
* [HIVE-14178] - Hive::needsToCopy should reuse FileUtils::equalsFileSystem
* [HIVE-14184] - Adding test for limit pushdown in presence of grouping sets
* [HIVE-14187] - JDOPersistenceManager objects remain cached if MetaStoreClient#close is not called
* [HIVE-14188] - LLAPIF: wrong user field is used from the token
* [HIVE-14192] - False positive error due to thrift
* [HIVE-14195] - HiveMetaStoreClient getFunction() does not throw NoSuchObjectException
* [HIVE-14197] - LLAP service driver precondition failure should include the values
* [HIVE-14205] - Hive doesn't support union type with AVRO file format
* [HIVE-14207] - Strip HiveConf hidden params in webui conf
* [HIVE-14210] - ExecDriver should call jobclient.close() to trigger cleanup
* [HIVE-14214] - ORC Schema Evolution and Predicate Push Down do not work together (no rows returned)
* [HIVE-14215] - Displaying inconsistent CPU usage data with MR execution engine
* [HIVE-14218] - LLAP: ACL validation fails if the user name is different from principal user name
* [HIVE-14222] - PTF: Operator initialization does not clean state
* [HIVE-14223] - beeline should look for jdbc standalone jar in dist/jdbc dir instead of dist/lib
* [HIVE-14236] - CTAS with UNION ALL puts the wrong stats in Tez
* [HIVE-14241] - Acid clashes with ConfVars.HIVEFETCHTASKCONVERSION <> "none"
* [HIVE-14242] - Backport ORC-53 to Hive
* [HIVE-14244] - bucketmap right outer join query throws ArrayIndexOutOfBoundsException
* [HIVE-14245] - NoClassDefFoundError when starting LLAP daemon
* [HIVE-14253] - Fix MinimrCliDriver test failure
* [HIVE-14262] - Inherit writetype from partition WriteEntity for table WriteEntity
* [HIVE-14263] - Log message when HS2 query is waiting on compile lock
* [HIVE-14265] - Partial stats in Join operator may lead to data size estimate of 0
* [HIVE-14267] - HS2 open_operations metrics not decremented when an operation gets timed out
* [HIVE-14268] - INSERT-OVERWRITE is not generating an INSERT event during hive replication
* [HIVE-14282] - HCatLoader ToDate() exception with hive partition table ,partitioned by column of DATE datatype
* [HIVE-14292] - ACID table creation fails on mysql with MySQLIntegrityConstraintViolationException
* [HIVE-14293] - PerfLogger.openScopes should be transient
* [HIVE-14296] - Session count is not decremented when HS2 clients do not shutdown cleanly.
* [HIVE-14297] - OrcRecordUpdater floods logs trying to create _orc_acid_version file
* [HIVE-14298] - NPE could be thrown in HMS when an ExpressionTree could not be made from a filter
* [HIVE-14301] - insert overwrite fails for nonpartitioned tables in s3
* [HIVE-14308] - While using column stats estimated data size may become 0
* [HIVE-14310] - ORC schema evolution should not completely disable PPD
* [HIVE-14311] - No need to schedule Heartbeat task if the query doesn't require locks
* [HIVE-14322] - Postgres db issues after Datanucleus 4.x upgrade
* [HIVE-14324] - ORC PPD for floats is broken
* [HIVE-14326] - Merging outer joins without conditions can lead to wrong results
* [HIVE-14330] - fix LockHandle TxnHandler.acquireLock(String key) retry logic
* [HIVE-14332] - Reduce logging from VectorMapOperator
* [HIVE-14333] - ORC schema evolution from float to double changes precision and breaks filters
* [HIVE-14335] - TaskDisplay's return value is not getting deserialized properly
* [HIVE-14338] - Delete/Alter table calls failing with HiveAccessControlException
* [HIVE-14339] - Fix UT failure for acid_globallimit.q
* [HIVE-14342] - Beeline output is garbled when executed from a remote shell
* [HIVE-14349] - Vectorization: LIKE should anchor the regexes
* [HIVE-14350] - Aborted txns cause false positive "Not enough history available..." msgs
* [HIVE-14355] - Schema evolution for ORC in llap is broken for int to string conversion
* [HIVE-14357] - TestDbTxnManager2#testLocksInSubquery failing in branch-2.1
* [HIVE-14359] - Hive on Spark might fail in HS2 with LDAP authentication in a kerberized cluster
* [HIVE-14363] - bucketmap inner join query fails due to NullPointerException in some cases
* [HIVE-14366] - Conversion of a Non-ACID table to an ACID table produces non-unique primary keys
* [HIVE-14377] - LLAP IO: issue with how estimate cache removes unneeded buffers
* [HIVE-14381] - Handle null value in WindowingTableFunction.WindowingIterator.next()
* [HIVE-14393] - Tuple in list feature fails if there's only 1 tuple in the list
* [HIVE-14394] - Reduce excessive INFO level logging
* [HIVE-14395] - Add the missing data files to Avro union tests (HIVE-14205 addendum)
* [HIVE-14397] - Queries ran after reopening of tez session launches additional sessions
* [HIVE-14399] - Fix test flakiness of org.apache.hive.hcatalog.listener.TestDbNotificationListener.cleanupNotifs
* [HIVE-14400] - Handle concurrent insert with dynamic partition
* [HIVE-14402] - Vectorization: Fix Mapjoin overflow deserialization
* [HIVE-14403] - LLAP node specific preemption will only preempt once on a node per AM
* [HIVE-14408] - thread safety issue in fast hashtable
* [HIVE-14414] - Fix TestHiveMetaStoreTxns UTs
* [HIVE-14418] - Hive config validation prevents unsetting the settings
* [HIVE-14424] - Address CLIRestoreTest failure
* [HIVE-14426] - Extensive logging on info level in WebHCat
* [HIVE-14433] - refactor LLAP plan cache avoidance and fix issue in merge processor
* [HIVE-14435] - Vectorization: missed vectorization for const varchar()
* [HIVE-14439] - LlapTaskScheduler should try scheduling tasks when a node is disabled
* [HIVE-14446] - Add switch to control BloomFilter in Hybrid grace hash join and make the FPP adjustable
* [HIVE-14447] - Set HIVE_TRANSACTIONAL_TABLE_SCAN to the correct job conf for FetchOperator
* [HIVE-14448] - Queries with predicate fail when ETL split strategy is chosen for ACID tables
* [HIVE-14457] - Partitions in encryption zone are still trashed though an exception is returned
* [HIVE-14479] - Add some join tests for acid table
* [HIVE-14480] - ORC ETLSplitStrategy should use thread pool when computing splits
* [HIVE-14483] - java.lang.ArrayIndexOutOfBoundsException org.apache.orc.impl.TreeReaderFactory$BytesColumnVectorUtil.commonReadByteArrays
* [HIVE-14513] - Enhance custom query feature in LDAP atn to support resultset of ldap groups
* [HIVE-14519] - Multi insert query bug
* [HIVE-14530] - Union All query returns incorrect results
* [HIVE-14538] - beeline throws exceptions with parsing hive config when using !sh statement
* [HIVE-14563] - StatsOptimizer treats NULL in a wrong way
* [HIVE-14566] - LLAP IO reads timestamp wrongly
* [HIVE-14591] - HS2 is shut down unexpectedly during the startup time
* [HIVE-14600] - LLAP zookeeper registry failures do not fail the daemon
* [HIVE-14608] - LLAP: slow scheduling due to LlapTaskScheduler not removing nodes on kill
* [HIVE-14619] - CASE folding can produce wrong expression
* [HIVE-14621] - LLAP: memory.mode = none has NPE
* [HIVE-14652] - incorrect results for not in on partition columns
* [HIVE-14674] - Incorrect syntax near the keyword 'with' using MS SQL Server
* [HIVE-14697] - Can not access kerberized HS2 Web UI
* [HIVE-14702] - LLAPIF: after a long period of inactivity, signing key may be removed from local store
* [HIVE-14715] - Hive throws NumberFormatException with query with Null value
* [HIVE-14743] - ArrayIndexOutOfBoundsException - HBASE-backed views' query with JOINs
* [HIVE-14764] - Enabling "hive.metastore.metrics.enabled" throws OOM in HiveMetastore
* [HIVE-14766] - ObjectStore.initialize() needs retry mechanisms in case of connection failures
* [HIVE-14774] - Canceling query using Ctrl-C in beeline might lead to stale locks
* [HIVE-14779] - make DbTxnManager.HeartbeaterThread a daemon
* [HIVE-14799] - Query operation are not thread safe during its cancellation
* [HIVE-14814] - metastoreClient is used directly in Hive cause NPE
* [HIVE-14889] - Beeline leaks sensitive environment variables of HiveServer2 when you type set;
* [HIVE-14959] - Fix DISTINCT with windowing when CBO is enabled/disabled
* [HIVE-15002] - HiveSessionImpl#executeStatementInternal may leave locks in an inconsistent state
* [HIVE-15022] - Missing hs2-connection-timed-out in BeeLine.properties
* [HIVE-15061] - Metastore types are sometimes case sensitive
* [HIVE-15090] - Temporary DB failure can stop ExpiredTokenRemover thread
* [HIVE-15099] - PTFOperator.PTFInvocation didn't properly reset the input partition
* [HIVE-15123] - LLAP UI: The UI should work even if the cache is disabled
* [HIVE-15137] - metastore add partitions background thread should use current username
* [HIVE-15181] - buildQueryWithINClause didn't properly handle multiples of ConfVars.METASTORE_DIRECT_SQL_MAX_ELEMENTS_IN_CLAUSE
* [HIVE-15295] - Fix HCatalog javadoc generation with Java 8
* [HIVE-15306] - Change NOTICE file to account for JSON license components
** Improvement
* [HIVE-10815] - Let HiveMetaStoreClient Choose MetaStore Randomly
* [HIVE-13631] - Support index in HBase Metastore
* [HIVE-13982] - Extensions to RS dedup: execute with different column order and sorting direction if possible
* [HIVE-14002] - Extend limit propagation to subsequent RS operators
* [HIVE-14018] - Make IN clause row selectivity estimation customizable
* [HIVE-14021] - When converting to CNF, fail if the expression exceeds a threshold
* [HIVE-14057] - Add an option in llapstatus to generate output to a file
* [HIVE-14080] - hive.metastore.schema.verification should check for schema compatiblity
* [HIVE-14167] - Use work directories provided by Tez instead of directly using YARN local dirs
* [HIVE-14213] - Add timeouts for various components in llap status check
* [HIVE-14228] - Better row count estimates for outer join during physical planning
* [HIVE-14383] - SparkClientImpl should pass principal and keytab to spark-submit instead of calling kinit explicitely
* [HIVE-14545] - HiveServer2 with http transport mode spends too much time just creating configs
** Task
* [HIVE-14202] - Change tez version used to 0.8.4
* [HIVE-15248] - Add Apache header license to TestCustomPartitionVertex
** Test
* [HIVE-14212] - hbase_queries result out of date on branch-2.1
* [HIVE-14316] - TestLlapTokenChecker.testCheckPermissions, testGetToken fail
* [HIVE-14713] - LDAP Authentication Provider should be covered with unit tests