Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dissociate BQ and Hive predicates? #37

Open
jphalip opened this issue Dec 30, 2022 · 0 comments
Open

Dissociate BQ and Hive predicates? #37

jphalip opened this issue Dec 30, 2022 · 0 comments

Comments

@jphalip
Copy link
Collaborator

jphalip commented Dec 30, 2022

We currently apply the predicates both in BQ and Hive. Let's figure out if we can optimize this by skipping the Hive predicates that have already been applied in BQ (i.e. predicate pushdowns). There might also be some Hive predicate functions/UDFs that don't have equivalents in BQ and so we might have to exclude those from the predicate pushdowns.

See relevant code here:

@Override
public DecomposedPredicate decomposePredicate(
JobConf jobConf, Deserializer deserializer, ExprNodeDesc exprNodeDesc) {
// TODO: See if we can dissociate the pushed predicates from the residual ones
DecomposedPredicate predicate = new DecomposedPredicate();
predicate.residualPredicate = (ExprNodeGenericFuncDesc) exprNodeDesc;
predicate.pushedPredicate = (ExprNodeGenericFuncDesc) exprNodeDesc;
return predicate;
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant