Join GitHub today
GitHub is home to over 50 million developers working together to host and review code, manage projects, and build software together.
Sign upExcess filtering when trying to join a BigQuery partitioned table #1165
Comments
Hey @shai-caazam! I would go for the data blending approach here https://cube.dev/blog/introducing-data-blending-api. It will eliminate the filtering problem as there will be run two queries. |
We have call center app that measures calls and orders made.
We use BigQuery for data warehouse:
Creating cubes for both these tables - everything is OK:
Calls:
Orders:
So far so good. Problem is when we want to LEFT JOIN calls with orders:
The problem is that the resulting SQL filters each table according to its required partition filter and in addition adds a filter on the joined table. This filter will remove any rows that have calls but don't have orders (right side of the join is null) because the joined row doesn't match the "outer" filter (its timestamp is NULL).
This results in partial results - cannot see rows that have only calls but no orders.
What we really need is filters to apply to the inner pre-joined tables only.
Any way to get around this?
Thanks