Substrait is an intermediate representation (IR) format for exchanging query plans between systems. It relieves the pain of SQL dialects, enables pushdown into different backends, removes redundant parsing and optimization in federated systems, and allows relatively painless backend replacement. Below - why it’s needed, how it works, and who supports it.
Query optimizers use knowledge of your data's nature, such as statistics and schema, to find optimal plans. Apache Calcite collectively refers to this information as metadata and provides a convenient API to extract operator's metadata within optimization routines. In this blog post, we will discuss the design of the metadata framework in Apache Calcite.
