Skip to content

[Feature Request]: Spark SQL Iceberg table support #463

@oerm

Description

@oerm

Topic

Support new SQL

Description

Currently linter does not support Iceberg syntax operations. One of the gaps identified was the lack of 4 parts table support convention which now can consist of catalog.db.table.branch. Fore example:

select
    h.made_current_at,
    s.operation,
    h.snapshot_id,
    h.is_current_ancestor,
    s.summary['spark.app.id']
from prod.db.table.history h
join prod.db.table.snapshots s
  on h.snapshot_id = s.snapshot_id
order by made_current_at;

Error text:

* Line:7 - 'table' is not valid at this position, expecting an existing function; Line:7 - '.' is not valid at this position; Line:7 - 'h' is not valid at this position; Line:8 - '.' is not valid at this position; Line:8 - 's' is not valid at this position;

Samples of Iceberg based syntax are available: https://iceberg.apache.org/docs/latest/spark-ddl/#create-table-as-select

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions