File tree Expand file tree Collapse file tree 2 files changed +6
-11
lines changed Expand file tree Collapse file tree 2 files changed +6
-11
lines changed Original file line number Diff line number Diff line change @@ -317,16 +317,9 @@ def _diff_between_actual_and_dlt_schema(
317317 destination table. It's used during schema synchronization to detect when columns may have
318318 been dropped from the destination and need to be removed from the dlt schema as well.
319319
320- However, dlt internal columns (_dlt_id, _dlt_load_id) are treated specially because:
321-
322- 1. Users rarely drop dlt internal columns manually, and if they did,
323- dlt cannot recover from this situation anyway.
324-
325- 2. Athena has a constraint where dlt columns exist in the data but not in the table metadata:
326-
327- - Athena external tables have fixed schemas defined at CREATE TABLE time
328- - These columns exist in the actual data files but don't appear in INFORMATION_SCHEMA
329- - This causes false positives where dlt columns appear "missing" when they're not
320+ However, dlt internal columns (_dlt_id, _dlt_load_id) are treated specially because
321+ users rarely drop dlt internal columns manually, and if they did,
322+ dlt cannot recover from this situation anyway.
330323
331324 Args:
332325 client (WithTableReflectionAndSql): The destination client with table reflection capabilities.
Original file line number Diff line number Diff line change @@ -84,6 +84,8 @@ def _drop_column_in_sql(
8484 col_defs = [
8585 f"{ client .escape_ddl_identifier ('id' )} bigint" ,
8686 f"{ client .escape_ddl_identifier ('name' )} string" ,
87+ f"{ client .escape_ddl_identifier ('_dlt_load_id' )} string" ,
88+ f"{ client .escape_ddl_identifier ('_dlt_id' )} string" ,
8789 ]
8890 ddl = (
8991 f"ALTER TABLE { client .make_qualified_ddl_table_name (table_name )} "
@@ -244,7 +246,7 @@ def test_sync_dlt_schema(
244246 logger_spy .assert_called ()
245247 assert logger_spy .call_count == 1
246248 expected_warning = (
247- "Removing table 'my_last_table' from the dlt schema would leave orphan table(s):"
249+ "Removing table 'my_last_table' from the dlt schema would leave orphaned table(s):"
248250 " 'my_last_table__children'. Drop these child tables in the destination and sync the dlt"
249251 " schema again."
250252 )
You can’t perform that action at this time.
0 commit comments