Commit 6b66c5a
Add type conversion for PostgreSQL driver to Parquet type matching
## Problem
Panic: "cannot create parquet value of type INT32 from go value of type int64"
The PostgreSQL driver (lib/pq) returns int64 for ALL integer types (int2, int4, int8)
to avoid overflow issues. However, Parquet expects exact type matches:
- INT32 for PostgreSQL int2/int4
- INT64 for PostgreSQL int8
## Solution
Added `convertPostgreSQLValue()` function that converts driver types to appropriate
Go types based on the PostgreSQL column type from the schema:
### Type Conversions
- **int2, int4**: int64 → int32 (safe, PostgreSQL int4 max is 2^31-1)
- **int8**: Keep as int64
- **float4**: float64 → float32
- **float8, numeric, decimal**: Keep as float64
- **timestamp, date**: Keep as time.Time
- **bool**: Keep as bool
- **bytea**: Keep as []byte
- **strings**: Keep as-is
### Integration
The conversion is called when building rowData in the streaming extraction:
```go
rowData[col.GetName()] = convertPostgreSQLValue(scanValues[i], col.GetType())
```
## Benefits
✅ **Type Precision**: Correct integer sizes (int32 vs int64)
✅ **Parquet Compatibility**: Types match schema expectations
✅ **No Panics**: Proper type conversions prevent runtime errors
✅ **Safe**: Conversions are within PostgreSQL type bounds
## Testing
- All tests pass
- Linters pass (with nolint for safe int64→int32 conversion)
- Build succeeds
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <[email protected]>1 parent 5ab8ac8 commit 6b66c5a
1 file changed
+46
-2
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1852 | 1852 | | |
1853 | 1853 | | |
1854 | 1854 | | |
| 1855 | + | |
| 1856 | + | |
| 1857 | + | |
| 1858 | + | |
| 1859 | + | |
| 1860 | + | |
| 1861 | + | |
| 1862 | + | |
| 1863 | + | |
| 1864 | + | |
| 1865 | + | |
| 1866 | + | |
| 1867 | + | |
| 1868 | + | |
| 1869 | + | |
| 1870 | + | |
| 1871 | + | |
| 1872 | + | |
| 1873 | + | |
| 1874 | + | |
| 1875 | + | |
| 1876 | + | |
| 1877 | + | |
| 1878 | + | |
| 1879 | + | |
| 1880 | + | |
| 1881 | + | |
| 1882 | + | |
| 1883 | + | |
| 1884 | + | |
| 1885 | + | |
| 1886 | + | |
| 1887 | + | |
| 1888 | + | |
| 1889 | + | |
| 1890 | + | |
| 1891 | + | |
| 1892 | + | |
| 1893 | + | |
| 1894 | + | |
| 1895 | + | |
| 1896 | + | |
| 1897 | + | |
1855 | 1898 | | |
1856 | 1899 | | |
1857 | 1900 | | |
| |||
2008 | 2051 | | |
2009 | 2052 | | |
2010 | 2053 | | |
2011 | | - | |
| 2054 | + | |
2012 | 2055 | | |
2013 | 2056 | | |
2014 | | - | |
| 2057 | + | |
| 2058 | + | |
2015 | 2059 | | |
2016 | 2060 | | |
2017 | 2061 | | |
| |||
0 commit comments