chore: add chunk bytes library in prep for singleton schema work

[josephschorr]

Description

Adds a chunk bytes library to the common datastore code for easy storing and loading of byte data that will be chunked across multiple rows in the database, as some databases have soft or hard limits on the size of BLOB fields.

This will be used by singleton schemas to store schemas and, likely eventually, schema metadata

Testing

Unit tests. This is very early and not used yet.

[codecov]

Codecov Report

❌ Patch coverage is 85.16484% with 27 lines in your changes missing coverage. Please review.
✅ Project coverage is 77.42%. Comparing base (01d72a1) to head (a94abd9).

Files with missing lines	Patch %	Lines
internal/datastore/common/chunkbytes.go	85.17%	17 Missing and 10 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2627      +/-   ##
==========================================
+ Coverage   77.39%   77.42%   +0.03%     
==========================================
  Files         449      450       +1     
  Lines       56613    56795     +182     
==========================================
+ Hits        43811    43968     +157     
- Misses      10045    10059      +14     
- Partials     2757     2768      +11     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[tstirrat15]

See comments, looks good to me

[tstirrat15]

For my own edification: when you allocate a slice of slices, is that essentially a slice of pointers?

[tstirrat15]

Why is this an any? and what is ^uint64(0) supposed to represent? Is that a clever way of writing maxint?

[tstirrat15]

Do these happen to line up because utf-8 is one byte per char for ascii characters?

[miparnisari]

I've amended your PR title from feat: to chore: because, for customers, this isn't a new feature yet

[miparnisari]

all these fields are already defined in SQLByteChunkerConfig. why not just hold a pointer to it? this would reduce duplicate code and prevent you from having to write

return &SQLByteChunker[T]{
		tableName:         config.TableName,
		nameColumn:        config.NameColumn,
		chunkIndexColumn:  config.ChunkIndexColumn,
		chunkDataColumn:   config.ChunkDataColumn,
		maxChunkSize:      config.MaxChunkSize,
		placeholderFormat: config.PlaceholderFormat,
		executor:          config.Executor,
		writeMode:         config.WriteMode,
		createdAtColumn:   config.CreatedAtColumn,
		deletedAtColumn:   config.DeletedAtColumn,
		aliveValue:        config.AliveValue,
	}

which is dangerous, because if you add a new configuration you have to remember to update that code too

[miparnisari]

Suggested change

	// SQLByteChunker provides methods for reading and writing byte data
	// that is chunked across multiple rows in a SQL table.
	type SQLByteChunker[T any] struct {
	type IntOrString interface {
	~uint64 | ~string
	}
	// SQLByteChunker provides methods for reading and writing byte data
	// that is chunked across multiple rows in a SQL table.
	type SQLByteChunker[T IntOrString] struct {

[miparnisari]

Suggested change

	// Validate that we have all chunks from 0 to N-1
	maxIndex := -1
	for index := range chunks {
	if index > maxIndex {
	maxIndex = index
	}
	}
	// Check for missing chunks
	for i := 0; i <= maxIndex; i++ {
	if _, exists := chunks[i]; !exists {
	return nil, fmt.Errorf("missing chunk at index %d", i)
	}
	}
	// Calculate total size
	totalSize := 0
	for _, chunk := range chunks {
	totalSize += len(chunk)
	}
	// Reassemble
	result := make([]byte, 0, totalSize)
	for i := 0; i <= maxIndex; i++ {
	result = append(result, chunks[i]...)
	}
	// Validate that we have all chunks from 0 to N-1 and calculate total size
	maxIndex := -1
	totalSize := 0
	for index := range chunks {
	maxIndex = max(maxIndex, index)
	totalSize += len(chunks[index])
	}
	// Reassemble, while checking for missing chunks
	result := make([]byte, 0, totalSize)
	for i := 0; i <= maxIndex; i++ {
	if _, exists := chunks[i]; !exists {
	return nil, fmt.Errorf("missing chunk at index %d", i)
	}
	result = append(result, chunks[i]...)
	}