Bytes docs + API polish: architecture/requirements docs, and tighten parsing / array ref safety

[peter-lawrey]

This PR is mostly documentation and API hygiene for Chronicle-Bytes, with a few small but real functional fixes and safer behaviours.

Functional changes

• Safer Binary*ArrayReference.bindValueAt(…)

  • BinaryIntArrayReference.bindValueAt(long index, IntValue value) and
    BinaryLongArrayReference.bindValueAt(long index, LongValue value) now:

    • Validate that value is a BinaryIntReference / BinaryLongReference before binding.
    • Throw IllegalArgumentException with a clear message instead of an unchecked ClassCastException.

  • This makes misuse easier to diagnose and avoids surprising runtime ClassCastExceptions.

• More robust parse8bit(…) for generic Appendable

  • ByteStringParser.parse8bit(Appendable, StopCharTester/StopCharsTester):

    • Previously only handled StringBuilder and Bytes; other Appendable implementations were effectively unsupported.

    • Now falls back via a temporary StringBuilder and copies into the target Appendable, so:

      • StringWriter, ByteStringWriter and other Appendable types work correctly.
      • A new helper ByteStringParser.appendTo(Appendable, CharSequence) wraps IOExceptions in IORuntimeException.

• BytesInternal vectorised content-equals error handling tightened

  • The Java-11 VectorizedMismatch path now:

    • Explicitly requires both sides to be HeapBytesStore for the vectorised fast path.
    • Throws UnsupportedOperationException if the preconditions are not met or the reflective call fails, instead of silently returning null.

  • This makes misconfiguration or unexpected store types fail loudly instead of silently dropping back on a partially initialised path. Call sites that rely on vectorisation will now see a clear failure if the optimisation cannot be used.

• BytesUtil.combineDoubleNewline switch exhaustiveness

  • Fixed the switch over header bytes to always return in all branches (including default), removing an implicit fall-through.
  • Behaviour is unchanged for recognised markers, but the control flow is now explicit and easier to reason about.

• Type-safe Bytes.equals(...) / NoBytesStore.equals(...) edge cases

  • NoBytesStore.equals(Object) now checks other.isEmpty() instead of length() == 0, making it robust for stores where length() might not be a cheap or meaningful call but emptiness is.
  • Various equality tests have been tightened and extended so that heap vs direct equality (Bytes.allocateElasticOnHeap vs Bytes.allocateElasticDirect) is consistently exercised.

• Mapped file / page-size edge cases

  • Additional tests and small fixes around:

    • MappedFile.safeLimit() and MappedBytes.inside(...) when using non-default page sizes.
    • MappedBytes.zeroOut(long from, long to) to ensure it honours safe limits even when huge page sizes differ from OS page size.

  • These primarily affect corner cases with custom page sizes and ensure we never write beyond the mapped region.

• Misc small correctness improvements

  • DistributedUniqueTimeProvider now exposes its DistributedUniqueTimeDeduplicator via a package-visible accessor for tighter testing and coordination.
  • Several tests for MappedBytes, MappedFile, and time providers now assert more invariants (e.g. that black-hole fields change, that temporary files are deleted or scheduled for deletion).

Behavioural changes are deliberately small and local; all existing tests still pass, and new tests cover the new behaviours.

---

Non-functional changes (docs, API surface, tests)

1. New agent / AI guidance docs

• CLAUDE.md

  • Detailed guidance for Claude Code / claude.ai agents working on Chronicle-Bytes:

    • Project overview and structure.
    • Build commands (mvn clean verify, -DskipTests, profiles quality, sonar).
    • Code style, threading, memory-management notes.
    • Expectations around off-heap resource release, assertReferencesReleased(), JMH benchmarks, etc.

• GEMINI.md

  • AI-generated Chronicle-Bytes project summary aimed at Gemini-style agents:

    • High-level overview (off-heap memory, 63-bit sizes, elastic buffers, stop-bit encoding).
    • Build and test commands (including -P run-benchmarks, -P assertions, -P quality, -P sonar).
    • Testing and coverage expectations.
    • Links back to the new AsciiDoc docs under src/main/docs/.

• AGENTS.md (Bytes-local)

  • Now explicitly states that:

    • Root AGENTS.md defines global rules; this file adds Chronicle-Bytes specifics.
    • Durable docs live in src/main/docs/ with README.adoc as the landing page.

  • Language / charset table tightened:

    • Clarifies ISO-8859-1 usage and how to check for stray non-ASCII with iconv or IDE settings.

2. Documentation restructure under src/main/docs/

• Most docs have moved / been consolidated into a canonical src/main/docs/ tree:

  • Overview & architecture:

    • architecture-overview.adoc – refreshed C4-style description of Bytes vs BytesStore, memory types, threading model, elastic buffers, reference counting, and container diagrams, including context within the OpenHFT stack.
    • algo-overview.adoc – non-cryptographic hash algorithms (OptimisedBytesStoreHash, VanillaBytesStoreHash, XxHash) with trade-offs and usage notes.
    • domestic-overview.adoc – internal utilities (e.g. ReentrantFileLock) and their trade-offs.

  • Requirements & decision log (Nine-Box aligned):

    • project-requirements.adoc – updated with anchors ([[CB-FN-001]] etc.), :sectnums:, and clearer tables for:

      • Functional (CB-FN-*),
      • Non-functional (CB-NF-P-*, CB-NF-S-*, CB-NF-O-*, CB-NF-MP-*),
      • Docs/tests/risk/ops (CB-DOC-*, CB-TEST-*, CB-RISK-*, CB-OPS-*).

    • functional-requirements.adoc – human-friendly slice over the CB-FN-* requirements, organised by domain (core API, encoding/utilities, pooling/lifecycle) with references back to project-requirements.adoc.

    • decision-log.adoc – new ADR-style log:

      • CB-FN-101 – Bytes vs BytesStore API split.
      • CB-FN-102 – elastic buffer growth vs realCapacity.
      • CB-FN-103 – off-heap atomic operations.
      • CB-FN-104 – zero-copy slicing semantics.
      • CB-NF-P-201 – encoding, alignment and pooling strategy.
      • CB-DOC-101 – documentation layering (requirements, narrative guides, testing/security).

  • Operational / quality docs:

    • memory-management.adoc – expanded explanation of reference counting, native memory, mapped files, resource tracing, elastic buffers and best practices, with simple lifetime diagrams.

    • pool-overview.adoc – trade-offs and usage of BytesPool + ScopedResourcePool, with clear guidance on when to pool vs allocate.

    • testing-strategy.adoc – maps test layers (unit, integration, JMH) to CB-TEST-* requirements and describes how they support ISO-style verification.

    • security-review.adoc – Chronicle-Bytes–focused security considerations covering:

      • Bounds checking / Unsafe usage (CB-NF-S-001, CB-NF-S-003),
      • Memory-mapped file sharing (CB-NF-S-002),
      • Resource exhaustion and elastic buffers (CB-NF-S-004, CB-FN-003),
      • Non-cryptographic hashing vs security usage.

    • system-properties.adoc – replaces old docs/systemProperties.adoc:

      • Canonical table of supported system properties (e.g. bytes.guarded, bytes.bounds.unchecked, trace.mapped.bytes, mappedFile.retain, bytes.max-array-len) and their backing constants.

  • API guides and FAQs:

    • api-guide.adoc – significantly enriched:

      • Getting started, core interfaces, creating Bytes instances (heap/direct/mapped), cursor invariants, common patterns, advanced topics (endian, fences, alignment).

    • FAQ.adoc – reorganised:

      • Simpler headings (no manual “1.” numbering, uses :sectnums:),
      • Q&A style with italicised questions,
      • Expanded answers for performance, memory, integration, and troubleshooting.

    • decimal-rendering.adoc & render-overview.adoc – explain the decimaliser / appender model, trade-offs between simple/maximum-precision/BigDecimal strategies, and guidance for hot vs exact paths.

    • util-overview.adoc – summary of utility helpers (compression, interning, length prefixes, escaping testers, decorated exceptions) with a small diagram.

    • ref-overview.adoc – ReentrantFileLock usage, features and trade-offs.

    • wire-integration.adoc – refreshed Wire integration guide:

      • Wire types, Marshallable, MethodWriter/MethodReader, schema evolution, IPC via Chronicle Queue, debugging tips, performance checklist.

  • Images / structure:

    • All diagram assets moved from docs/images/... to src/main/docs/images/... with updated references (e.g. Figure1.png, IntelliJ screenshots).

• All new docs follow the Chronicle house style:

  • British English, ISO-8859-1, :sectnums:, :lang: en-GB, :source-highlighter: rouge.
  • No manual numbering in headings; rely on AsciiDoc section numbering.

3. README / microbench README polish

• README.adoc:

  • Normalises attributes and ordering (:toc:, :lang:, :source-highlighter: rouge, :toclevels:).
  • Updates system properties link to the new canonical src/main/docs/system-properties.adoc.
  • Adds a pointer to project-requirements.adoc and decision-log.adoc for more formal design and requirement rationale.
  • Adds opts=novalidate to code blocks that are pure examples (avoid Asciidoctor complaining about Java syntax).

• microbenchmarks/README.adoc:

  • Adds :toc: and :source-highlighter: rouge.
  • Normalises headings (“How to run”, “Adding benchmarks”), and fixes minor grammar.

4. JavaDoc and package-info improvements

• Added / updated package-info.java across:

  • net.openhft.chronicle.bytes, .internal, .algo, .domestic, .pool, .ref, .render, .util, .internal.migration.

• Tightened JavaDoc for many public types and methods:

  • Bytes, BytesStore, BytesIn, BytesOut, RandomDataInput, RandomDataOutput, StreamingDataInput, StreamingDataOutput, BytesRingBuffer, RingBufferReader, DistributedUniqueTimeProvider, XxHash, OptimisedBytesStoreHash, NativeBytesStore, SingleMappedFile, Invocation, BinaryBooleanReference, ByteStringAppender, AppendableUtil, etc.

• Most of these are narrative/doc-only and do not change runtime behaviour.

5. ByteBuffer compatibility helper

• New internal helper: internal.BufferUtil:

  • Wraps position, limit, clear, flip calls on ByteBuffer via ((Buffer) bb) casts.

  • Replaces direct calls in:

    • NativeBytes, UncheckedNativeBytes, VanillaBytes, some tests, and a few other classes.

  • Fixes Java 8 vs Java 9+ compatibility around ByteBuffer covariant return types without relying on “redundant” casts that some IDEs / tools remove.

6. Tests and benchmarks

• Many tests updated for:

  • ISO-8859-1 explicit encoding (.getBytes(ISO_8859_1) / new String(..., ISO_8859_1)).
  • Clearer names and assertions (e.g. isDirectMemoryReturnsExpectedValue, compactResetsPositionsAndLimits, equality tests exercising heap/direct combinations).
  • Deterministic randomness (new Random(1L) instead of default-seeded RNG).
  • Extra assertions to ensure black-hole variables and temp files are actually used/deleted.

• JLBH / perf harnesses:

  • Small clean-ups (finals, average time calculations) with no behavioural impact on the measured code.

7. Build / dependency

• pom.xml:

  • Bump net.openhft:third-party-bom from 3.27ea5 to 3.27ea7.
  • This pulls in updated third-party versions (including the SLF4J/Logback updates already researched elsewhere), without any direct Chronicle-Bytes code changes in this PR.

[sonarqubecloud]

Quality Gate passed

Issues
360 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
1.4% Duplication on New Code

See analysis details on SonarQube Cloud