v4.6.1

What's Changed

• Update misaligned script by @thomasyu888 in #1144
• [SYNPY-1423] update mkdocstrings dependency for 1.8.0 by @danlu1 in #1145
• Add procps to Dockerfile to enable use of container in Nextflow by @adamjtaylor in #1146
• [SYNPY-1548] Swap to a FIFO queue by @BryanFauble in #1147

New Contributors

• @adamjtaylor made their first contribution in #1146

Full Changelog: v4.6.0...v4.6.1-rc

v4.6.0

Highlights

• Patching an issue present in 4.5.1 that broke downloads from google bucket files.
• Support for python 3.12
• Drop support for python 3.8

What's Changed

• [SYNPY-1521] Fixes asDataFrame kwarg Collision Issue by @BWMac in #1132
• [SYNPY-1520] Support python 3.12 by @BryanFauble in #1133
• [SYNPY-1517] Deprecate Python 3.8 Support by @BWMac in #1134
• [SYNPY-1524] Fixes ghcr-build-and-push-on-develop and ghcr-build-and-push-on-release Workflows by @BWMac in #1136
• [SYNPY-1524] Updates Dockerfile by @BWMac in #1137
• [SYNPY-1525] Verify expiration param in url by @BryanFauble in #1139
• [SYNPY-1513] Validate input submission ID in getSubmission(...) by @jaymedina in #1135
• [SYNPY-1522] Add note on lost shared libraries to install instructions for conda by @jaymedina in #1140

Full Changelog: v4.5.1...v4.6.0-rc

v4.5.1

4.5.1 (2024-09-19)

Highlights

• Telemetry auto instrumentation:
  • By employing the usage of the opentelemetry-python-contrib
    we are able to take advantage of the support for auto instrumenting several
    libraries in use by this client. This telemetry data is paramount for exceptional
    observability. Check out the readme for a quickstart.
• owner_id (Or the ID of the logged in profile) is available on logged in Synapse instance via syn.credentials.owner_id

What's Changed

• [FDS-2386] Add owner id to the credentials to be used later on by @BryanFauble in #1130
• [FDS-2386] Global client caching, Telemetry auto instruementation by @BryanFauble in #1129

Full Changelog: v4.5.0...v4.5.1-rc

v4.5.0

Highlights

• Improved handling of pre-authorized requests:
  • During long running asynchronous download operations (i.e. syncing a folder with many files),
    some pre-authorized URLs were expiring before they were used to download the file. This
    change will now check the expiration datetime of the URL before using it.
    If the URL is expired it will be refreshed.
• Added support for pandas.read_csv keyword arguments:
  • Added support for all pandas.read_csv keyword arguments to the asDataFrame method of the CsvFileTable class.
  • This enables custom handling of missing values and default values when reading a Synapse table to a pandas DataFrame. See the pandas documentation for a full list of available keyword arguments.

What's Changed

• [SYNPY-1514] Handle Expired Pre-Signed URLs by @BWMac in #1126
• [SYNPY-1509] Updates NA handling for CSVFileTable.asDataFrame by @BWMac in #1127
• [FDS-2373] Update retry delay, add some otel traces, add retry to url creation by @BryanFauble in #1128

Full Changelog: v4.4.1...v4.5.0

v4.4.1

What's Changed

• Update file tutorial with missing line by @rxu17 in #1123
• [SYNPY-1436] avoid duplicate user profile call during login by @danlu1 in #1124
• [SYNPY-1511] Correct missing synapse client propogation and caching by @BryanFauble in #1125

New Contributors

• @rxu17 made their first contribution in #1123

Full Changelog: v4.4.0...v4.4.1-rc

v4.4.0

Highlights

• Downloading files stability improvements:
  • The download algorithm has been re-written to focus on stability, reliability,
    and performance.
  • Checkout the results of our benchmarking.
    Compared to v4.3.0 we are seeing a 10-50% decrease in overall transfer time for
    most test cases.
• New Docker containers every release:
  • Every production, release candidate, and develop release will create a new docker
    image available here
• Credentials passed by command line argument will now be evaluated before credentials
  stored in the ~/.synapseConfig file.

What's Changed

• [SYNPY-1341 & SYNPY-1479] Change syntax for conditional of ghcr release job by @jaymedina in #1102
• [SYNPY-1483] Update default order for credential provider by @BryanFauble in #1105
• [SYNPY-1482] Updating to remove loop executor during annotation store by @BryanFauble in #1104
• [SYNPY-1476] Splitting logic for downloading content from client.py by @BryanFauble in #1103
• [SYNPY-1485] feat: added isort to pre-commit hook by @linglp in #1108
• [SYNPY-1485] Running pre-commit on all files to run isort by @BryanFauble in #1109
• [SYNPY-1447] Update fillna method to work directly off original df by @jaymedina in #1113
• [SYNPY-1457] Fix deprecated Node.js build warnings by @jaymedina in #1112
• [SYNPY-1487] Fix missing coverage report by @jaymedina in #1114
• [SYNPY-1476] Refactoring multithreaded downloads to be more fault tolerant by @BryanFauble in #1107
• [SYNPY-1490] Merge develop into master by @BryanFauble in #1116
• [SYNPY-1494] Remove process pool executor by @BryanFauble in #1117
• Update os matrix for check-deploy by @BryanFauble in #1118
• [SYNPY-1476] Skip reading response content on error by @BryanFauble in #1119
• [SYNPY-1504] Remove download_location from File and rely only on Path by @BryanFauble in #1120

Full Changelog: v4.3.1...v4.4.0

v4.3.1

Bug Fixes

• [SYNPY-1494] - loky crashed uwsgi process in schematic manifest submission

Full Changelog: v4.3.0...v4.3.1-rc

v4.3.0

Highlights

• New tutorial:
  • Uploading data in bulk
    is our newest tutorial. It covers the basics of working with manifest files to manage synapse projects.
• Updates to syncToSynapse:
  • The syncToSynapse function has been refactored to improve performance and
    reliability.
  • Minor behavior change: File entities will no longer have it's version
    incremented during no-op changes. Only when file content, or fields on the file
    has been updated will a version number be incremented.
  • Optional booleans merge_existing_annotations and
    associate_activity_to_new_version have been added. Both are used to give more
    fine tuned control when working with this interface.
  • Check out the changes in the reference docs.

What's Changed

• [SYNPY-1474] Correct order of credentials in comment by @BryanFauble in #1091
• [SYNPY-1456] Flaky integration tests due to connection issues by @BryanFauble in #1085
• [SYNPY-1456] Revert to old ubuntu by @BryanFauble in #1092
• [SYNPY-1466] update user agent for command line by @danlu1 in #1090
• Update README.md by @allaway in #1093
• [SYNPY-1470] Remove SonarCloud's dependency on test job by @danlu1 in #1095
• [SYNPY-1470] Always run SonarCloud as long as previous jobs are not cancelled by @danlu1 in #1096
• [SYNPY-1356] Refactor sync to synapse by @BryanFauble in #1083
• [SYNPY-1356] Logic around disassociating Activity from File Entity by @BryanFauble in #1098
• [SYNPY-1341] Add CI step to containerize and upload synpy to the GHCR by @jaymedina in #1100
• [SYNPY-1384] Create uploading data in bulk tutorial by @BryanFauble in #1101

Full Changelog: v4.2.0...v4.3.0-rc

v4.2.0

Highlights

• Continued Async Support:
  • Multi-threaded Uploads Are Now Async: This change optimizes the upload process to handle multiple tasks simultaneously: uploading file parts concurrently, performing MD5 checksum calculations in separate processes, and making HTTP calls asynchronously. Further, memory management techniques were implemented to prevent crashes on low-resource EC2 instances. Benchmark results can be found here.
  • HTTPX Support & Async Client Instances: In this update, support for the HTTPX library has been introduced, enhancing the functionality of Synapse with two new async client instances for seamless interaction with Synapse and Storage Providers. Additionally, the retry and back-off mechanisms have been revamped to improve reliability and performance, ensuring smoother operation even under challenging network conditions.

What's Changed

• [SYNPY-1419] Httpx async client by @BryanFauble in #1076
• [SYNPY-1453] Guard around modified time item in cache by @BryanFauble in #1080
• [SYNPY-1420] Re-write uploads to mix async, multi-threading, and multi-processing by @BryanFauble in #1078
• [SYNPY-1417] Updates for the annotation model by @BryanFauble in #1081
• [SYNPY-1456] Flaky integration tests by @BryanFauble in #1082
• [SYNPY-1465] Adding Annotations to sync and async docs by @jaymedina in #1087

Full Changelog: v4.1.1...v4.2.0

v4.1.1

Highlights

• New Downloading Data in Bulk Tutorial
  • See Downloading data in bulk tutorial for more details on downloading data in bulk from Synapse.
• Downloading Files Troubleshooting Improvement
  • Improved error logging for when users are downloading files using commands like get-download-list from Synapse.

What's Changed

• Regen pipfile.lock for dependabot alert by @BryanFauble in #1069
• [SYNPY-1383] Downloading data in bulk tutorial by @BryanFauble in #1070
• [SYNPY-1443] Correct which fields are merged if there is a change by @BryanFauble in #1072
• [SYNPY-1442] Log update to capture exception when download-list fails by @BryanFauble in #1071
• [SYNPY-1445] Raise exception on HTTP error when retrieving file from storage location by @BryanFauble in #1074

Full Changelog: v4.1.0...v4.1.1