2024.8.1

Changes

• Reduce frequency of unmanaged memory use warning @phofl (#8834)
• Update gpuCI RAPIDS_VER to 24.10 @github-actions (#8786)
• Avoid RuntimeError: dictionary changed size during iteration in Server._shift_counters() @hendrikmakait (#8828)
• Improve concurrent close for scheduler @hendrikmakait (#8829)
• MINOR: Extract truncation logic out of partial concatenation in P2P rechunking @hendrikmakait (#8826)
• Avoid excessive attribute access overhead for remove_from_task_prefix_count @fjetter (#8821)
• Avoid key validation if validation is disabled @fjetter (#8822)
• Log worker\_client event @jrbourbeau (#8819)
• Drop support for Python 3.9 @phofl (#8793)

See the Changelog for more information.

2024.8.0

Changes

• Run graph normalisation after dask order @phofl (#8818)
• Update large graph size warning to remove scatter recommendation @phofl (#8815)
• Fail tasks exceeding no-workers-timeout @hendrikmakait (#8806)
• Fix exception handling for NannyPlugin.setup and NannyPlugin.teardown @hendrikmakait (#8811)
• Fix exception handling for WorkerPlugin.setup and WorkerPlugin.teardown @hendrikmakait (#8810)
• Fix typo in transitions event logging @alex-rakowski (#8812)
• Fix if-else for send_recv_from_rpc @phofl (#8809)
• Ensure that adaptive only stops once @hendrikmakait (#8807)
• Reduce noise from GC-related logging @hendrikmakait (#8804)
• Remove unused delete_interval and synchronize_worker_interval from Scheduler @hendrikmakait (#8801)
• Change log level for Compute Failed log message @phofl (#8802)
• Add Prometheus metric for time spent on GC @hendrikmakait (#8803)
• Add Prometheus metrics for dask_scheduler_workers_{added|removed}_total @hendrikmakait (#8798)
• Add log event for worker-ttl-timeout @hendrikmakait (#8800)
• Add Prometheus metrics for dask_scheduler_client_connections_{added|removed}_total @hendrikmakait (#8799)
• Fix restarting workers with PackageInstall plugin @hendrikmakait (#8794)
• Make stealing more robust @hendrikmakait (#8788)
• leave a warning about future instantiation @fjetter (#8782)

See the Changelog for more information.

2024.7.1

Changes

• Ensure Locks always register with scheduler @fjetter (#8781)
• Temporarily pin setuptools \< 71 @jrbourbeau (#8785)
• Restore len() on TaskPrefix @hendrikmakait (#8783)
• Avoid false positives for p2p-failed log event @hendrikmakait (#8777)
• Expose paused and retired workers separately in prometheus @phofl (#8613)
• Creating transitions-failures log event @alex-rakowski (#8776)
• Implement HLG layer for P2P rechunking @hendrikmakait (#8751)
• Ensure Lock will not lock up in case of worker failures @fjetter (#8770)
• Add another test for a possible deadlock scenario caused by #8703 @hendrikmakait (#8769)
• Raise an error if compute on persisted collection with released futures @fjetter (#8764)
• Re-raise P2PConsistencyError from failed P2P tasks. @hendrikmakait (#8748)
• Robuster faster tests memory sampler @fjetter (#8758)
• fix scheduler_bokeh::test_shuffling @fjetter (#8766)
• increase timeouts for pubsub::test_client_worker @fjetter (#8765)
• Factor out async taskgroup @fjetter (#8756)
• Don't sort keys lexicographically in worker table @fjetter (#8753)
• Use functools.cache instead of functools.lru_cache for url_escape & _expects_comm @jonded94 (#8762)
• Robuster deeply nested structures @fjetter (#8730)
• Adding HLG to MAP @alex-rakowski (#8740)
• Add close worker button to worker info page @jrbourbeau (#8742)

See the Changelog for more information.

2024.7.0

Changes

• Fix assert\_eq import from cudf @jrbourbeau (#8747)
• Log traceback upon task error @hendrikmakait (#8746)
• Update system monitor when polling Prometheus metrics @hendrikmakait (#8745)
• Bump pandas to 2.0 in mindeps build @jrbourbeau (#8743)
• Refactor event logging functionality into broker @hendrikmakait (#8731)
• Drop support for pandas 1.X @hendrikmakait (#8741)
• Remove is_python_shutting_down @hendrikmakait (#8492)
• Fix test_task_state_instance_are_garbage_collected @hendrikmakait (#8735)
• Fix flake in test_deadline caused by floating-point inaccuracy @hendrikmakait (#8736)
• Fix pynvml handles @quasiben (#8693)
• get_ip: handle getting 0.0.0.0 (#2554) @AdamWill (#8712)
• Remove FutureWarning in test_task_state_instance_are_garbage_collected @hendrikmakait (#8734)
• Fix mindeps-testing on CI @hendrikmakait (#8728)
• Extract tests related to event-logging into separate file @hendrikmakait (#8733)
• TST: Use safer context for ProcessPoolExecutor @QuLogic (#8715)
• Cache URL encoding of worker addresses in dashboard @fjetter (#8725)
• More robust bokeh test_shuffling @fjetter (#8727)
• doc: fix typo @SultanOrazbayev (#8711)
• More useful warning if a plugin type is provided instead of instance @fjetter (#8689)
• Improve error on cancelled tasks due to disconnect @hendrikmakait (#8705)
• Deprecate Pub and Sub @hendrikmakait (#8724)
• TST: Fix wait condition on test_forget_errors @QuLogic (#8714)
• Skip test_deadlock_dependency_of_queued_released is queueing is disabled @hendrikmakait (#8723)
• Fix test_quiet_client_close @hendrikmakait (#8722)
• Fix cleanup iteration in save_sys_modules @QuLogic (#8713)
• Add quotes to missing bokeh installation commands @jrbourbeau (#8717)

See the Changelog for more information.

2024.6.2

Changes

• profile._f_lineno: handle next_line being None in Python 3.13 @AdamWill (#8710)

See the Changelog for more information.

2024.6.1

Changes

• Fix deadlock when transitioning dependency of queued task to released @hendrikmakait (#8703)
• Bump release-drafter/release-drafter from 5 to 6 @dependabot (#8699)

See the Changelog for more information.

2024.6.0

Changes

• Avoid rounding error in test_prometheus_collect_count_total_by_cost_multipliers @hendrikmakait (#8687)
• Log key collision count in update_graph log event @hendrikmakait (#8692)
• Automate GitHub Releases when new tags are pushed @jacobtomlinson (#8626)
• Fix log_event plugin hook with multiple topics @hendrikmakait (#8691)
• Rename safe to expected in Scheduler.remove_worker. @hendrikmakait (#8686)
• [P2P] Log event during failure @hendrikmakait (#8663)
• Eagerly update aggregate statistics for TaskPrefix instead of calculating them on-demand @hendrikmakait (#8681)
• Improve graph submission time for P2P rechunking by avoiding unpack recursion into indices @fjetter (#8672)
• Adding_suspicious_worker @alex-rakowski (#8647)
• Improved errors and reduced logging for P2P RPC calls @hendrikmakait (#8666)
• Adjust P2P tests for dask-expr @hendrikmakait (#8662)
• Iterate over copy of Server.digests_total_since_heartbeat to avoid RuntimeError @hendrikmakait (#8670)
• Log task state in "Compute Failed" message @hendrikmakait (#8668)
• Add Prometheus gauge for task groups @hendrikmakait (#8661)
• Fix too strict assertion in shuffle code for pandas subclasses @jorisvandenbossche (#8667)
• Reduce noise from erring tasks that are not supposed to be running @hendrikmakait (#8664)
• Reduce task group count for partial P2P rechunks @hendrikmakait (#8655)
• Update gpuCI RAPIDS_VER to 24.08 @github-actions (#8652)
• Submit collections metadata to scheduler @fjetter (#8612)
• DOC: fix indent in code example in Update task-launch.rst @raybellwaves (#8650)
• Avoid multiple WorkerState sphinx error @jrbourbeau (#8643)
• Fix clocks on Windows with Python 3.13 @vstinner (#8642)
• Fix "Print host info" CI step on Mac OS (arm64) @hendrikmakait (#8638)
• Move P2P's split_by_* tests @hendrikmakait (#8631)
• Bump actions/checkout from 4.1.2 to 4.1.3 @dependabot (#8628)

See the Changelog for more information.

2024.4.2

Version 2024.4.2