Duplicate output callbacks across Python and JS namespaces fail if both set to allow_duplicate

[dfrtz]

Describe the bug

Duplicate output callbacks across Python and JavaScript namespaces fail if both set to allow_duplicate. They work, and allow parallel runs, if only one is enabled, and the other disabled.

This may be related to #2486, but I believe these specific examples indicates more of a bug than expected behavior. The reason I say this is a "bug", is because it does technically work, even with the same Input/Output combo, but only if you do not put allow_duplicate on both. If you set it on either/or, then you are allowed to overlap it successfully. If you set it on both, then it complains that it needs to be set, even though it is set (confusing, regardless of final outcome of this ask).

Additionally, while I will demonstrate this occurs with matching outputs, it does also happen if the output combo is not the same (but the input is).

Expected behavior

If there are two callbacks (one python, and one javascript), even if matching signatures, they should not be considered duplicates, in order to maximize client application performance and flexibility.

Environment

Happens on older and current versions. Tested on latest as of:

dash                 2.18.2
dash-core-components 2.0.0
dash-html-components 2.0.0
dash-table           5.0.0

Examples

Minimum repo:

import time

import dash
from dash import Input
from dash import Output
from dash import html

app = dash.Dash(__name__)
app.layout = html.Div([
    html.Button('Button', id='btn-py'),
    html.Div('Press a button', id='output'),
])
app.clientside_callback(
    """function jsCallback(nClicks) {return `JavaScript called ${nClicks} times`;}""",
    # Output("output", "children"),
    Output("output", "children", allow_duplicate=True),
    Input("btn-py", "n_clicks"),
    prevent_initial_call=True,
)

@app.callback(
    # Output("output", "children"),
    Output("output", "children", allow_duplicate=True),
    Input("btn-py", "n_clicks"),
    prevent_initial_call=True,
)
def py_callback(n_clicks) -> str:
    time.sleep(3)
    return f'Python called {n_clicks} times'


if __name__ == "__main__":
    app.run_server(
        debug=True,
    )

To make it work without error, as I would argue the expected behavior should be, uncomment one of the ones without allow_duplicate and comment out the one with allow_duplicate in the same callback.

Example of the error message, which is a bit misleading in order to make it work, since to make it work you must actually turn it off on one of them.

Duplicate callback outputs
In the callback for output(s):
  output.children@6bffe2e30aca9ad92bc2624962a39adfeb1183a1601f8386314e482125bd4027
Output 0 (output.children@6bffe2e30aca9ad92bc2624962a39adfeb1183a1601f8386314e482125bd4027) is already in use.
To resolve this, set `allow_duplicate=True` on
duplicate outputs, or combine the outputs into
one callback function, distinguishing the trigger
by using `dash.callback_context` if necessary.

"Real world" example signature, sanitized from a production application, this works (and vice versa for where it is set):

@app.callback(
    Output("rerun-timer", "disabled"),
    Output("output", "children"),
    Output("rerun-btn", "children"),
    Output("popup", "data"),
    Input("rerun-btn", "n_clicks"),
    State("request", "data"),
)
def onPress...

app.clientside_callback(
    ClientsideFunction(namespace="page1", function_name="onPress"),
    Output("output", "children"),
    Output("rerun-btn", "children"),
    Input("rerun-btn", "n_clicks"),
)

Additional thoughts

This is just a quick example to illustrate the issue. I understand there are arguments to be made for "combining" callbacks, but in my opinion that only applies to within the same "namespace" (python vs javascript). I also understand there is other functionality built in to callbacks such as loading_state and running=. At least for my Dash projects over the years (personal and professional), I have found many situations where it is beneficial to be allowed this type of combination, even while leveraging all the other features. This allows maximizing performance of the application if certain actions can be placed entirely in Javascript, and others in Python.

For a while, even before native duplicates were allowed in Dash, I have been patching dash.Dash to allow a duplicate_output_group argument while creating the callback IDs to allow full control over duplicates via Python vs Javascript namespaces. Basically it just attaches either -py or -js to the end of the callback IDs. I was going to submit this namespace logic upstream a blue moon or two ago, but never managed to get around to it before the native duplicate logic was built in. I am hoping to leverage the native allow_duplicate logic of Dash moving forward, so that I no longer have to patch the ID creation in applications.

If it is decided this should not be allowed, then I can continue patching, but it would be nice to have full control over this behavior natively in the applications I maintain. If that is the choice, then I would ask that the error needs tweaking. Additionally, I could explain more about the namespace logic I use currently in dash.Dash, if there is interest in supporting this in a different way (matching duplicates allowed via namespaces).

[dfrtz]

Just for the heck of it, I am going to describe how I allow namespaces for duplicate callbacks in Dash apps, just in case anyone is curious, and it is decided to consider the behavior in this issue "working as designed" 😄

Here is the flow of the overrides I use currently to allow optimizing actions between Javascript, and Python.
dash.Dash.callback
calls:
dash._callback.callback
with:
duplicate_output_group="py"

dash.Dash.clientside_callback
calls:
dash._callback.register_clientside_callback
with:
duplicate_output_group="js"

dash._callback.register_clientside_callback
is updated to have:
duplicate_output_group: str = "global"
and calls:
dash._callback.insert_callback
with:
duplicate_output_group

dash._callback.insert_callback
is updated to have:
duplicate_output_group: str = "global"
and calls:
dash._utils.create_callback_id
with:
duplicate_output_group

dash._utils.create_callback_id
is updated to have:
duplicate_output_group: str = "global"
and performs the following in the ID creation:
_id += f"@{hashed_inputs}{duplicate_output_group}"

If this were added to the core library, or something similar, then it would be possible to allow users to explicitly allow duplicate callbacks across both Python and JavaScript. I can see an argument for not allowing duplicates across the global namespace by default, but in my opinion devs should be able to make the final decision on whether to have parallel callbacks, in order to maximize UX, and reduce load on servers. Ultimately they know their apps best.

I have been using this for a long time in various applications, without any noticeable side effects (that I have seen anyways), but I can't guarantee that it does not impact some use case. If it does, perhaps there is alternative way to support this feature. In my opinion, this is a great ability to have if not allowing duplicates across namespaces by default, and greatly improves flexibility in what applications can be created, while allowing devs to optimize performance and UX.

[gvwilson]

@T4rk1n do you agree this is a bug, and if so, is it still present in 3.0?