Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Please port to numpy-2 #1149

Open
yarda opened this issue Jan 6, 2025 · 20 comments
Open

Please port to numpy-2 #1149

yarda opened this issue Jan 6, 2025 · 20 comments
Assignees
@jopohl

Comments

@yarda
Copy link
Contributor

yarda commented Jan 6, 2025

Is your feature request related to a problem?

Please port to numpy-2. Some distros (e.g. Fedora) no more provide numpy<2.

Describe the solution you'd like

The GUI works with numpy-2.

Describe alternatives you've considered

No way to distro package it to fedora-42, because there is no numpy-1

Additional context

numpy-2 migration guide: https://numpy.org/devdocs/numpy_2_0_migration_guide.html
Maybe I could port it myself, but it will take time, so at least reporting it for the RFE to be tracked.
@yarda
Copy link
Contributor Author

yarda commented Jan 6, 2025

BTW I compiled it with numpy-2.2.1 and it seems it runs fine, at least I didn't observe the startup crash. Maybe if compiled with numpy-2 it can be run with numpy-2? And the crash will occur only if compiled with numpy-2 and run with numpy-1? If this is the case, the numpy dep should be relaxed.

@andynoack
Copy link
Collaborator

Hey @yarda,
thanks for trying out! We appreciate a pull request if you are able to get it running with the current numpy version. The unit tests of URH should inidicate whether there are problems.
Best regards!

@yarda
Copy link
Contributor Author

yarda commented Jan 6, 2025

Sure, I will check and I will also enable the unit tests during Fedora build process. Could you point me to instructions how to run them all? I tried python3 setup.py test but it seems this is not the correct way.

@andynoack
Copy link
Collaborator

You can execute the tests manually or with PyCharm for example. You can, however, also use the CI Pipeline from GitHub that we have setup for this purpose.

@yarda
Copy link
Contributor Author

yarda commented Jan 7, 2025

I used pytest -s -v tests to run the tests. Bad thing is that it requires X server, but I think I could workaround it with the null X server on the build machine. Now, I run it manually. The numpy-1 was run on Fedora 41, the numpy-2 was run on Fedora 42 (rawhide), but the python versions should be the same.

numpy-1:
342 tests passed

numpy-2:
333 tests passed, 3 failed, 6 skipped due to coredump

So it isn't compatible, but it isn't so bad.

@yarda
Copy link
Contributor Author

yarda commented Jan 7, 2025

numpy-1 warnings:

====================================================================================== warnings summary ======================================================================================
tests/auto_interpretation/test_auto_interpretation_integration.py: 1 warning
tests/auto_interpretation/test_center_detection.py: 1 warning
tests/auto_interpretation/test_message_segmentation.py: 1 warning
tests/auto_interpretation/test_noise_detection.py: 1 warning
tests/test_filter.py: 6 warnings
  /usr/lib64/python3.13/site-packages/urh/signalprocessing/Signal.py:210: DeprecationWarning: Python 3.14 will, by default, filter extracted tar archives and reject files or modify their metadata. Use the filter argument to control this behavior.
    obj.extract(members[0], QDir.tempPath())

tests/awre/test_address_engine.py: 2 warnings
tests/awre/test_awre_real_protocols.py: 2 warnings
tests/awre/test_generated_protocols.py: 9 warnings
tests/test_fuzzing_profile.py: 1 warning
tests/test_maincontroller_gui.py: 1 warning
tests/test_project_manager.py: 5 warnings
tests/test_protocol_label_dialog.py: 3 warnings
tests/test_signal_tab_GUI.py: 1 warning
tests/test_simulator_tab_gui.py: 1 warning
  /usr/lib64/python3.13/site-packages/urh/signalprocessing/ProtocolAnalyzer.py:733: DeprecationWarning: Testing an element's truth value will always return True in future versions.  Use specific 'len(elem)' or 'elem is not None' test instead.
    if not root:

tests/awre/test_address_engine.py: 5 warnings
tests/awre/test_awre_real_protocols.py: 8 warnings
tests/awre/test_generated_protocols.py: 15 warnings
tests/test_fuzzing_profile.py: 2 warnings
tests/test_maincontroller_gui.py: 2 warnings
tests/test_project_manager.py: 9 warnings
tests/test_protocol_label_dialog.py: 3 warnings
tests/test_signal_tab_GUI.py: 1 warning
tests/test_simulator.py: 6 warnings
tests/test_simulator_tab_gui.py: 8 warnings
  /usr/lib64/python3.13/site-packages/urh/signalprocessing/Ruleset.py:143: DeprecationWarning: Testing an element's truth value will always return True in future versions.  Use specific 'len(elem)' or 'elem is not None' test instead.
    if tag:

tests/awre/test_address_engine.py: 1218 warnings
tests/awre/test_awre_preprocessing.py: 2057 warnings
tests/awre/test_awre_real_protocols.py: 340 warnings
tests/awre/test_checksum_engine.py: 40 warnings
tests/awre/test_generated_protocols.py: 1463 warnings
tests/awre/test_length_engine.py: 673 warnings
tests/awre/test_partially_labeled.py: 410 warnings
tests/awre/test_sequence_number_engine.py: 139 warnings
tests/test_analysis_tab_GUI.py: 712 warnings
  /usr/lib64/python3.13/site-packages/urh/awre/Histogram.py:102: DeprecationWarning: tostring() is deprecated. Use tobytes() instead.
    values[vector[start : start + length].tostring()].append(i)

tests/awre/test_address_engine.py: 46 warnings
tests/awre/test_awre_real_protocols.py: 22 warnings
tests/awre/test_generated_protocols.py: 51 warnings
  /usr/lib64/python3.13/site-packages/urh/awre/engines/AddressEngine.py:534: DeprecationWarning: tostring() is deprecated. Use tobytes() instead.
    result[p1].add(val.tostring())

tests/awre/test_address_engine.py: 46 warnings
tests/awre/test_awre_real_protocols.py: 22 warnings
tests/awre/test_generated_protocols.py: 51 warnings
  /usr/lib64/python3.13/site-packages/urh/awre/engines/AddressEngine.py:535: DeprecationWarning: tostring() is deprecated. Use tobytes() instead.
    result[p2].add(val.tostring())

tests/awre/test_address_engine.py: 530 warnings
tests/awre/test_awre_real_protocols.py: 55 warnings
tests/awre/test_generated_protocols.py: 217 warnings
  /usr/lib64/python3.13/site-packages/urh/awre/engines/AddressEngine.py:326: DeprecationWarning: tostring() is deprecated. Use tobytes() instead.
    if i in rng.message_indices and rng.value.tostring() in addresses

tests/awre/test_address_engine.py: 410 warnings
tests/awre/test_awre_real_protocols.py: 44 warnings
tests/awre/test_generated_protocols.py: 55 warnings
  /usr/lib64/python3.13/site-packages/urh/awre/engines/AddressEngine.py:357: DeprecationWarning: tostring() is deprecated. Use tobytes() instead.
    rng.value.tostring()

tests/awre/test_address_engine.py: 33 warnings
tests/awre/test_awre_real_protocols.py: 11 warnings
tests/awre/test_generated_protocols.py: 37 warnings
tests/awre/test_partially_labeled.py: 18 warnings
  /usr/lib64/python3.13/site-packages/urh/awre/engines/AddressEngine.py:211: DeprecationWarning: tostring() is deprecated. Use tobytes() instead.
    if rng.value.tostring() == address

tests/awre/test_address_engine.py: 120 warnings
tests/awre/test_awre_real_protocols.py: 11 warnings
tests/awre/test_generated_protocols.py: 162 warnings
  /usr/lib64/python3.13/site-packages/urh/awre/engines/AddressEngine.py:330: DeprecationWarning: tostring() is deprecated. Use tobytes() instead.
    address = matching[0].value.tostring()

tests/awre/test_address_engine.py: 240 warnings
tests/awre/test_awre_real_protocols.py: 19 warnings
tests/awre/test_generated_protocols.py: 183 warnings
  /usr/lib64/python3.13/site-packages/urh/awre/engines/AddressEngine.py:343: DeprecationWarning: tostring() is deprecated. Use tobytes() instead.
    and rng.value.tostring() in addresses

tests/awre/test_address_engine.py: 240 warnings
tests/awre/test_awre_real_protocols.py: 18 warnings
tests/awre/test_generated_protocols.py: 55 warnings
  /usr/lib64/python3.13/site-packages/urh/awre/engines/AddressEngine.py:347: DeprecationWarning: tostring() is deprecated. Use tobytes() instead.
    lambda r: r.value.tostring() == address, prev_matching

tests/awre/test_address_engine.py: 8 warnings
tests/awre/test_awre_real_protocols.py: 6 warnings
tests/awre/test_generated_protocols.py: 30 warnings
tests/awre/test_partially_labeled.py: 12 warnings
  /usr/lib64/python3.13/site-packages/urh/awre/engines/AddressEngine.py:76: DeprecationWarning: tostring() is deprecated. Use tobytes() instead.
    p: [addr.tostring()]

tests/awre/test_address_engine.py: 8 warnings
tests/awre/test_awre_real_protocols.py: 6 warnings
tests/awre/test_generated_protocols.py: 30 warnings
tests/awre/test_partially_labeled.py: 12 warnings
  /usr/lib64/python3.13/site-packages/urh/awre/engines/AddressEngine.py:318: DeprecationWarning: tostring() is deprecated. Use tobytes() instead.
    address = self.known_addresses_by_participant[participant].tostring()

tests/awre/test_generated_protocols.py::TestGeneratedProtocols::test_with_checksum
  /tmp/tests/awre/test_generated_protocols.py:155: DeprecationWarning: tostring() is deprecated. Use tobytes() instead.
    known_participant_addresses[0].tostring(),

tests/awre/test_generated_protocols.py::TestGeneratedProtocols::test_with_checksum
  /tmp/tests/awre/test_generated_protocols.py:159: DeprecationWarning: tostring() is deprecated. Use tobytes() instead.
    known_participant_addresses[1].tostring(),

tests/awre/test_generated_protocols.py::TestGeneratedProtocols::test_with_one_address_one_message_type
  /tmp/tests/awre/test_generated_protocols.py:224: DeprecationWarning: tostring() is deprecated. Use tobytes() instead.
    known_participant_addresses[0].tostring(),

tests/awre/test_generated_protocols.py::TestGeneratedProtocols::test_with_one_address_one_message_type
  /tmp/tests/awre/test_generated_protocols.py:228: DeprecationWarning: tostring() is deprecated. Use tobytes() instead.
    known_participant_addresses[1].tostring(),

tests/awre/test_generated_protocols.py::TestGeneratedProtocols::test_with_only_one_address
  /tmp/tests/awre/test_generated_protocols.py:175: DeprecationWarning: tostring() is deprecated. Use tobytes() instead.
    known_participant_addresses[0].tostring(),

tests/awre/test_generated_protocols.py::TestGeneratedProtocols::test_with_only_one_address
  /tmp/tests/awre/test_generated_protocols.py:179: DeprecationWarning: tostring() is deprecated. Use tobytes() instead.
    known_participant_addresses[1].tostring(),

tests/awre/test_generated_protocols.py::TestGeneratedProtocols::test_without_preamble_24_messages
  /tmp/tests/awre/test_generated_protocols.py:247: DeprecationWarning: tostring() is deprecated. Use tobytes() instead.
    known_participant_addresses[0].tostring(),

tests/awre/test_generated_protocols.py::TestGeneratedProtocols::test_without_preamble_24_messages
  /tmp/tests/awre/test_generated_protocols.py:251: DeprecationWarning: tostring() is deprecated. Use tobytes() instead.
    known_participant_addresses[1].tostring(),

tests/test_file_operator.py::TestFileOperator::test_uncompress_archives
tests/test_file_operator.py::TestFileOperator::test_uncompress_archives
tests/test_file_operator.py::TestFileOperator::test_uncompress_archives
  /usr/lib64/python3.13/site-packages/urh/util/FileOperator.py:262: DeprecationWarning: Python 3.14 will, by default, filter extracted tar archives and reject files or modify their metadata. Use the filter argument to control this behavior.
    obj.extract(member, temp_dir)

tests/test_fuzzing_profile.py: 2 warnings
tests/test_simulator.py: 6 warnings
tests/test_simulator_tab_gui.py: 6 warnings
  /usr/lib64/python3.13/site-packages/urh/signalprocessing/Message.py:633: DeprecationWarning: Testing an element's truth value will always return True in future versions.  Use specific 'len(elem)' or 'elem is not None' test instead.
    if message_type_tag:

tests/test_modulator_gui.py::TestModulatorGUI::test_signal_view
  /usr/lib64/python3.13/site-packages/urh/controller/dialogs/ModulatorDialog.py:99: RuntimeWarning: divide by zero encountered in scalar divide
    num2 = abs(int(num2 / num1)) * num1

tests/test_project_manager.py::TestProjectManager::test_save_and_load_participants
tests/test_project_manager.py::TestProjectManager::test_save_and_load_participants
tests/test_project_manager.py::TestProjectManager::test_save_and_load_participants
tests/test_project_manager.py::TestProjectManager::test_save_and_load_with_fieldtypes
tests/test_simulator_tab_gui.py::TestSimulatorTabGUI::test_save_and_load
  /usr/lib64/python3.13/site-packages/urh/util/ProjectManager.py:512: DeprecationWarning: Testing an element's truth value will always return True in future versions.  Use specific 'len(elem)' or 'elem is not None' test instead.
    if messages_tag:

tests/test_protocol_sniffer.py: 2 warnings
tests/test_send_recv_dialog_gui.py: 1 warning
tests/test_simulator.py: 8 warnings
tests/test_simulator_tab_gui.py: 2 warnings
  /usr/lib64/python3.13/site-packages/urh/signalprocessing/IQArray.py:96: DeprecationWarning: tostring() is deprecated. Use tobytes() instead.
    return self.__data.tostring()

tests/test_send_recv_dialog_gui.py::TestSendRecvDialog::test_receive
  /tmp/tests/test_send_recv_dialog_gui.py:166: DeprecationWarning: tostring() is deprecated. Use tobytes() instead.
    sock.sendall(data.tostring())

tests/test_send_recv_dialog_gui.py::TestSendRecvDialog::test_spectrum
  /tmp/tests/test_send_recv_dialog_gui.py:201: DeprecationWarning: tostring() is deprecated. Use tobytes() instead.
    sock.sendall(data.tostring())

tests/test_send_recv_dialog_gui.py::TestSendRecvDialog::test_spectrum
tests/test_spectrogram.py::TestSpectrogram::test_cancel_filtering
tests/test_spectrogram.py::TestSpectrogram::test_create_spectrogram_image
  /usr/lib64/python3.13/site-packages/urh/signalprocessing/Spectrogram.py:206: RuntimeWarning: invalid value encountered in cast
    return np.take(colormap, normalized_values.astype(int), axis=0, mode="clip")

tests/test_simulator.py::TestSimulator::test_external_program_simulator
  /usr/lib/python3.13/site-packages/_pytest/threadexception.py:82: PytestUnhandledThreadExceptionWarning: Exception in thread Thread-23 (simulate)
  
  Traceback (most recent call last):
    File "/usr/lib64/python3.13/threading.py", line 1041, in _bootstrap_inner
      self.run()
      ~~~~~~~~^^
    File "/usr/lib64/python3.13/threading.py", line 992, in run
      self._target(*self._args, **self._kwargs)
      ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/usr/lib64/python3.13/site-packages/urh/simulator/Simulator.py", line 327, in simulate
      self.stop(msg="Finished")
      ~~~~~~~~~^^^^^^^^^^^^^^^^
    File "/usr/lib64/python3.13/site-packages/urh/simulator/Simulator.py", line 133, in stop
      self.simulation_thread.join(2.5)
      ~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^
    File "/usr/lib64/python3.13/threading.py", line 1085, in join
      raise RuntimeError("cannot join current thread")
  RuntimeError: cannot join current thread
  
    warnings.warn(pytest.PytestUnhandledThreadExceptionWarning(msg))

tests/test_simulator.py::TestSimulator::test_simulation_flow
tests/test_simulator_tab_gui.py::TestSimulatorTabGUI::test_save_and_load_standalone
  /usr/lib64/python3.13/site-packages/urh/simulator/SimulatorConfiguration.py:265: DeprecationWarning: Testing an element's truth value will always return True in future versions.  Use specific 'len(elem)' or 'elem is not None' test instead.
    if modulators_tag:

tests/test_simulator.py::TestSimulator::test_simulation_flow
tests/test_simulator_tab_gui.py::TestSimulatorTabGUI::test_save_and_load_standalone
  /usr/lib64/python3.13/site-packages/urh/simulator/SimulatorConfiguration.py:271: DeprecationWarning: Testing an element's truth value will always return True in future versions.  Use specific 'len(elem)' or 'elem is not None' test instead.
    if participants_tag:

tests/test_simulator.py::TestSimulator::test_simulation_flow
tests/test_simulator_tab_gui.py::TestSimulatorTabGUI::test_save_and_load_standalone
  /usr/lib64/python3.13/site-packages/urh/simulator/SimulatorConfiguration.py:280: DeprecationWarning: Testing an element's truth value will always return True in future versions.  Use specific 'len(elem)' or 'elem is not None' test instead.
    if decodings_tag:

tests/test_simulator.py::TestSimulator::test_simulation_flow
tests/test_simulator_tab_gui.py::TestSimulatorTabGUI::test_save_and_load_standalone
  /usr/lib64/python3.13/site-packages/urh/simulator/SimulatorConfiguration.py:286: DeprecationWarning: Testing an element's truth value will always return True in future versions.  Use specific 'len(elem)' or 'elem is not None' test instead.
    if rx_config_tag:

tests/test_simulator.py::TestSimulator::test_simulation_flow
tests/test_simulator_tab_gui.py::TestSimulatorTabGUI::test_save_and_load_standalone
  /usr/lib64/python3.13/site-packages/urh/simulator/SimulatorConfiguration.py:292: DeprecationWarning: Testing an element's truth value will always return True in future versions.  Use specific 'len(elem)' or 'elem is not None' test instead.
    if tx_config_tag:

tests/test_simulator.py: 61 warnings
tests/test_simulator_tab_gui.py: 16 warnings
  /usr/lib64/python3.13/site-packages/urh/simulator/SimulatorExpressionParser.py:127: DeprecationWarning: ast.Num is deprecated and will be removed in Python 3.14; use ast.Constant instead
    if isinstance(node, ast.Num):

tests/test_simulator.py: 11 warnings
  /usr/lib64/python3.13/site-packages/urh/simulator/SimulatorExpressionParser.py:94: DeprecationWarning: ast.Str is deprecated and will be removed in Python 3.14; use ast.Constant instead
    elif isinstance(node, ast.Str):

tests/test_simulator.py::TestSimulator::test_simulation_flow
tests/test_simulator.py::TestSimulator::test_simulation_flow
tests/test_simulator.py::TestSimulator::test_simulation_flow
tests/test_simulator.py::TestSimulator::test_simulation_flow
tests/test_simulator.py::TestSimulator::test_simulation_flow
  /usr/lib64/python3.13/site-packages/urh/simulator/SimulatorExpressionParser.py:98: DeprecationWarning: ast.Num is deprecated and will be removed in Python 3.14; use ast.Constant instead
    elif isinstance(node, ast.Num):

tests/test_simulator.py::TestSimulator::test_simulation_flow
tests/test_simulator.py::TestSimulator::test_simulation_flow
tests/test_simulator.py::TestSimulator::test_simulation_flow
tests/test_simulator.py::TestSimulator::test_simulation_flow
tests/test_simulator.py::TestSimulator::test_simulation_flow
  /usr/lib64/python3.13/site-packages/urh/simulator/SimulatorExpressionParser.py:99: DeprecationWarning: Attribute n is deprecated and will be removed in Python 3.14; use value instead
    return node.n

tests/test_simulator.py::TestSimulator::test_simulation_flow
  /usr/lib/python3.13/site-packages/_pytest/threadexception.py:82: PytestUnhandledThreadExceptionWarning: Exception in thread Thread-28 (simulate)
  
  Traceback (most recent call last):
    File "/usr/lib64/python3.13/threading.py", line 1041, in _bootstrap_inner
      self.run()
      ~~~~~~~~^^
    File "/usr/lib64/python3.13/threading.py", line 992, in run
      self._target(*self._args, **self._kwargs)
      ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/usr/lib64/python3.13/site-packages/urh/simulator/Simulator.py", line 327, in simulate
      self.stop(msg="Finished")
      ~~~~~~~~~^^^^^^^^^^^^^^^^
    File "/usr/lib64/python3.13/site-packages/urh/simulator/Simulator.py", line 133, in stop
      self.simulation_thread.join(2.5)
      ~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^
    File "/usr/lib64/python3.13/threading.py", line 1085, in join
      raise RuntimeError("cannot join current thread")
  RuntimeError: cannot join current thread
  
    warnings.warn(pytest.PytestUnhandledThreadExceptionWarning(msg))

tests/test_simulator_tab_gui.py::TestSimulatorTabGUI::test_open_simulator_dialog_and_send_message
  /usr/lib/python3.13/site-packages/_pytest/threadexception.py:82: PytestUnhandledThreadExceptionWarning: Exception in thread Thread-35 (simulate)
  
  Traceback (most recent call last):
    File "/usr/lib64/python3.13/threading.py", line 1041, in _bootstrap_inner
      self.run()
      ~~~~~~~~^^
    File "/usr/lib64/python3.13/threading.py", line 992, in run
      self._target(*self._args, **self._kwargs)
      ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/usr/lib64/python3.13/site-packages/urh/simulator/Simulator.py", line 248, in simulate
      self.process_message()
      ~~~~~~~~~~~~~~~~~~~~^^
    File "/usr/lib64/python3.13/site-packages/urh/simulator/Simulator.py", line 383, in process_message
      self.log_message("  Received {} data bits".format(len(received_msg)))
                                                        ~~~^^^^^^^^^^^^^^
  TypeError: object of type 'NoneType' has no len()
  
    warnings.warn(pytest.PytestUnhandledThreadExceptionWarning(msg))

@yarda
Copy link
Contributor Author

yarda commented Jan 7, 2025

numpy-2 failures:

FAILED tests/awre/test_generated_protocols.py::TestGeneratedProtocols::test_without_preamble - AttributeError: 'NoneType' object has no attribute 'start'
FAILED tests/test_modulator.py::TestModulator::test_c_modulation_method_gfsk - ValueError: Buffer dtype mismatch, expected 'float32_t' but got 'double'
FAILED tests/test_modulator.py::TestModulator::test_gfsk - ValueError: Buffer dtype mismatch, expected 'float32_t' but got 'double'

numpy-2 tracebacks:

tests/test_modulator_gui.py::TestModulatorGUI::test_edit_modulation Traceback (most recent call last):
  File "/usr/lib64/python3.13/site-packages/urh/controller/dialogs/ModulatorDialog.py", line 564, in on_modulation_type_changed
    self.draw_modulated()
    ~~~~~~~~~~~~~~~~~~~^^
  File "/usr/lib64/python3.13/site-packages/urh/controller/dialogs/ModulatorDialog.py", line 224, in draw_modulated
    self.ui.gVModulated.plot_data(self.current_modulator.modulate(pause=0).imag)
                                  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/usr/lib64/python3.13/site-packages/urh/signalprocessing/Modulator.py", line 239, in modulate
    result = signal_functions.modulate_c(
        data,
    ...<12 lines>...
        self.gauss_filter_width,
    )
  File "src/urh/cythonext/signal_functions.pyx", line 56, in urh.cythonext.signal_functions.modulate_c
  File "src/urh/cythonext/signal_functions.pyx", line 73, in urh.cythonext.signal_functions.modulate_c
  File "src/urh/cythonext/signal_functions.pyx", line 116, in urh.cythonext.signal_functions.__modulate
  File "src/urh/cythonext/signal_functions.pyx", line 211, in urh.cythonext.signal_functions.get_gauss_filtered_freqs_phases
  File "src/urh/cythonext/signal_functions.pyx", line 241, in urh.cythonext.signal_functions.gauss_fir
ValueError: Buffer dtype mismatch, expected 'float32_t' but got 'double'
Fatal Python error: Aborted

Current thread 0x00007fb69d429740 (most recent call first):
  File "/tmp/tests/test_modulator_gui.py", line 119 in test_edit_modulation
  File "/usr/lib64/python3.13/unittest/case.py", line 606 in _callTestMethod
  File "/usr/lib64/python3.13/unittest/case.py", line 651 in run
  File "/usr/lib64/python3.13/unittest/case.py", line 707 in __call__
  File "/usr/lib/python3.13/site-packages/_pytest/unittest.py", line 351 in runtest
  File "/usr/lib/python3.13/site-packages/_pytest/runner.py", line 174 in pytest_runtest_call
  File "/usr/lib/python3.13/site-packages/pluggy/_callers.py", line 103 in _multicall
  File "/usr/lib/python3.13/site-packages/pluggy/_manager.py", line 120 in _hookexec
  File "/usr/lib/python3.13/site-packages/pluggy/_hooks.py", line 513 in __call__
  File "/usr/lib/python3.13/site-packages/_pytest/runner.py", line 242 in <lambda>
  File "/usr/lib/python3.13/site-packages/_pytest/runner.py", line 341 in from_call
  File "/usr/lib/python3.13/site-packages/_pytest/runner.py", line 241 in call_and_report
  File "/usr/lib/python3.13/site-packages/_pytest/runner.py", line 132 in runtestprotocol
  File "/usr/lib/python3.13/site-packages/_pytest/runner.py", line 113 in pytest_runtest_protocol
  File "/usr/lib/python3.13/site-packages/pluggy/_callers.py", line 103 in _multicall
  File "/usr/lib/python3.13/site-packages/pluggy/_manager.py", line 120 in _hookexec
  File "/usr/lib/python3.13/site-packages/pluggy/_hooks.py", line 513 in __call__
  File "/usr/lib/python3.13/site-packages/_pytest/main.py", line 362 in pytest_runtestloop
  File "/usr/lib/python3.13/site-packages/pluggy/_callers.py", line 103 in _multicall
  File "/usr/lib/python3.13/site-packages/pluggy/_manager.py", line 120 in _hookexec
  File "/usr/lib/python3.13/site-packages/pluggy/_hooks.py", line 513 in __call__
  File "/usr/lib/python3.13/site-packages/_pytest/main.py", line 337 in _main
  File "/usr/lib/python3.13/site-packages/_pytest/main.py", line 283 in wrap_session
  File "/usr/lib/python3.13/site-packages/_pytest/main.py", line 330 in pytest_cmdline_main
  File "/usr/lib/python3.13/site-packages/pluggy/_callers.py", line 103 in _multicall
  File "/usr/lib/python3.13/site-packages/pluggy/_manager.py", line 120 in _hookexec
  File "/usr/lib/python3.13/site-packages/pluggy/_hooks.py", line 513 in __call__
  File "/usr/lib/python3.13/site-packages/_pytest/config/__init__.py", line 175 in main
  File "/usr/lib/python3.13/site-packages/_pytest/config/__init__.py", line 201 in console_main
  File "/usr/bin/pytest", line 8 in <module>

numpy-2 warnings:

tests/auto_interpretation/test_auto_interpretation_integration.py: 1 warning
tests/auto_interpretation/test_center_detection.py: 1 warning
tests/auto_interpretation/test_message_segmentation.py: 1 warning
tests/auto_interpretation/test_noise_detection.py: 1 warning
tests/test_filter.py: 6 warnings
  /usr/lib64/python3.13/site-packages/urh/signalprocessing/Signal.py:210: DeprecationWarning: Python 3.14 will, by default, filter extracted tar archives and reject files or modify their metadata. Use the filter argument to control this behavior.
    obj.extract(members[0], QDir.tempPath())

tests/awre/test_address_engine.py: 2 warnings
tests/awre/test_awre_real_protocols.py: 2 warnings
tests/awre/test_generated_protocols.py: 9 warnings
tests/test_fuzzing_profile.py: 1 warning
tests/test_maincontroller_gui.py: 1 warning
tests/test_project_manager.py: 4 warnings
tests/test_protocol_label_dialog.py: 3 warnings
tests/test_signal_tab_GUI.py: 1 warning
tests/test_simulator_tab_gui.py: 1 warning
  /usr/lib64/python3.13/site-packages/urh/signalprocessing/ProtocolAnalyzer.py:733: DeprecationWarning: Testing an element's truth value will always return True in future versions.  Use specific 'len(elem)' or 'elem is not None' test instead.
    if not root:

tests/awre/test_address_engine.py: 5 warnings
tests/awre/test_awre_real_protocols.py: 8 warnings
tests/awre/test_generated_protocols.py: 15 warnings
tests/test_fuzzing_profile.py: 2 warnings
tests/test_maincontroller_gui.py: 2 warnings
tests/test_project_manager.py: 7 warnings
tests/test_protocol_label_dialog.py: 3 warnings
tests/test_signal_tab_GUI.py: 1 warning
tests/test_simulator.py: 6 warnings
tests/test_simulator_tab_gui.py: 8 warnings
  /usr/lib64/python3.13/site-packages/urh/signalprocessing/Ruleset.py:143: DeprecationWarning: Testing an element's truth value will always return True in future versions.  Use specific 'len(elem)' or 'elem is not None' test instead.
    if tag:

tests/awre/test_address_engine.py: 1220 warnings
tests/awre/test_awre_preprocessing.py: 2057 warnings
tests/awre/test_awre_real_protocols.py: 340 warnings
tests/awre/test_checksum_engine.py: 40 warnings
tests/awre/test_generated_protocols.py: 1463 warnings
tests/awre/test_length_engine.py: 673 warnings
tests/awre/test_partially_labeled.py: 410 warnings
tests/awre/test_sequence_number_engine.py: 139 warnings
tests/test_analysis_tab_GUI.py: 712 warnings
  /usr/lib64/python3.13/site-packages/urh/awre/Histogram.py:102: DeprecationWarning: tostring() is deprecated. Use tobytes() instead.
    values[vector[start : start + length].tostring()].append(i)

tests/awre/test_address_engine.py: 46 warnings
tests/awre/test_awre_real_protocols.py: 22 warnings
tests/awre/test_generated_protocols.py: 49 warnings
  /usr/lib64/python3.13/site-packages/urh/awre/engines/AddressEngine.py:534: DeprecationWarning: tostring() is deprecated. Use tobytes() instead.
    result[p1].add(val.tostring())

tests/awre/test_address_engine.py: 46 warnings
tests/awre/test_awre_real_protocols.py: 22 warnings
tests/awre/test_generated_protocols.py: 49 warnings
  /usr/lib64/python3.13/site-packages/urh/awre/engines/AddressEngine.py:535: DeprecationWarning: tostring() is deprecated. Use tobytes() instead.
    result[p2].add(val.tostring())

tests/awre/test_address_engine.py: 530 warnings
tests/awre/test_awre_real_protocols.py: 55 warnings
tests/awre/test_generated_protocols.py: 197 warnings
  /usr/lib64/python3.13/site-packages/urh/awre/engines/AddressEngine.py:326: DeprecationWarning: tostring() is deprecated. Use tobytes() instead.
    if i in rng.message_indices and rng.value.tostring() in addresses

tests/awre/test_address_engine.py: 410 warnings
tests/awre/test_awre_real_protocols.py: 44 warnings
tests/awre/test_generated_protocols.py: 55 warnings
  /usr/lib64/python3.13/site-packages/urh/awre/engines/AddressEngine.py:357: DeprecationWarning: tostring() is deprecated. Use tobytes() instead.
    rng.value.tostring()

tests/awre/test_address_engine.py: 33 warnings
tests/awre/test_awre_real_protocols.py: 11 warnings
tests/awre/test_generated_protocols.py: 35 warnings
tests/awre/test_partially_labeled.py: 18 warnings
  /usr/lib64/python3.13/site-packages/urh/awre/engines/AddressEngine.py:211: DeprecationWarning: tostring() is deprecated. Use tobytes() instead.
    if rng.value.tostring() == address

tests/awre/test_address_engine.py: 120 warnings
tests/awre/test_awre_real_protocols.py: 11 warnings
tests/awre/test_generated_protocols.py: 142 warnings
  /usr/lib64/python3.13/site-packages/urh/awre/engines/AddressEngine.py:330: DeprecationWarning: tostring() is deprecated. Use tobytes() instead.
    address = matching[0].value.tostring()

tests/awre/test_address_engine.py: 240 warnings
tests/awre/test_awre_real_protocols.py: 19 warnings
tests/awre/test_generated_protocols.py: 164 warnings
  /usr/lib64/python3.13/site-packages/urh/awre/engines/AddressEngine.py:343: DeprecationWarning: tostring() is deprecated. Use tobytes() instead.
    and rng.value.tostring() in addresses

tests/awre/test_address_engine.py: 240 warnings
tests/awre/test_awre_real_protocols.py: 18 warnings
tests/awre/test_generated_protocols.py: 55 warnings
  /usr/lib64/python3.13/site-packages/urh/awre/engines/AddressEngine.py:347: DeprecationWarning: tostring() is deprecated. Use tobytes() instead.
    lambda r: r.value.tostring() == address, prev_matching

tests/awre/test_address_engine.py: 8 warnings
tests/awre/test_awre_real_protocols.py: 6 warnings
tests/awre/test_generated_protocols.py: 30 warnings
tests/awre/test_partially_labeled.py: 12 warnings
  /usr/lib64/python3.13/site-packages/urh/awre/engines/AddressEngine.py:76: DeprecationWarning: tostring() is deprecated. Use tobytes() instead.
    p: [addr.tostring()]

tests/awre/test_address_engine.py: 8 warnings
tests/awre/test_awre_real_protocols.py: 6 warnings
tests/awre/test_generated_protocols.py: 30 warnings
tests/awre/test_partially_labeled.py: 12 warnings
  /usr/lib64/python3.13/site-packages/urh/awre/engines/AddressEngine.py:318: DeprecationWarning: tostring() is deprecated. Use tobytes() instead.
    address = self.known_addresses_by_participant[participant].tostring()

tests/awre/test_generated_protocols.py::TestGeneratedProtocols::test_with_checksum
  /tmp/tests/awre/test_generated_protocols.py:155: DeprecationWarning: tostring() is deprecated. Use tobytes() instead.
    known_participant_addresses[0].tostring(),

tests/awre/test_generated_protocols.py::TestGeneratedProtocols::test_with_checksum
  /tmp/tests/awre/test_generated_protocols.py:159: DeprecationWarning: tostring() is deprecated. Use tobytes() instead.
    known_participant_addresses[1].tostring(),

tests/awre/test_generated_protocols.py::TestGeneratedProtocols::test_with_four_broken
tests/awre/test_generated_protocols.py::TestGeneratedProtocols::test_with_one_address_one_message_type
tests/awre/test_generated_protocols.py::TestGeneratedProtocols::test_with_only_one_address
tests/awre/test_generated_protocols.py::TestGeneratedProtocols::test_without_preamble
tests/awre/test_generated_protocols.py::TestGeneratedProtocols::test_without_preamble_24_messages
tests/awre/test_generated_protocols.py::TestGeneratedProtocols::test_without_preamble_random_data
tests/awre/test_generated_protocols.py::TestGeneratedProtocols::test_without_preamble_random_data2
tests/test_analysis_tab_GUI.py::TestAnalysisTabGUI::test_analyze_button_enocean
tests/test_analysis_tab_GUI.py::TestAnalysisTabGUI::test_analyze_button_fsk
  /usr/lib64/python3.13/site-packages/urh/awre/CommonRange.py:61: RuntimeWarning: overflow encountered in scalar subtract
    self.__convert_number(self.start)

tests/awre/test_generated_protocols.py::TestGeneratedProtocols::test_with_one_address_one_message_type
  /tmp/tests/awre/test_generated_protocols.py:224: DeprecationWarning: tostring() is deprecated. Use tobytes() instead.
    known_participant_addresses[0].tostring(),

tests/awre/test_generated_protocols.py::TestGeneratedProtocols::test_with_one_address_one_message_type
  /tmp/tests/awre/test_generated_protocols.py:228: DeprecationWarning: tostring() is deprecated. Use tobytes() instead.
    known_participant_addresses[1].tostring(),

tests/awre/test_generated_protocols.py::TestGeneratedProtocols::test_with_only_one_address
  /tmp/tests/awre/test_generated_protocols.py:175: DeprecationWarning: tostring() is deprecated. Use tobytes() instead.
    known_participant_addresses[0].tostring(),

tests/awre/test_generated_protocols.py::TestGeneratedProtocols::test_with_only_one_address
  /tmp/tests/awre/test_generated_protocols.py:179: DeprecationWarning: tostring() is deprecated. Use tobytes() instead.
    known_participant_addresses[1].tostring(),

tests/awre/test_generated_protocols.py::TestGeneratedProtocols::test_without_preamble
tests/awre/test_sequence_number_engine.py::TestSequenceNumberEngine::test_16bit_seq_nr
tests/awre/test_sequence_number_engine.py::TestSequenceNumberEngine::test_sequence_number_little_endian_16_bit
  /usr/lib64/python3.13/site-packages/urh/awre/FormatFinder.py:495: RuntimeWarning: overflow encountered in scalar negative
    -sum(r.length_in_bits for r in sol),

tests/awre/test_generated_protocols.py::TestGeneratedProtocols::test_without_preamble_24_messages
  /tmp/tests/awre/test_generated_protocols.py:247: DeprecationWarning: tostring() is deprecated. Use tobytes() instead.
    known_participant_addresses[0].tostring(),

tests/awre/test_generated_protocols.py::TestGeneratedProtocols::test_without_preamble_24_messages
  /tmp/tests/awre/test_generated_protocols.py:251: DeprecationWarning: tostring() is deprecated. Use tobytes() instead.
    known_participant_addresses[1].tostring(),

tests/test_file_operator.py::TestFileOperator::test_uncompress_archives
tests/test_file_operator.py::TestFileOperator::test_uncompress_archives
tests/test_file_operator.py::TestFileOperator::test_uncompress_archives
  /usr/lib64/python3.13/site-packages/urh/util/FileOperator.py:262: DeprecationWarning: Python 3.14 will, by default, filter extracted tar archives and reject files or modify their metadata. Use the filter argument to control this behavior.
    obj.extract(member, temp_dir)

tests/test_fuzzing_profile.py: 2 warnings
tests/test_simulator.py: 6 warnings
tests/test_simulator_tab_gui.py: 6 warnings
  /usr/lib64/python3.13/site-packages/urh/signalprocessing/Message.py:633: DeprecationWarning: Testing an element's truth value will always return True in future versions.  Use specific 'len(elem)' or 'elem is not None' test instead.
    if message_type_tag:

tests/test_project_manager.py::TestProjectManager::test_save_and_load_participants
tests/test_project_manager.py::TestProjectManager::test_save_and_load_participants
tests/test_project_manager.py::TestProjectManager::test_save_and_load_participants
tests/test_project_manager.py::TestProjectManager::test_save_and_load_with_fieldtypes
tests/test_simulator_tab_gui.py::TestSimulatorTabGUI::test_save_and_load
  /usr/lib64/python3.13/site-packages/urh/util/ProjectManager.py:512: DeprecationWarning: Testing an element's truth value will always return True in future versions.  Use specific 'len(elem)' or 'elem is not None' test instead.
    if messages_tag:

tests/test_protocol_sniffer.py: 2 warnings
tests/test_send_recv_dialog_gui.py: 1 warning
tests/test_simulator.py: 8 warnings
tests/test_simulator_tab_gui.py: 2 warnings
  /usr/lib64/python3.13/site-packages/urh/signalprocessing/IQArray.py:96: DeprecationWarning: tostring() is deprecated. Use tobytes() instead.
    return self.__data.tostring()

tests/test_send_recv_dialog_gui.py::TestSendRecvDialog::test_receive
  /tmp/tests/test_send_recv_dialog_gui.py:166: DeprecationWarning: tostring() is deprecated. Use tobytes() instead.
    sock.sendall(data.tostring())

tests/test_send_recv_dialog_gui.py::TestSendRecvDialog::test_spectrum
  /tmp/tests/test_send_recv_dialog_gui.py:201: DeprecationWarning: tostring() is deprecated. Use tobytes() instead.
    sock.sendall(data.tostring())

tests/test_send_recv_dialog_gui.py::TestSendRecvDialog::test_spectrum
tests/test_spectrogram.py::TestSpectrogram::test_cancel_filtering
tests/test_spectrogram.py::TestSpectrogram::test_create_spectrogram_image
  /usr/lib64/python3.13/site-packages/urh/signalprocessing/Spectrogram.py:206: RuntimeWarning: invalid value encountered in cast
    return np.take(colormap, normalized_values.astype(int), axis=0, mode="clip")

tests/test_simulator.py::TestSimulator::test_external_program_simulator
  /usr/lib/python3.13/site-packages/_pytest/threadexception.py:82: PytestUnhandledThreadExceptionWarning: Exception in thread Thread-23 (simulate)
  
  Traceback (most recent call last):
    File "/usr/lib64/python3.13/threading.py", line 1041, in _bootstrap_inner
      self.run()
      ~~~~~~~~^^
    File "/usr/lib64/python3.13/threading.py", line 992, in run
      self._target(*self._args, **self._kwargs)
      ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/usr/lib64/python3.13/site-packages/urh/simulator/Simulator.py", line 327, in simulate
      self.stop(msg="Finished")
      ~~~~~~~~~^^^^^^^^^^^^^^^^
    File "/usr/lib64/python3.13/site-packages/urh/simulator/Simulator.py", line 133, in stop
      self.simulation_thread.join(2.5)
      ~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^
    File "/usr/lib64/python3.13/threading.py", line 1085, in join
      raise RuntimeError("cannot join current thread")
  RuntimeError: cannot join current thread
  
    warnings.warn(pytest.PytestUnhandledThreadExceptionWarning(msg))

tests/test_simulator.py::TestSimulator::test_simulation_flow
tests/test_simulator_tab_gui.py::TestSimulatorTabGUI::test_save_and_load_standalone
  /usr/lib64/python3.13/site-packages/urh/simulator/SimulatorConfiguration.py:265: DeprecationWarning: Testing an element's truth value will always return True in future versions.  Use specific 'len(elem)' or 'elem is not None' test instead.
    if modulators_tag:

tests/test_simulator.py::TestSimulator::test_simulation_flow
tests/test_simulator_tab_gui.py::TestSimulatorTabGUI::test_save_and_load_standalone
  /usr/lib64/python3.13/site-packages/urh/simulator/SimulatorConfiguration.py:271: DeprecationWarning: Testing an element's truth value will always return True in future versions.  Use specific 'len(elem)' or 'elem is not None' test instead.
    if participants_tag:

tests/test_simulator.py::TestSimulator::test_simulation_flow
tests/test_simulator_tab_gui.py::TestSimulatorTabGUI::test_save_and_load_standalone
  /usr/lib64/python3.13/site-packages/urh/simulator/SimulatorConfiguration.py:280: DeprecationWarning: Testing an element's truth value will always return True in future versions.  Use specific 'len(elem)' or 'elem is not None' test instead.
    if decodings_tag:

tests/test_simulator.py::TestSimulator::test_simulation_flow
tests/test_simulator_tab_gui.py::TestSimulatorTabGUI::test_save_and_load_standalone
  /usr/lib64/python3.13/site-packages/urh/simulator/SimulatorConfiguration.py:286: DeprecationWarning: Testing an element's truth value will always return True in future versions.  Use specific 'len(elem)' or 'elem is not None' test instead.
    if rx_config_tag:

tests/test_simulator.py::TestSimulator::test_simulation_flow
tests/test_simulator_tab_gui.py::TestSimulatorTabGUI::test_save_and_load_standalone
  /usr/lib64/python3.13/site-packages/urh/simulator/SimulatorConfiguration.py:292: DeprecationWarning: Testing an element's truth value will always return True in future versions.  Use specific 'len(elem)' or 'elem is not None' test instead.
    if tx_config_tag:

tests/test_simulator.py: 61 warnings
tests/test_simulator_tab_gui.py: 16 warnings
  /usr/lib64/python3.13/site-packages/urh/simulator/SimulatorExpressionParser.py:127: DeprecationWarning: ast.Num is deprecated and will be removed in Python 3.14; use ast.Constant instead
    if isinstance(node, ast.Num):

tests/test_simulator.py: 11 warnings
  /usr/lib64/python3.13/site-packages/urh/simulator/SimulatorExpressionParser.py:94: DeprecationWarning: ast.Str is deprecated and will be removed in Python 3.14; use ast.Constant instead
    elif isinstance(node, ast.Str):

tests/test_simulator.py::TestSimulator::test_simulation_flow
tests/test_simulator.py::TestSimulator::test_simulation_flow
tests/test_simulator.py::TestSimulator::test_simulation_flow
tests/test_simulator.py::TestSimulator::test_simulation_flow
tests/test_simulator.py::TestSimulator::test_simulation_flow
  /usr/lib64/python3.13/site-packages/urh/simulator/SimulatorExpressionParser.py:98: DeprecationWarning: ast.Num is deprecated and will be removed in Python 3.14; use ast.Constant instead
    elif isinstance(node, ast.Num):

tests/test_simulator.py::TestSimulator::test_simulation_flow
tests/test_simulator.py::TestSimulator::test_simulation_flow
tests/test_simulator.py::TestSimulator::test_simulation_flow
tests/test_simulator.py::TestSimulator::test_simulation_flow
tests/test_simulator.py::TestSimulator::test_simulation_flow
  /usr/lib64/python3.13/site-packages/urh/simulator/SimulatorExpressionParser.py:99: DeprecationWarning: Attribute n is deprecated and will be removed in Python 3.14; use value instead
    return node.n

tests/test_simulator.py::TestSimulator::test_simulation_flow
  /usr/lib/python3.13/site-packages/_pytest/threadexception.py:82: PytestUnhandledThreadExceptionWarning: Exception in thread Thread-28 (simulate)
  
  Traceback (most recent call last):
    File "/usr/lib64/python3.13/threading.py", line 1041, in _bootstrap_inner
      self.run()
      ~~~~~~~~^^
    File "/usr/lib64/python3.13/threading.py", line 992, in run
      self._target(*self._args, **self._kwargs)
      ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/usr/lib64/python3.13/site-packages/urh/simulator/Simulator.py", line 327, in simulate
      self.stop(msg="Finished")
      ~~~~~~~~~^^^^^^^^^^^^^^^^
    File "/usr/lib64/python3.13/site-packages/urh/simulator/Simulator.py", line 133, in stop
      self.simulation_thread.join(2.5)
      ~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^
    File "/usr/lib64/python3.13/threading.py", line 1085, in join
      raise RuntimeError("cannot join current thread")
  RuntimeError: cannot join current thread
  
    warnings.warn(pytest.PytestUnhandledThreadExceptionWarning(msg))

tests/test_simulator_tab_gui.py::TestSimulatorTabGUI::test_open_simulator_dialog_and_send_message
  /usr/lib/python3.13/site-packages/_pytest/threadexception.py:82: PytestUnhandledThreadExceptionWarning: Exception in thread Thread-35 (simulate)
  
  Traceback (most recent call last):
    File "/usr/lib64/python3.13/threading.py", line 1041, in _bootstrap_inner
      self.run()
      ~~~~~~~~^^
    File "/usr/lib64/python3.13/threading.py", line 992, in run
      self._target(*self._args, **self._kwargs)
      ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/usr/lib64/python3.13/site-packages/urh/simulator/Simulator.py", line 248, in simulate
      self.process_message()
      ~~~~~~~~~~~~~~~~~~~~^^
    File "/usr/lib64/python3.13/site-packages/urh/simulator/Simulator.py", line 383, in process_message
      self.log_message("  Received {} data bits".format(len(received_msg)))
                                                        ~~~^^^^^^^^^^^^^^
  TypeError: object of type 'NoneType' has no len()
  
    warnings.warn(pytest.PytestUnhandledThreadExceptionWarning(msg))

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html

@yarda
Copy link
Contributor Author

yarda commented Jan 7, 2025

So it seems most problems are float vs double.

@yarda
Copy link
Contributor Author

yarda commented Jan 7, 2025

Details of failures:

========================================================================================== FAILURES ==========================================================================================
________________________________________________________________________ TestGeneratedProtocols.test_without_preamble ________________________________________________________________________

self = <tests.awre.test_generated_protocols.TestGeneratedProtocols testMethod=test_without_preamble>

    def test_without_preamble(self):
        alice = Participant("Alice", address_hex="24")
        broadcast = Participant("Broadcast", address_hex="ff")
    
        mb = MessageTypeBuilder("data")
        mb.add_label(FieldType.Function.SYNC, 16)
        mb.add_label(FieldType.Function.LENGTH, 8)
        mb.add_label(FieldType.Function.SRC_ADDRESS, 8)
        mb.add_label(FieldType.Function.SEQUENCE_NUMBER, 8)
    
        pg = ProtocolGenerator(
            [mb.message_type],
            syncs_by_mt={mb.message_type: "0x8e88"},
            preambles_by_mt={mb.message_type: "10" * 8},
            participants=[alice, broadcast],
        )
    
        for i in range(20):
            data_bits = 16 if i % 2 == 0 else 32
            source = pg.participants[i % 2]
            destination = pg.participants[(i + 1) % 2]
            pg.generate_message(
                data="1010" * (data_bits // 4), source=source, destination=destination
            )
    
        # self.save_protocol("without_preamble", pg)
        self.clear_message_types(pg.messages)
        ff = FormatFinder(pg.messages)
        ff.known_participant_addresses.clear()
    
        ff.run()
        self.assertEqual(len(ff.message_types), 1)
    
        mt = ff.message_types[0]
        sync = mt.get_first_label_with_type(FieldType.Function.SYNC)
        self.assertEqual(sync.start, 0)
        self.assertEqual(sync.length, 16)
    
        length = mt.get_first_label_with_type(FieldType.Function.LENGTH)
>       self.assertEqual(length.start, 16)
E       AttributeError: 'NoneType' object has no attribute 'start'

tests/awre/test_generated_protocols.py:85: AttributeError
________________________________________________________________________ TestModulator.test_c_modulation_method_gfsk _________________________________________________________________________

self = <tests.test_modulator.TestModulator testMethod=test_c_modulation_method_gfsk>

    def test_c_modulation_method_gfsk(self):
        bits = array.array("B", [1, 0, 1, 0, 1, 1, 0, 0, 0, 1])
        parameters = array.array("f", [-10e3, 10e3])
>       result = modulate_c(bits, 100, "GFSK", parameters, 1, 1, 40e3, 0, 1e6, 1000, 0)

tests/test_modulator.py:157: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
src/urh/cythonext/signal_functions.pyx:56: in urh.cythonext.signal_functions.modulate_c
    ???
src/urh/cythonext/signal_functions.pyx:73: in urh.cythonext.signal_functions.modulate_c
    ???
src/urh/cythonext/signal_functions.pyx:116: in urh.cythonext.signal_functions.__modulate
    ???
src/urh/cythonext/signal_functions.pyx:211: in urh.cythonext.signal_functions.get_gauss_filtered_freqs_phases
    ???
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

>   ???
E   ValueError: Buffer dtype mismatch, expected 'float32_t' but got 'double'

src/urh/cythonext/signal_functions.pyx:241: ValueError
__________________________________________________________________________________ TestModulator.test_gfsk ___________________________________________________________________________________

self = <tests.test_modulator.TestModulator testMethod=test_gfsk>

    def test_gfsk(self):
        target_file = os.path.join(tempfile.gettempdir(), "test.complex")
    
        modulator = Modulator("gfsk")
        modulator.modulation_type = "GFSK"
        modulator.samples_per_symbol = 100
        modulator.sample_rate = 1e6
        modulator.parameters[1] = 20e3
        modulator.parameters[0] = -10e3
>       data1 = modulator.modulate([True, False, False, True, False], 9437)

tests/test_modulator.py:77: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/usr/lib64/python3.13/site-packages/urh/signalprocessing/Modulator.py:239: in modulate
    result = signal_functions.modulate_c(
src/urh/cythonext/signal_functions.pyx:56: in urh.cythonext.signal_functions.modulate_c
    ???
src/urh/cythonext/signal_functions.pyx:73: in urh.cythonext.signal_functions.modulate_c
    ???
src/urh/cythonext/signal_functions.pyx:116: in urh.cythonext.signal_functions.__modulate
    ???
src/urh/cythonext/signal_functions.pyx:211: in urh.cythonext.signal_functions.get_gauss_filtered_freqs_phases
    ???
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

>   ???
E   ValueError: Buffer dtype mismatch, expected 'float32_t' but got 'double'

src/urh/cythonext/signal_functions.pyx:241: ValueError

@yarda
Copy link
Contributor Author

yarda commented Jan 7, 2025

Except of the AttributeError: 'NoneType' object has no attribute 'start' which I currently have no idea where the problem is, the rest is probably caused by the gauss_fir in src/urh/cythonext/signal_functions.pyx:241. I guess it's related to the numpy-2 change in data type promotion, which is now according to the NEP 50, i.e. the float_32 was promoted to float_64 which maps to the C type double. Probably conversion function .astype(np.float32) on the right place should fix it?

@yarda
Copy link
Contributor Author

yarda commented Jan 7, 2025

With the following patch:

--- a/src/urh/cythonext/signal_functions.pyx
+++ b/src/urh/cythonext/signal_functions.pyx
@@ -238,9 +238,10 @@ cdef np.ndarray[np.float32_t, ndim=1] gauss_fir(float sample_rate, uint32_t samp
                                                 int(filter_width * samples_per_symbol) + 1,
                                                 dtype=np.float32)
     cdef float ts = samples_per_symbol / sample_rate  # symbol time
-    cdef np.ndarray[np.float32_t] h = np.sqrt((2 * np.pi) / (np.log(2))) * bt / ts * np.exp(
+    cdef np.ndarray[np.double_t] h = np.sqrt((2 * np.pi) / (np.log(2))) * bt / ts * np.exp(
         -(((np.sqrt(2) * np.pi) / np.sqrt(np.log(2)) * bt * k / samples_per_symbol) ** 2))
-    return h / h.sum()
+    cdef np.ndarray[np.float32_t] hh = h.astype(np.float32)
+    return hh / hh.sum()
 
 cdef float clamp(float x) nogil:
     if x < -1.0:

tests results: 1 failed, 341 passed

The patch is far from being optimal any hints for optimization are welcome.

Regarding the remaining 1 failure, it's in:

self = <tests.awre.test_generated_protocols.TestGeneratedProtocols testMethod=test_without_preamble>

    def test_without_preamble(self):
        alice = Participant("Alice", address_hex="24")
        broadcast = Participant("Broadcast", address_hex="ff")
    
        mb = MessageTypeBuilder("data")
        mb.add_label(FieldType.Function.SYNC, 16)
        mb.add_label(FieldType.Function.LENGTH, 8)
        mb.add_label(FieldType.Function.SRC_ADDRESS, 8)
        mb.add_label(FieldType.Function.SEQUENCE_NUMBER, 8)
    
        pg = ProtocolGenerator(
            [mb.message_type],
            syncs_by_mt={mb.message_type: "0x8e88"},
            preambles_by_mt={mb.message_type: "10" * 8},
            participants=[alice, broadcast],
        )
    
        for i in range(20):
            data_bits = 16 if i % 2 == 0 else 32
            source = pg.participants[i % 2]
            destination = pg.participants[(i + 1) % 2]
            pg.generate_message(
                data="1010" * (data_bits // 4), source=source, destination=destination
            )
    
        # self.save_protocol("without_preamble", pg)
        self.clear_message_types(pg.messages)
        ff = FormatFinder(pg.messages)
        ff.known_participant_addresses.clear()
    
        ff.run()
        self.assertEqual(len(ff.message_types), 1)
    
        mt = ff.message_types[0]
        sync = mt.get_first_label_with_type(FieldType.Function.SYNC)
        self.assertEqual(sync.start, 0)
        self.assertEqual(sync.length, 16)
    
        length = mt.get_first_label_with_type(FieldType.Function.LENGTH)
>       self.assertEqual(length.start, 16)
E       AttributeError: 'NoneType' object has no attribute 'start'

tests/awre/test_generated_protocols.py:85: AttributeError

and here I have no idea what's wrong, any hints?

If we come up with complete solution, I am OK to create PR.

@yarda
Copy link
Contributor Author

yarda commented Jan 7, 2025

Cleaner patch:

--- a/src/urh/cythonext/signal_functions.pyx
+++ b/src/urh/cythonext/signal_functions.pyx
@@ -238,9 +238,9 @@ cdef np.ndarray[np.float32_t, ndim=1] gauss_fir(float sample_rate, uint32_t samp
                                                 int(filter_width * samples_per_symbol) + 1,
                                                 dtype=np.float32)
     cdef float ts = samples_per_symbol / sample_rate  # symbol time
-    cdef np.ndarray[np.float32_t] h = np.sqrt((2 * np.pi) / (np.log(2))) * bt / ts * np.exp(
+    cdef np.ndarray[np.double_t] h = np.sqrt((2 * np.pi) / (np.log(2))) * bt / ts * np.exp(
         -(((np.sqrt(2) * np.pi) / np.sqrt(np.log(2)) * bt * k / samples_per_symbol) ** 2))
-    return h / h.sum()
+    return (h / h.sum()).astype(np.float32)

@yarda
Copy link
Contributor Author

yarda commented Jan 7, 2025

The proposed patch requires check for numpy version, because it will fail with numpy-1, e.g. something like:

    if np.lib.NumpyVersion(np.__version__) >= '2.0.0b1':
        ...
    else:
        ...

But I am not sure how to write it in cython.

@andynoack
Copy link
Collaborator

Thank you very much for you work so far! Do you like to open a pull request? I will add Johannes to this issue for reviewing.

@yarda
Copy link
Contributor Author

yarda commented Jan 8, 2025

Yes, I will open PR when I found a solution. There are still two things I need to resolve:

  1. add numpy version conditional into signal_functions.pyx for the patch to work with numpy-1 and 2. Unfortunately, I am not good at cython, so all my attempts how to conditionalize it (nicely) so far failed.
  2. I have no idea what's wrong with the test_generated_protocols.py, the traceback is not very useful and I am now digging into the FieldType.Function.LENGTH code, but my progress is slow.

@yarda
Copy link
Contributor Author

yarda commented Jan 9, 2025

I probably resolved the 1., PR will follow. Still fighting with the 2.

yarda added a commit to yarda/urh that referenced this issue Jan 9, 2025
@yarda
Fixed gauss_fir to work with numpy-2
7e2d831 
It's related to jopohl#1149

Signed-off-by: Jaroslav Škarvada <jskarvad@redhat.com>
@yarda yarda mentioned this issue Jan 9, 2025
Merged
@yarda
Copy link
Contributor Author

yarda commented Jan 9, 2025

The 2. seems to be some problem in the FormatFinder.run().

@yarda
Copy link
Contributor Author

yarda commented Jan 9, 2025 •
edited
Loading

The problem seems to be in the SequenceNumberEngine.

@chenrui333 chenrui333 mentioned this issue Jan 12, 2025
Merged
6 tasks
yarda added a commit to yarda/urh that referenced this issue Jan 14, 2025
@yarda
Fixed gauss_fir to work with numpy-2
d82004c 
It's related to jopohl#1149

Signed-off-by: Jaroslav Škarvada <jskarvad@redhat.com>
andynoack pushed a commit that referenced this issue Jan 14, 2025
@yarda
Fixed gauss_fir to work with numpy-2 (#1150)
0f9365b 
It's related to #1149

Signed-off-by: Jaroslav Škarvada <jskarvad@redhat.com>
@andynoack
Copy link
Collaborator

Can we mark this issue as resolved?

@yarda
Copy link
Contributor Author

yarda commented Jan 16, 2025

Can we mark this issue as resolved?

Unfortunately, it isn't resolved yet. There is still numpy-2 related failure of the test_generated_protocols.py. I am still on it, but there is no traceback that could help and I am not familiar with the code and with the math used. But I am slowly progressing. There was also a bit delay, because I focused on the wrong thing - it uses sets and then iterates over them and it seems on Fedora 42 the sets are internally sorted differently than on Fedora 41 which gives different bit dumps. But it turned out this is not the core of the problem, the core problem is some change in numpy-2 which I hasn't been able to identify yet. I also had to work on different more urgent problem, but I think I will be able to resolve it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jopohl jopohl

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@yarda @jopohl @andynoack