Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

C# Async with .NET Core Kaitai.Struct.Runtime.Async #17

Open
wants to merge 46 commits into
base: master
Choose a base branch
from

Conversation

pluskal
Copy link
Contributor

@pluskal pluskal commented Jan 3, 2020

Relates to kaitai-io/kaitai_struct#640 and kaitai-io/kaitai_struct_compiler#189

This PR adds an additional unit tests that ensures correct behavior of the runtime itself that is not dependent on the compiled KSY files.

An abstraction formed by IKaitaiStreamBase, IKaitaiAsyncStream is introduced to decouple code and implementation to allow reimplementation of the Stream if needed. (I was unable to update the compiler to accept interface instead of direct type due to my lack of Scala knowledge and coupling in the compiler and translator code.)

The ultimate goal is to introduce Asynchronous implementation that allows usage in massively parallel applications that consumes data in async streams. To achieve that, IReaderContext is introduced to abstract multiple implementations of the "Stream".

  • Implementation of standard stream is provided by the StreamReaderContext that suffers from a lack of asynchronicity due to the .NET's Stream API that does not support asynchronous seek operation.
  • The true asynchronous implementation is provided by PipeReaderContext.

The folder structure, project naming is changed to reflect C#/.NET conventions.

@pluskal pluskal force-pushed the Async branch 2 times, most recently from 347a849 to f0dca07 Compare January 6, 2020 10:30
pluskal and others added 9 commits January 8, 2020 09:07
Add Async support to Runtime

Closes #1

See merge request marta/kaitai_struct_csharp_runtime!3
Pipe reader

See merge request marta/kaitai_struct_csharp_runtime!7
…ad' into 'master'

Resolve "Ensure that ReadResult is not default before first read."

Closes kaitai-io#4

See merge request marta/kaitai_struct_csharp_runtime!9
…ls' into 'master'

Resolve "Position should remain the same on Size, Pos, IsEof calls"

Closes kaitai-io#5

See merge request marta/kaitai_struct_csharp_runtime!11
@pluskal pluskal changed the title WIP: C# Async with .NET Core Kaitai.Struct.Runtime.Async C# Async with .NET Core Kaitai.Struct.Runtime.Async Feb 19, 2020
@pluskal pluskal requested a review from GreyCat February 19, 2020 14:47
pluskal added 6 commits June 12, 2020 12:45
Resolve "`Nullable<T>` for optional fields are call"

Closes kaitai-io#6

See merge request marta/kaitai_struct_csharp_runtime!14
…ster'

Resolve "ImplicitNullable fails during serialization."

Closes kaitai-io#7

See merge request marta/kaitai_struct_csharp_runtime!16
pluskal and others added 18 commits July 17, 2020 08:47
…lemement-advance' into 'master'

Resolve "KaitaiStreamAsync encapsulating pipereader should implemement Advance"

Closes kaitai-io#8

See merge request marta/kaitai_struct_csharp_runtime!18
Resolve "Add CancellationToken support."

Closes kaitai-io#9

See merge request marta/kaitai_struct_csharp_runtime!19
…-ends-with-exception' into 'master'

Resolve "Forward PipeReader.SeekAsync on completed ReadResult ends with exception"

Closes kaitai-io#10

See merge request marta/kaitai_struct_csharp_runtime!20
…o-examined-the-whole-buffer' into 'master'

Resolve "Fix PipeReaderContext.ReadByteAsync should advance to examined the whole buffer"

Closes kaitai-io#11

See merge request marta/kaitai_struct_csharp_runtime!21
… deprecation warning message - generalmimon committed on Apr 26, 2020
Rebase on 0.9

See merge request marta/kaitai_struct_csharp_runtime!22
…am-ikaitaistreambase' into 'master'

Resolve "Update Exceptions to use interfaces IKaitaiAsyncStream, IKaitaiStreamBase"

Closes kaitai-io#12

See merge request marta/kaitai_struct_csharp_runtime!23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants