Help understanding the "right" way to handle IO, exceptions and nullability

[drk-mtr]

I'm having a play with LanguageExt.Core 5.0.0-alpha.6. As a relative newcomer to C# I'm struggling to determine the best practice for IO when using this brilliant library.

My code below is horrible, but hopefully it's enough to illustrate what I'm scratching my head over.

I think I understand the basics when handling a Task that may throw, but I'm unclear on how best to handle things when an async method may throw, but may also return null.

My inclination would be to return a type of IO<Option<MyType>> but then I'm faced with a new problem, because I think I understand how to chain IO<MyType> or Option<MyType> but I don't know how to chain IO<Option<MyType>> . I'm not even sure whether IO is the correct wrapper to be using tbh.

Here's the code without using LanguageExt. It makes a call to a police data api, deserializes the response, then logs it to the console:

using System.Text.Json;

namespace LanguageExtTrial;

public readonly record struct Force(string Id, string Name);

public static class PoliceData
{
    private static async Task<Stream> GetForcesStream()
    {
        using var client = new HttpClient();

        // Could throw, could return null.
        var response = await client.GetAsync("https://data.police.uk/api/forces");

        return await response.Content.ReadAsStreamAsync();
    }

    private static async Task<Force[]> DeserializeForces(Stream forces)
    {
        // Could throw, could return null.
        return await JsonSerializer.DeserializeAsync<Force[]>(
            forces, 
            new JsonSerializerOptions(JsonSerializerDefaults.Web));
    }

    public static async Task<Force[]> GetForces()
    {
        var stream = await GetForcesStream();

        return await DeserializeForces(stream);
    }
}

public class Program
{
    static async Task Main()
    {
        Console.WriteLine(JsonSerializer.Serialize(await PoliceData.GetForces()));
    }
}

And with LanguageExt:

using System.Text.Json;
using LanguageExt;
using LanguageExt.Common;
using static LanguageExt.Prelude;

namespace LanguageExtTrial;

public readonly record struct Force(string Id, string Name);

public static class PoliceData
{
    private static IO<Stream> GetForcesStream() =>
        liftIO(async () => {
            using var client = new HttpClient();

            var response = await client.GetAsync("https://data.police.uk/api/forces");

            return await response.Content.ReadAsStreamAsync();
        });

    private static IO<Force[]> DeserializeForces(Stream forces) =>
        liftIO(async () => {
            // This could return null, but I'm not handling that case and should be.
            return await JsonSerializer.DeserializeAsync<Force[]>(
                forces, 
                new JsonSerializerOptions(JsonSerializerDefaults.Web));
        });

    public static IO<Force[]> GetForces() => GetForcesStream().Bind(DeserializeForces);
}

public class Program
{
    static void Main()
    {
        // Type inference complains if I try to perform the Console.Writeline side effect in here, unless 
        // I return `unit` in both cases... which I guess makes sense but also seems a little ugly - so I've 
        // opted to return a string in both cases and perform the side effect later... not sure if this is sensible.
        var res = PoliceData.GetForces()
            .Match(
                result => JsonSerializer.Serialize(result),
                error => error.Message
            )
            .Run();
        
        Console.WriteLine(res); 
    }
}

Maybe I should be doing something like this - but at this point the nested Match call seems a bit silly. If I'm perfectly happy to handle the null in the same way as I'd handle an exception, I don't know how to express this.

public static class PoliceData
{
    private static IO<Stream> GetForcesStream() =>
        liftIO(async () => {
            using var client = new HttpClient();

            var response = await client.GetAsync("https://data.police.uk/api/forces");

            return await response.Content.ReadAsStreamAsync();
        });

    private static IO<Validation<Error, Force[]>> DeserializeForces(Stream forces) =>
        liftIO(async () => {
            var maybeForces = await JsonSerializer.DeserializeAsync<Force[]>(
                forces, 
                new JsonSerializerOptions(JsonSerializerDefaults.Web));

            return maybeForces is null
                ? Fail<Error, Force[]>(Error.New("Forces was null"))
                : Pure(maybeForces);
        });

    public static IO<Validation<Error, Force[]>> GetForces() => 
        GetForcesStream().Bind(DeserializeForces);
}

public class Program
{
    static void Main()
    {
        var res = PoliceData.GetForces()
            .Match(
                result => result
                    .Match(
                        Succ: success => JsonSerializer.Serialize(success),
                        Fail: failure => failure.Message),
                error => error.Message
            )
            .Run();
        
        Console.WriteLine(res); 
    }
}

I guess my questions are:

• Am I on the right track here or is this way off?
• If I'm happy to handle null in the same way as an IO error such that I can chain easily, how should I express this?
• Would it be better to write these methods as pure tasks and wrap them with liftIO elsewhere?
• If I want to call a MethodB that returns IO<something> within a MethodA that uses liftIO, is it considered ok to call MethodB().Run() within MethodA because this will be lazy? Or is it only ever ok to call Run() at application boundaries?

I've left the whole "runtime" concept out of this altogether as I'm trying to take baby steps!

Appreciate these are very beginner level question so thanks in advance for any guidance :)

[louthy]

You're on the right track, but yeah the moment it starts feeling messy, you're likely to be veering off course (or, there's a feature from v5 that I haven't added yet!)

So, the IO<A> type is the 'raw unit of IO' in v5 language-ext. It doesn't have lots of features like Eff<A>, although it is no slouch!

Errors are handled slightly differently too. From within a liftIO your best bet (to raise errors) is to throw an exception. The code within the liftIO is supposed to be the 'messy bit' that we capture an make nice and fluffy by wrapping it in the IO type. Outside of the liftIO (once you have a real IO monad) you have the opportunity to @catch | @catchM | @catchOf the errors and handle them in a more 'civilised' way (see the example below).

By the way, if you create your own error types that derive from Expected or Exceptional then you can also use expected and exceptional to catch errors by type.

The idea is that for (types that support catching) you can use the | operator like series of catch blocks:

    var result = ioOperation 
                  | @catch(Errors.Cancelled, ...)
                  | @catch(Errors.Timeout, ...)

The various overloads allow you to map to:

• Alternative errors (for wrapping inner errors with contextual errors, or 'censoring' errors -- i.e. "there was a problem")
• Values (i.e. ignoring the error and providing a default)
• Alternative monad - an IO monad to run if failed.

They allow matching through Error codes, types, etc.

You can also use Match or IfFail.

Here's my version with the IO monad:

public readonly record struct Force(string Id, string Name);

public static class PoliceData
{
    static IO<Stream> GetForcesStream() =>
        liftIO(async e =>
        {
            using var client = new HttpClient();
            var response = await client.GetAsync("https://data.police.uk/api/forces", e.Token);
            return await response.Content.ReadAsStreamAsync(e.Token);
        });

    static IO<Seq<Force>> DeserializeForces(Stream forces) =>
        from results in liftIO(async e =>
            await JsonSerializer.DeserializeAsync<Force[]>(forces, new JsonSerializerOptions(JsonSerializerDefaults.Web), e.Token)
                switch
                {
                    null => throw new SerializationException(),
                    var x => x
                })
        select toSeq(results);
    
    public static IO<Seq<Force>> GetForces() =>
        from stream in GetForcesStream()
        from forces in DeserializeForces(stream)
        select forces;
}

public static class ConsoleIO
{
    public static IO<Unit> WriteLine(string value) =>
        IO.lift(() =>
        {
            Console.WriteLine(value);
            return unit;
        });
}

public class Program
{
    static void Main() =>
        MainIO.Run();
    
    static readonly IO<Unit> MainIO =
        from result in PoliceData.GetForces().Map(Serialise) | catchOf((Error e) => e.Message)
        from _      in ConsoleIO.WriteLine(result)     
        select unit;          

    static string Serialise<A>(A value) =>
        JsonSerializer.Serialize(value);
}

NOTE: I haven't tested any of this yet (especially the resource tracking), so although this will all work when v5 is released, there may well be bugs.

The alternative is the use the (non-runtime) Eff<A>. The benefit of using Eff is that it does resource tracking. Which means we can break down the IO into smaller components.

So, first I would create a module for Http operations:

public static class Http
{
    /// <summary>
    /// Create a new client
    /// </summary>
    public static Eff<HttpClient> client =>
        Resource.use<Eff, HttpClient>(() => new HttpClient()).As();

    /// <summary>
    /// HTTP GET from a URI
    /// </summary>
    public static Eff<HttpResponseMessage> get(HttpClient client, string uri) =>
        liftIO(e => client.GetAsync(uri, e.Token));

    /// <summary>
    /// Read the response as a stream
    /// </summary>
    public static Eff<Stream> asStream(HttpResponseMessage stream) =>
        liftIO(e => stream.Content.ReadAsStreamAsync(e.Token));
}

Notice the use of Resource.use -- this does the tracking of the disposable resource.

Then do a similar thing for Json:

public static class Json
{
    /// <summary>
    /// Standardise the de-serialisation error to make it easier to match with  
    /// </summary>
    public static Error DeSerialisationError = Error.New(23423, "Deserialisation failed");
    
    /// <summary>
    /// Default options - doesn't need to be split out
    /// </summary>
    private static JsonSerializerOptions defaultOptions = new (JsonSerializerDefaults.Web);
    
    /// <summary>
    /// De-serialise a JSON stream 
    /// </summary>
    public static Eff<A> deserialise<A>(Stream stream) =>
        from value  in liftIO(async e => await JsonSerializer.DeserializeAsync<A>(stream, defaultOptions, e.Token))
        from result in value switch
                       {
                          null  => Fail(DeSerialisationError),
                          var v => SuccessEff(v)
                       }
        select result;

    /// <summary>
    /// Simple wrapper for serialisation - I assume this could throw an exception,
    /// so it should probably be an effect too.  But this is ok for now 
    /// </summary>
    public static string serialise<A>(A value) =>
        JsonSerializer.Serialize(value);
}

By building wrappers for the smallest units of IO (above) it then makes composing them into bigger IO units much more elegant. Here's the PoliceData class:

public static class PoliceData
{
    static Eff<Stream> GetForcesStream() =>
        from c in Http.client
        from r in Http.get(c, "https://data.police.uk/api/forces")
        from s in Http.asStream(r)
        from _ in Resource.release<Eff, HttpClient>(c)
        select s;

    static Eff<Seq<Force>> DeserializeForces(Stream forces) =>
        Json.deserialise<Force[]>(forces).Map(toSeq);
    
    public static Eff<Seq<Force>> GetForces() =>
        from stream in GetForcesStream()
        from forces in DeserializeForces(stream)
        select forces;
}

I'd challenge anybody to say that's not elegant ;-)

Then we just need to slightly modify the Program:

public class Program
{
    static void Main() =>
        MainEff.Run().ThrowIfFail();

    static readonly Eff<Unit> MainEff =
        from result in PoliceData.GetForces().Map(Json.serialise) | catchOf((Error e) => e.Message)
        from _      in ConsoleIO.WriteLine(result)     
        select unit;          
}

The big idea is to capture those pieces of messy IO at their smallest unit and 'control' them by making them into well-behaved IO or Eff types. They can then be combined to make more complex stuff. Of course, it's possible to put large blocks of code inside a liftIO, but you're back in imperative-land and that's where the errors are and where it's harder to reason -- although sometimes it can be more efficient to lift larger blocks into a single IO operation; but that's the kind of trade-off you worry about later.

[drk-mtr]

This is incredibly useful - and very clearly explained.

It's also helped a few other concepts (setting up standard errors, resource tracking, passing of tokens, the use of .As() etc.) to start to click in to place for me.

I'd challenge anybody to say that's not elegant ;-)

Indeed!

Thanks very much for taking the time to help. And thanks for the brilliant library, really excited with the direction of movement for v5.

Time to have a play :)

[louthy]

By the way. The other thing you mentioned around IO<Option<A>> and how to chain operations. That is what the new monad-transformers are for. I have only just got to monads on the blog series about the new types, so there's not any documentation/information yet. But, if you search online for how to do monad-transformers in Haskell, it's pretty much exactly the same!

But there are several monad transformers (types that end with T), like OptionT, EitherT, etc. that compose with other monads. If you want an Option<IO<A>> instead use OptionT<IO, A>. It augments IO with Option semantics.

If you look at the implementation of Eff<RT, A> you'll see it is many monad-transformers stacked:

public record Eff<RT, A>(StateT<RT, ResourceT<IO>, A> effect) : K<Eff<RT>, A>

• The state monad for managing the runtime
• The resource monad for tracking resources
• and the IO monad for doing IO side-effecting operations

This is definitely a more advanced level up and probably one you don't need (as both IO and Eff can handle an alternative Error value); but it's good to realise that this is how monads compose. You can't use different monadic types in the same LINQ expression, you must use a composed type that has all of the features you need. This is part of the declarative process of specifying what any one expression does.