zoukankan      html  css  js  c++  java
  • [Functional Programming] mapReduce over Async operations and fanout results in Pair(rejected, resolved) (fanout, flip, mapReduce)

    This post is similar to previous post. The difference is in this post, we are going to see how to handle both successfuly result and error result by using Pair functor.

    So, still we have our funs.js: which is the same as previous post.

    const fs = require('fs');
    const {Async, constant, composeK, curry} = require('crocks');
    const {fromNode} = Async;
    
    const access = fromNode(fs.access);
    const readFile = fromNode(fs.readFile);
    
    const accessAsync = curry((mode, path) =>
      access(path, mode)
      .map(constant(path)));
    
    // readFileAsync :: Option -> a -> Async Error b
    const readFileAsync = curry((option, path) =>
        readFile(path, option));
    
    const checkRead = accessAsync(fs.constants.F_OK);
    const readTextFile = readFileAsync('utf-8');
    
    // loadTextFile :: String -> Async Error String
    const loadTextFile = composeK(
        readTextFile,
        checkRead
    );
    
    const fork = a => a.fork(
        console.log.bind(null, 'rej'),
        console.log.bind(null, 'res')
    );
    
    module.exports = {
        loadTextFile,
        fork
    }

    For our main.js, we still have the same data input:

    const data = [
        'text.txt',
        'text.big.txt',
        'notfound.txt',
    ];

    This time the difference of requirements are:

    1. we want to read those files one by one, keep all the successfully results in Pair(result, _);

    2. we want to keep the error result in Pair(_, error);

    const concatSpecial = (acc, currAsync) =>
        acc.chain(
            xs => currAsync.bimap(
                e => Pair(xs, e),
                currVal =>  xs.concat(currVal))
        );
    
    // Async (Pair [String] Error) [String]
    const flow = mapReduce(
        loadTextFile,
        concatSpecial,
        Async.Resolved([])
    );
    
    flow(data).fork(
        e => console.log(e.snd(), e.fst()), // Pair(success, error)
        r => console.log(r), // Just success result
    )

    We are still using 'mapRedcue' to map over each filename, fetching the content; then we call 'concatSpecial' method, we want to concat all the successful result into one array. Therefore we give an empty array wrapped in Async:

    const flow = mapReduce(
        loadTextFile,
        concatSpecial,
        Async.Resolved([])
    );

    We can do some pointfree refactor for 'concatSpical', it's not necssary, but just as a partice:

    const fn = flip(
        xs => bimap(
            e => Pair(xs, e),
            currVal =>  xs.concat(currVal)
        )
    );
    
    const concatSpecial = (acc, currAsync) =>
        acc.chain(
            fn(currAsync)
        );

    For the function 'fn', we should take 'xs' as first param, then 'currAsync' as second param. 

    But since we also pass in 'currAsync' as first param, then we need to use 'flip':

    acc.chain(
        fn(currAsync) // pass currAsync as firt, then xs => fn(currAsync)(xs)
    );

    We can also replace 'Pair' with 'fanout':

    const fn = flip(
        xs => bimap(
            fanout(constant(xs), identity),
            currVal =>  xs.concat(currVal)
        )
    );

    ---

    Full code:

    const {fork, loadTextFile} = require('./funs.js');
    const {Async, bimap, fanout, constant, flip, Pair, identity, mapReduce} = require('crocks');
    
    const data = [
        'text.txt',
        'text.big.txt',
        'notfound.txt',
    ];
    
    const fn = flip(
        xs => bimap(
            e => Pair(xs, e),
            fanout(constant(xs), identity),
            currVal =>  xs.concat(currVal)
        )
    );
    /*
    const concatSpecial = (acc, currAsync) =>
        acc.chain(
            xs => currAsync.bimap(
                e => Pair(xs, e),
                currVal =>  xs.concat(currVal))
        );*/
    const concatSpecial = (acc, currAsync) =>
        acc.chain(
            fn(currAsync)
        );
    // Async (Pair [String] Error) [String]
    const flow = mapReduce(
        loadTextFile,
        concatSpecial,
        Async.Resolved([])
    );
    
    flow(data).fork(
        e => console.log(e.snd(), e.fst()), // Pair(success, error)
        r => console.log(r), // Just success result
    )
  • 相关阅读:
    一些数据集
    经典的图像匹配算法----SIFT
    LDA处理文档主题分布代码
    Dirichlet Process
    主题模型-LDA浅析
    matplotlib —— 添加文本信息(text)
    xgboost原理及应用
    XGBoost参数调优
    MySQL中实现Oracle里面 rank()over ( PARTITION BY ORDER BY) 分类分组功能
    4.12 省选模拟赛 LCA on tree 树链剖分 树状数组 分析答案变化量
  • 原文地址:https://www.cnblogs.com/Answer1215/p/10507231.html
Copyright © 2011-2022 走看看