Parse a large text file using Node.js streams - part 2

, , , , ,

Have a look at the part 1 for the necessary context. Here we go.

Question: count the name occurrences

We need to identify the most common first name in the data and how many times it occurs.
Map/reduce is convenient here, the name is in the column 8, so we isolate the column than accumulate te occurrences in an object. 

1
2
3
4
5
6
7
8
9
10
11
12
13
const occurrences = {}
const namesOccurences = () => {
  __(generator)
    .map(line => line.split('|')[8])
    .reduce(0, (a, name) => {
      occurrences[name] = (occurrences[name] || 0) + 1
      return name
    })
    .done(() => {
      console.log('Highland> names occurrences:')
      console.log(occurrences)
    })
}

Question: donations per month

Count how many donations occurred in each month and print out the results. The donation amount is in the column 5.

This time we extract the reduce function for more clarity:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
const accumulator = {}
const sumByMonth = function (a, item) {
  accumulator[item.month] = (accumulator[item.month] || 0) + 1
  return item
}

const countDonationsPerMonth = () => {
  __(generator)
    .map(line => { return {month: line.split('|')[4].substring(4, 6), line: line,}})
    .reduce(0, sumByMonth)
    .done(() => {
      console.log('Highland> donations per month:')
      console.log(accumulator)
    })
}

Considerations

The hardest part of this 2 questions was building the reduce function, which requires some research on Stackoverflow serious knowledge.

The rest is easy, clear, readable, reliable and fast thanks to Streams and Highland.

Conclusions.

A complete working example is in this Github repo.

Would love feedback.
Is the topic interesting? The solution nice? Would you like to read more about Streams or Highland? Or how to avoid lodash and use EcmaScript properly?

I do not have a comment section here but Twitter is perfect.

Have a lovely day.