Lines Tokenizers

The tokenizers/lines module gathers the library’s various lines tokenizers aiming at splitting raw text into lists of lines.

Summary

• naive

naive

Naive function using a simple regular expression to split raw text into lines.

import lines from 'talisman/tokenizers/lines';

lines('First line.\nSecond line.');
>>> [
  'First line.',
  'Second line.'
]