diff options
| author | John MacFarlane <jgm@berkeley.edu> | 2021-02-28 22:51:26 -0800 |
|---|---|---|
| committer | John MacFarlane <jgm@berkeley.edu> | 2021-02-28 22:53:42 -0800 |
| commit | 7e38b8e55a49f027b6aaa690d84225b0374eb057 (patch) | |
| tree | 4e2111bf82a17b74381da5aadb02a18c363cb2d9 /src/Text | |
| parent | 2463fbf61d2ea8636e70c44624dc5bc1668fa4fd (diff) | |
| download | pandoc-7e38b8e55a49f027b6aaa690d84225b0374eb057.tar.gz | |
T.P.Readers.LaTeX: Don't export tokenize, untokenize.
[API change]
These were only exported for testing, which seems the
wrong thing to do. They don't belong in the public
API and are not really usable as they are, without access
to the Tok type which is not exported.
Removed the tokenize/untokenize roundtrip test.
We put a quickcheck property in the comments which
may be used when this code is touched (if it is).
Diffstat (limited to 'src/Text')
| -rw-r--r-- | src/Text/Pandoc/Readers/LaTeX.hs | 2 | ||||
| -rw-r--r-- | src/Text/Pandoc/Readers/LaTeX/Parsing.hs | 9 |
2 files changed, 9 insertions, 2 deletions
diff --git a/src/Text/Pandoc/Readers/LaTeX.hs b/src/Text/Pandoc/Readers/LaTeX.hs index 772263578..e63fbc185 100644 --- a/src/Text/Pandoc/Readers/LaTeX.hs +++ b/src/Text/Pandoc/Readers/LaTeX.hs @@ -22,8 +22,6 @@ module Text.Pandoc.Readers.LaTeX ( readLaTeX, rawLaTeXInline, rawLaTeXBlock, inlineCommand, - tokenize, - untokenize ) where import Control.Applicative (many, optional, (<|>)) diff --git a/src/Text/Pandoc/Readers/LaTeX/Parsing.hs b/src/Text/Pandoc/Readers/LaTeX/Parsing.hs index a5a39d3c9..db58b333d 100644 --- a/src/Text/Pandoc/Readers/LaTeX/Parsing.hs +++ b/src/Text/Pandoc/Readers/LaTeX/Parsing.hs @@ -292,6 +292,15 @@ applyMacros s = (guardDisabled Ext_latex_macros >> return s) <|> Left e -> Prelude.fail (show e) Right s' -> return s' +{- +When tokenize or untokenize change, test with this +QuickCheck property: + +> tokUntokRoundtrip :: String -> Bool +> tokUntokRoundtrip s = +> let t = T.pack s in untokenize (tokenize "random" t) == t +-} + tokenize :: SourceName -> Text -> [Tok] tokenize sourcename = totoks (initialPos sourcename) |