aboutsummaryrefslogtreecommitdiff
path: root/src/Text/Pandoc/Filter.hs
AgeCommit message (Collapse)AuthorFilesLines
2021-10-27Switch back from HsYAML to yaml.John MacFarlane1-8/+12
Reasons: - Performance: HsYAML is around 20 times slower in parsing large YAML bibliographies (#6084). - An issue was submitted to HsYAML, but it hasn't gotten any attention. HsYAML seems borderline unmaintained; it hasn't had a commit in over a year. - Unfortunately this goes back on our attempts to free ourselves from C dependencies (#4535). But I don't see a better alternative until a better pure Haskell parser is available. Closes #6084. Notes: - We've removed the FromYAML instances for all types that had them, since this is a HsYAML-specific typeclass [API change]. (The yaml package just uses From/ToJSON.) - Unlike HsYAML (in the configuration we were using), yaml parses 'Y', 'N', 'Yes', 'No', 'On', 'Off' as boolean values. Users may need to quote these when they are meant to be interpreted as strings. Similarly, 'null' is parsed as a YAML null value (and will be treated as an empty string by pandoc rather than the string 'null'). Quoting it will force it to be interpreted as a string. - Some tests had to be adjusted accordingly. - Pandoc now behaves better when the YAML metadata contains escaping errors: instead of just falling back on treating the section as a table, it raises a YAML parsing error.
2021-08-24Text.Pandoc.Filter: Generalize type of applyFilters...John MacFarlane1-5/+5
from PandocIO to any instance of MonadIO and PandocMonad. [API change]
2021-01-08Update copyright notices for 2021 (#7012)Albert Krewinkel1-1/+1
2020-11-02T.P.Filter: allow shorter YAML representation of CiteprocAlbert Krewinkel1-3/+5
The map-based YAML representation of filters expects `type` and `path` fields. The path field had to be present for all filter types, but is not used for citeproc filters. The field can now be omitted when type is "citeproc", as described in the MANUAL.
2020-09-21Add built-in citation support using new citeproc library.John MacFarlane1-3/+12
This deprecates the use of the external pandoc-citeproc filter; citation processing is now built in to pandoc. * Add dependency on citeproc library. * Add Text.Pandoc.Citeproc module (and some associated unexported modules under Text.Pandoc.Citeproc). Exports `processCitations`. [API change] * Add data files needed for Text.Pandoc.Citeproc: default.csl in the data directory, and a citeproc directory that is just used at compile-time. Note that we've added file-embed as a mandatory rather than a conditional depedency, because of the biblatex localization files. We might eventually want to use readDataFile for this, but it would take some code reorganization. * Text.Pandoc.Loging: Add `CiteprocWarning` to `LogMessage` and use it in `processCitations`. [API change] * Add tests from the pandoc-citeproc package as command tests (including some tests pandoc-citeproc did not pass). * Remove instructions for building pandoc-citeproc from CI and release binary build instructions. We will no longer distribute pandoc-citeproc. * Markdown reader: tweak abbreviation support. Don't insert a nonbreaking space after a potential abbreviation if it comes right before a note or citation. This messes up several things, including citeproc's moving of note citations. * Add `csljson` as and input and output format. This allows pandoc to convert between `csljson` and other bibliography formats, and to generate formatted versions of CSL JSON bibliographies. * Add module Text.Pandoc.Writers.CslJson, exporting `writeCslJson`. [API change] * Add module Text.Pandoc.Readers.CslJson, exporting `readCslJson`. [API change] * Added `bibtex`, `biblatex` as input formats. This allows pandoc to convert between BibLaTeX and BibTeX and other bibliography formats, and to generated formatted versions of BibTeX/BibLaTeX bibliographies. * Add module Text.Pandoc.Readers.BibTeX, exporting `readBibTeX` and `readBibLaTeX`. [API change] * Make "standalone" implicit if output format is a bibliography format. This is needed because pandoc readers for bibliography formats put the bibliographic information in the `references` field of metadata; and unless standalone is specified, metadata gets ignored. (TODO: This needs improvement. We should trigger standalone for the reader when the input format is bibliographic, and for the writer when the output format is markdown.) * Carry over `citationNoteNum` to `citationNoteNumber`. This was just ignored in pandoc-citeproc. * Text.Pandoc.Filter: Add `CiteprocFilter` constructor to Filter. [API change] This runs the processCitations transformation. We need to treat it like a filter so it can be placed in the sequence of filter runs (after some, before others). In FromYAML, this is parsed from `citeproc` or `{type: citeproc}`, so this special filter may be specified either way in a defaults file (or by `citeproc: true`, though this gives no control of positioning relative to other filters). TODO: we need to add something to the manual section on defaults files for this. * Add deprecation warning if `upandoc-citeproc` filter is used. * Add `--citeproc/-C` option to trigger citation processing. This behaves like a filter and will be positioned relative to filters as they appear on the command line. * Rewrote the manual on citatations, adding a dedicated Citations section which also includes some information formerly found in the pandoc-citeproc man page. * Look for CSL styles in the `csl` subdirectory of the pandoc user data directory. This changes the old pandoc-citeproc behavior, which looked in `~/.csl`. Users can simply symlink `~/.csl` to the `csl` subdirectory of their pandoc user data directory if they want the old behavior. * Add support for CSL bibliography entry formatting to LaTeX, HTML, Ms writers. Added CSL-related CSS to styles.html.
2020-03-22Finer grained imports of Text.Pandoc.Class submodules (#6203)Albert Krewinkel1-1/+2
This should speed-up recompilation after changes in `Text.Pandoc.Class`, as the number of modules affected by a change will be smaller in general. It also offers faster insights into the parts of `T.P.Class` used within a module.
2020-03-15Use implicit Prelude (#6187)Albert Krewinkel1-2/+0
* Use implicit Prelude The previous behavior was introduced as a fix for #4464. It seems that this change alone did not fix the issue, and `stack ghci` and `cabal repl` only work with GHC 8.4.1 or newer, as no custom Prelude is loaded for these versions. Given this, it seems cleaner to revert to the implicit Prelude. * PandocMonad: remove outdated check for base version Only base versions 4.9 and later are supported, the check for `MIN_VERSION_base(4,8,0)` is therefore unnecessary. * Always use custom prelude Previously, the custom prelude was used only with older GHC versions, as a workaround for problems with ghci. The ghci problems are resolved by replacing package `base` with `base-noprelude`, allowing for consistent use of the custom prelude across all GHC versions.
2020-03-13Update copyright year (#6186)Albert Krewinkel1-1/+1
* Update copyright year * Copyright: add notes for Lua and Jira modules
2020-02-13A bit more cleanup (#6141)Joseph C. Sible1-3/+3
* Remove unnecessary fmaps and only do toMilliseconds once * Share the input tuple intead of making a new one * Lift return out of if * Simplify case statements * Lift DottedNum out of the case statements * Use st instead of mbs * Use setState instead of updateState now that we have the whole state around
2020-02-04Add timing info for filters in `--verbose` mode.John MacFarlane1-4/+18
+ Add RunningFilter, FilterCompleted constructors to LogMessage + When verbose mode is specified (verbosity == INFO), print a notice when running a filter and when a filter completes (including timing). Closes #6112.
2019-11-06Add and apply filters in order (not reversed).John MacFarlane1-4/+4
This changes `applyFilters` from Text.Pandoc.Filter so that it does a left fold rather than a right fold, applying the filters in the order listed. [behavior change] The command-line arguments are accumulated in order instead of reverse order. A first step twoards #5881.
2019-10-10Improve parsing of --defaults.John MacFarlane1-0/+21
- Add FromYAML instances to Opt and to all subsidiary types. - Remove the use of HsYAML-aeson, which doesn't give good position information on errors. - Rename some fields in Opt to better match cli options or reflect what the ycontain [API change]: + optMetadataFile -> optMetadataFiles + optPDFEngineArgs -> optPDFEngineOpts + optWrapText -> optWrap - Add IpynbOutput enumerated type to Text.Pandoc.App.Opts. Use this instead fo a string for optIpynbOutput. - Add FromYAML instance for Filter in Text.Pandoc.Filters. With these changes parsing of defaults files should be complete and should give decent error messages. Now (unlike before) we get an error if an unknown field is used.
2019-10-07Remove derive_json_via_th flag; always use TH.John MacFarlane1-13/+0
This cuts down on code duplication and reduces the chance for errors. See #4083.
2019-03-01Remove license boilerplate.John MacFarlane1-17/+0
The haddock module header contains essentially the same information, so the boilerplate is redundant and just one more thing to get out of sync.
2019-02-04Add missing copyright notices and remove license boilerplate (#5112)Albert Krewinkel1-2/+2
Quite a few modules were missing copyright notices. This commit adds copyright notices everywhere via haddock module headers. The old license boilerplate comment is redundant with this and has been removed. Update copyright years to 2019. Closes #4592.
2018-11-11Fix CPP conditional for TH pragmaAlbert Krewinkel1-1/+1
The condition was from an earlier version.
2018-11-05Use DERIVE_JSON_VIA_TH in Text.Pandoc.Filter.John MacFarlane1-6/+6
Previously AVOID_TEMPLATE_HASKELL was used (with the opposite valence). I assume this was a leftover from an earlier version.
2018-11-04Add cabal flag `derive_json_via_th`Albert Krewinkel1-3/+18
Disabling the flag will cause derivation of ToJSON and FromJSON instances via GHC Generics instead of Template Haskell. The flag is enabled by default, as deriving via Generics can be slow (see #4083).
2018-10-28T.P.Filter: centralize filter path expansionAlbert Krewinkel1-2/+11
2018-10-28T.P.Lua: merge runLuaFilter into T.P.Filter.Lua (API change)Albert Krewinkel1-4/+3
The function `runLuaFilter` was only used in Text.Pandoc.Filter.Lua, use apply from the that module instead.
2018-03-18Use NoImplicitPrelude and explicitly import Prelude.John MacFarlane1-0/+2
This seems to be necessary if we are to use our custom Prelude with ghci. Closes #4464.
2018-01-19hlint code improvements.John MacFarlane1-1/+1
2018-01-15Renaming: Json -> JSON in modules and functions.John MacFarlane1-2/+2
2018-01-13Move filter functions to separate moduleAlbert Krewinkel1-0/+60