Add built-in citation support using new citeproc library.

This deprecates the use of the external pandoc-citeproc
filter; citation processing is now built in to pandoc.

* Add dependency on citeproc library.
* Add Text.Pandoc.Citeproc module (and some associated unexported
  modules under Text.Pandoc.Citeproc).  Exports `processCitations`.
  [API change]
* Add data files needed for Text.Pandoc.Citeproc:  default.csl
  in the data directory, and a citeproc directory that is just
  used at compile-time.  Note that we've added file-embed as a mandatory
  rather than a conditional depedency, because of the biblatex
  localization files. We might eventually want to use readDataFile
  for this, but it would take some code reorganization.
* Text.Pandoc.Loging: Add `CiteprocWarning` to `LogMessage` and use it
  in `processCitations`. [API change]
* Add tests from the pandoc-citeproc package as command tests (including
  some tests pandoc-citeproc did not pass).
* Remove instructions for building pandoc-citeproc from CI and
  release binary build instructions.  We will no longer distribute
  pandoc-citeproc.
* Markdown reader: tweak abbreviation support.  Don't insert a
  nonbreaking space after a potential abbreviation if it comes right before
  a note or citation.  This messes up several things, including citeproc's
  moving of note citations.
* Add `csljson` as and input and output format. This allows pandoc
  to convert between `csljson` and other bibliography formats,
  and to generate formatted versions of CSL JSON bibliographies.
* Add module Text.Pandoc.Writers.CslJson, exporting `writeCslJson`. [API
  change]
* Add module Text.Pandoc.Readers.CslJson, exporting `readCslJson`. [API
  change]
* Added `bibtex`, `biblatex` as input formats.  This allows pandoc
  to convert between BibLaTeX and BibTeX and other bibliography formats,
  and to generated formatted versions of BibTeX/BibLaTeX bibliographies.
* Add module Text.Pandoc.Readers.BibTeX, exporting `readBibTeX` and
  `readBibLaTeX`. [API change]
* Make "standalone" implicit if output format is a bibliography format.
  This is needed because pandoc readers for bibliography formats put
  the bibliographic information in the `references` field of metadata;
  and unless standalone is specified, metadata gets ignored.
  (TODO: This needs improvement. We should trigger standalone for the
  reader when the input format is bibliographic, and for the writer
  when the output format is markdown.)
* Carry over `citationNoteNum` to `citationNoteNumber`.  This was just
  ignored in pandoc-citeproc.
* Text.Pandoc.Filter: Add `CiteprocFilter` constructor to Filter.
  [API change] This runs the processCitations transformation.
  We need to treat it like a filter so it can be placed
  in the sequence of filter runs (after some, before others).
  In FromYAML, this is parsed from `citeproc` or `{type: citeproc}`,
  so this special filter may be specified either way in a defaults file
  (or by `citeproc: true`, though this gives no control of positioning
  relative to other filters).  TODO: we need to add something to the
  manual section on defaults files for this.
* Add deprecation warning if `upandoc-citeproc` filter is used.
* Add `--citeproc/-C` option to trigger citation processing.
  This behaves like a filter and will be positioned
  relative to filters as they appear on the command line.
* Rewrote the manual on citatations, adding a dedicated Citations
  section which also includes some information formerly found in
  the pandoc-citeproc man page.
* Look for CSL styles in the `csl` subdirectory of the pandoc user data
  directory.  This changes the old pandoc-citeproc behavior, which looked
  in `~/.csl`.  Users can simply symlink `~/.csl` to the `csl`
  subdirectory of their pandoc user data directory if they want
  the old behavior.
* Add support for CSL bibliography entry formatting to LaTeX, HTML,
  Ms writers.  Added CSL-related CSS to styles.html.

2 files changed, 9 insertions, 12 deletions

diff --git a/.github/workflows/nightly.yml b/.github/workflows/nightly.yml
index 4f6b2f68b..66e9857a8 100644
--- a/.github/workflows/nightly.yml
+++ b/.github/workflows/nightly.yml
@@ -18,19 +18,17 @@ jobs:
           ghc --version
           cabal --version
           cabal v2-update
-          cabal v2-build --dependencies-only . pandoc-citeproc
+          cabal v2-build --dependencies-only all
     - name: Build
       run: |
           export PATH=/opt/cabal/3.2/bin:/opt/ghc/8.8.3/bin:$PATH
-          cabal v2-install . pandoc-citeproc
+          cabal v2-install all
           strip $HOME/.cabal/bin/pandoc
-          strip $HOME/.cabal/bin/pandoc-citeproc
     - name: Install artifact
       run: |
           export ARTIFACTS=nightly-linux/pandoc-nightly-linux-$(date +%Y-%m-%d)
           mkdir -p ${ARTIFACTS}
           cp $HOME/.cabal/bin/pandoc ${ARTIFACTS}/
-          cp $HOME/.cabal/bin/pandoc-citeproc ${ARTIFACTS}/
           cp COPYRIGHT ${ARTIFACTS}/
           echo "Built from ${GITHUB_SHA}" > ${ARTIFACTS}/README.nightly.txt
     - uses: actions/upload-artifact@master
@@ -51,14 +49,14 @@ jobs:
     - name: Install dependencies
       run: |
           stack update
-          stack build --dependencies-only pandoc pandoc-citeproc
+          stack build --dependencies-only pandoc
     - name: Build artifacts
       shell: cmd
       run: |
           for /f %%a in ('powershell -Command "Get-Date -format yyyy-MM-dd"') do set THEDATE=%%a
           set ARTIFACTS=%CD%\nightly-windows\pandoc-nightly-windows-%THEDATE%
           mkdir %ARTIFACTS%
-          stack build pandoc pandoc-citeproc
+          stack build pandoc
           forfiles /P .\.stack-work\install /M pandoc*.exe /S /C "cmd /C copy @path %%ARTIFACTS%%"
           copy COPYRIGHT %ARTIFACTS%
           ren %ARTIFACTS%\COPYRIGHT COPYRIGHT.txt
@@ -81,15 +79,14 @@ jobs:
     - name: Install dependencies
       run: |
           ./stack update
-          ./stack build --dependencies-only pandoc pandoc-citeproc
+          ./stack build --dependencies-only pandoc
     - name: Build artifacts
       run: |
           export ARTIFACTS=nightly-macos/pandoc-nightly-macos-$(date +%Y-%m-%d)
           mkdir -p ${ARTIFACTS}
-          ./stack build pandoc pandoc-citeproc
+          ./stack build pandoc
           for f in $(find .stack-work/install -name 'pandoc*' -perm +001 -type f); do cp $f ${ARTIFACTS}/; done
           mv ${ARTIFACTS}/pandoc ${ARTIFACTS}/pandoc
-          mv ${ARTIFACTS}/pandoc-citeproc ${ARTIFACTS}/pandoc-citeproc
           cp COPYRIGHT ${ARTIFACTS}/
           echo "Built from ${GITHUB_SHA}" > ${ARTIFACTS}/README.nightly.txt
     - uses: actions/upload-artifact@master
diff --git a/.github/workflows/release-candidate.yml b/.github/workflows/release-candidate.yml
index f71d71b8b..e4d2fffe9 100644
--- a/.github/workflows/release-candidate.yml
+++ b/.github/workflows/release-candidate.yml
@@ -48,12 +48,12 @@ jobs:
     - name: Install dependencies
       run: |
           stack update
-          stack build ${{ matrix.versions.stack_opts }} --dependencies-only pandoc pandoc-citeproc
+          stack build ${{ matrix.versions.stack_opts }} --dependencies-only pandoc
     - name: Create windows package
       shell: cmd
       run: |
           for /f %%a in ('powershell -Command "Get-Date -format yyyy-MM-dd"') do set THEDATE=%%a
-          stack build  ${{ matrix.versions.stack_opts }} pandoc pandoc-citeproc
+          stack build  ${{ matrix.versions.stack_opts }} pandoc
           for /f "tokens=1-2 delims= " %%a in ('stack exec pandoc -- --version') do (
             @set VERSION=%%b
             goto :next
@@ -67,7 +67,7 @@ jobs:
           @set WINDOWS=%CD%\windows
           @set RELEASE=%WINDOWS%\pandoc-%VERSION%
           mkdir %RELEASE%
-          forfiles /P .\.stack-work\install /M pandoc*.exe /S /C "cmd /C echo @path && echo %%WINDOWS%% && copy @path %%RELEASE%%"
+          forfiles /P .\.stack-work\install /M pandoc.exe /S /C "cmd /C echo @path && echo %%WINDOWS%% && copy @path %%RELEASE%%"
           stack exec pandoc -- -s --toc MANUAL.txt -o %RELEASE%\MANUAL.html
           stack exec pandoc -- -s COPYING.md -t rtf -o %RELEASE%\COPYING.rtf
           copy %RELEASE%\COPYING.rtf %WINDOWS%