Add built-in citation support using new citeproc library.

This deprecates the use of the external pandoc-citeproc
filter; citation processing is now built in to pandoc.

* Add dependency on citeproc library.
* Add Text.Pandoc.Citeproc module (and some associated unexported
  modules under Text.Pandoc.Citeproc).  Exports `processCitations`.
  [API change]
* Add data files needed for Text.Pandoc.Citeproc:  default.csl
  in the data directory, and a citeproc directory that is just
  used at compile-time.  Note that we've added file-embed as a mandatory
  rather than a conditional depedency, because of the biblatex
  localization files. We might eventually want to use readDataFile
  for this, but it would take some code reorganization.
* Text.Pandoc.Loging: Add `CiteprocWarning` to `LogMessage` and use it
  in `processCitations`. [API change]
* Add tests from the pandoc-citeproc package as command tests (including
  some tests pandoc-citeproc did not pass).
* Remove instructions for building pandoc-citeproc from CI and
  release binary build instructions.  We will no longer distribute
  pandoc-citeproc.
* Markdown reader: tweak abbreviation support.  Don't insert a
  nonbreaking space after a potential abbreviation if it comes right before
  a note or citation.  This messes up several things, including citeproc's
  moving of note citations.
* Add `csljson` as and input and output format. This allows pandoc
  to convert between `csljson` and other bibliography formats,
  and to generate formatted versions of CSL JSON bibliographies.
* Add module Text.Pandoc.Writers.CslJson, exporting `writeCslJson`. [API
  change]
* Add module Text.Pandoc.Readers.CslJson, exporting `readCslJson`. [API
  change]
* Added `bibtex`, `biblatex` as input formats.  This allows pandoc
  to convert between BibLaTeX and BibTeX and other bibliography formats,
  and to generated formatted versions of BibTeX/BibLaTeX bibliographies.
* Add module Text.Pandoc.Readers.BibTeX, exporting `readBibTeX` and
  `readBibLaTeX`. [API change]
* Make "standalone" implicit if output format is a bibliography format.
  This is needed because pandoc readers for bibliography formats put
  the bibliographic information in the `references` field of metadata;
  and unless standalone is specified, metadata gets ignored.
  (TODO: This needs improvement. We should trigger standalone for the
  reader when the input format is bibliographic, and for the writer
  when the output format is markdown.)
* Carry over `citationNoteNum` to `citationNoteNumber`.  This was just
  ignored in pandoc-citeproc.
* Text.Pandoc.Filter: Add `CiteprocFilter` constructor to Filter.
  [API change] This runs the processCitations transformation.
  We need to treat it like a filter so it can be placed
  in the sequence of filter runs (after some, before others).
  In FromYAML, this is parsed from `citeproc` or `{type: citeproc}`,
  so this special filter may be specified either way in a defaults file
  (or by `citeproc: true`, though this gives no control of positioning
  relative to other filters).  TODO: we need to add something to the
  manual section on defaults files for this.
* Add deprecation warning if `upandoc-citeproc` filter is used.
* Add `--citeproc/-C` option to trigger citation processing.
  This behaves like a filter and will be positioned
  relative to filters as they appear on the command line.
* Rewrote the manual on citatations, adding a dedicated Citations
  section which also includes some information formerly found in
  the pandoc-citeproc man page.
* Look for CSL styles in the `csl` subdirectory of the pandoc user data
  directory.  This changes the old pandoc-citeproc behavior, which looked
  in `~/.csl`.  Users can simply symlink `~/.csl` to the `csl`
  subdirectory of their pandoc user data directory if they want
  the old behavior.
* Add support for CSL bibliography entry formatting to LaTeX, HTML,
  Ms writers.  Added CSL-related CSS to styles.html.

1 files changed, 73 insertions, 0 deletions

diff --git a/test/command/biblatex-test-case-conversion.md b/test/command/biblatex-test-case-conversion.md
new file mode 100644
index 000000000..7e10ebf60
--- /dev/null
+++ b/test/command/biblatex-test-case-conversion.md
@@ -0,0 +1,73 @@
+```
+% pandoc -f biblatex -t markdown -s
+@comment{
+    - bibtex and biblatex
+        - expect titles in title case
+        - styles use titles as is, or convert them to sentence case
+        - strings wrapped {} are not converted
+    - all CSL styles at <http://github.com/citation-style-language/styles>
+      and <https://www.zotero.org/styles/>
+        - expect titles in sentence case
+        - styles use titles as is, or convert them to title case
+        - except for (hardcoded) list of stop words, see
+          <http://citationstyles.org/downloads/specification.html#title-case-conversion>
+        - citeproc-js (MLZ only?) also recognizes a markup syntax for
+          suppressing title-case changes on a range of text (see
+          <https://forums.zotero.org/discussion/21991/excessive-capitalization-of-citation-titles/#Item_22>):
+            - `<span class="nocase"/>lowercase</span>`
+    - Proposal:
+        - When converting to yaml, convert English titles to sentence case,
+            - for all strings wrapped in {} where {} is not part of a latex
+              command, ...
+            - ... when starting with an uppercase letter: suppress
+conversion, remove the {}
+            - ... when starting with a lowercase letter ("nm", "iPod"):
+              suppress conversion, replace the {} with
+              <span class="nocase"/></span>
+            - Note: Camel case ("iPod") needs to be protected in
+              bibtex/biblatex anyway; the only "extension" (wrt bibtex/biblatex
+              specs) we'd be introducing is wrapping lowercase-only strings in
+              {}, something that is never necessary on the latex side but
+              won't break anything there either.
+        - citeproc-hs/pandoc-citeproc should be modified to honour this new
+          syntax and suppress conversion to title case for strings wrapped
+          in `<span class="nocase"/></span>`.
+        - Expected output, using one of the title-case CSL styles, here
+          chicago-author-date.csl:
+
+            Author, Ann. 2013. “A Title, in English, with a Proper Name and
+an
+            ACRONYM and a camelCase Word and Some Units, 400 nm, 3 cm, and
+a Quote,
+            *Alea iacta est*.” *Journal*.
+    }
+
+@article{item1,
+    Author = {Author, Ann},
+    Date = {2013},
+    Hyphenation = {english},
+    Journaltitle = {Journal},
+    Title = {A Title, in {English}, with a {Proper Name} and an {ACRONYM}
+and a {camelCase} Word and Some Units, 400~{nm}, 3~{cm}, and a Quote,
+\textit{{Alea} {iacta est}}}
+}
+
+^D
+---
+nocite: '[@*]'
+references:
+- author:
+  - family: Author
+    given: Ann
+  container-title: Journal
+  id: item1
+  issued: 2013
+  language: 'en-US'
+  title: 'A title, in English, with a Proper Name and an ACRONYM and a
+    [camelCase]{.nocase} word and some units, 400 [nm]{.nocase},
+    3 [cm]{.nocase}, and a quote, *Alea [iacta est]{.nocase}*'
+  type: 'article-journal'
+---
+
+
+```